This paper presents an edge-based smart border surveillance system integrating multi-modal sensing with lightweight deep learning for real-time intrusion detection. The system combines a Raspberry Pi 5, PIR motion sensor, KY-037 acoustic sensor, and NoIR camera in an event-driven architecture. Motion or abnormal sound triggers visual analysis using a TensorFlow Lite–optimized YOLOv5 model deployed for on-device inference. Experimental evaluation across 10 controlled scenarios under daytime and low-light conditions achieved an overall detection accuracy of 80%, with precision and recall of 0.89 for human and vehicle detection. The measured end-to-end latency ranged from 1.6–1.9 s. Average CPU utilization during inference was 55–60%, with peak usage of 72%, and total power consumption measured 6–8 W during active operation. The decision-level sensor fusion approach reduced unnecessary visual processing and minimized false activations compared to continuous vision-based monitoring. The system operates entirely at the edge without cloud dependency, enabling low-latency and bandwidth-efficient deployment in remote border environments.