In order to achieve zero downtime in Industry 4.0, real-time visibility is a must to ensure timely action. In this paper, we propose a smart manufacturing system (SMS-DT) which makes use of digital twin (DT) technology to offer real-time production tracking and anomaly detection. Our proposed approach combines an IoT-based sensing layer that collects temperatures, vibrations, and cycle times of CNC machines, a high-fidelity digital twin that utilizes BiLSTM with attention mechanism to predict states and remaining useful lives (RUL) of the machines, and a real-time anomaly detection model that employs graph neural network (GNN) to model interdependence between machines. Deployed in a 15-CNC-machine testbed for 6 months, SMS-DT is able to achieve 94.7% detection accuracy, 12.8% unplanned downtime reduction, and 18.3% OEE enhancement. Compared with SCADA-only and conventional predictive maintenance systems, SMS-DT outperforms in terms of detection latency and false positive rate.