Design of 2D gimbal face localization and fatigue driving detection system
Abstract
Aiming at the traditional fatigue driving detection system with a single function, unable to locate the face, greatly affected by external lighting factors and high hardware requirements, this design proposes a system that uses OpenMV equipped with a CMOS camera as the main controller, recognizes the face contour by using Haar classifiers, and controls the two-dimensional gimbal to realize the face localization function, with the MCU as the slave, and utilizes the MCU to read the blinking frequency per time unit to determine the fatigue driving. The microcontroller is the slave, using the microcontroller to read the blinking frequency captured by OpenMV per unit time to judge the fatigue driving, when the blinking frequency is higher than 30 times/min, the microcontroller triggers the sound and light alarm. At the same time, the neck and waist signal analysis module is added to collect more comprehensive driver fatigue information. For complex road conditions such as sharp curves, up and down slopes, the microcontroller can pre-judge in advance through the positioning module to remind the driver in the form of alarm. The experiment proves that the system adopts lightweight design scheme, applicable to the field of in-vehicle electronics, both with the advantages of diversified functions, high measurement accuracy, intuitive display, stable operation, etc. The combination of 2D PTZ face localization and fatigue driving detection in the mode of master-slave improves the real-time and robustness of the control system, which is useful for the improvement of fatigue driving detection by using machine vision.
References
1. Zhang A, Li Q, Li Z, Li J. Multimodal fusion convolutional neural network based on sEMG and accelerometer signals for intersubject upper limb movement classification. IEEE Sensors Journal. 2023; 23(11): 12334–12345.
2. Salah M, Ayyad A, Ramadan M, et al. High speed neuromorphic vision-based inspection of countersinks in automated manufacturing processes. Journal of Intelligent Manufacturing. 2024; 35(7): 3067–3081.
3. Yang H, Liu L, Min W, et al. Driver yawning detection based on subtle facial action recognition. IEEE Transactions on Multimedia. 2020; 23: 572–583.
4. Bai J, Yu W, Xiao Z, et al. Two-stream spatial–temporal graph convolutional networks for driver drowsiness detection. IEEE Transactions on Cybernetics. 2022; 52(12): 13821–13833.
5. Ansari S, Naghdy F, Du H P. Driver mental fatigue detection based on head posture using new modified ReLU-BiLSTM deep neural network. IEEE Transactions on Intelligent Transportation Systems. 2022; 23(8): 10957–10969.
6. Brar DS, Kumar A, Mittal U, et al. Face detection for real world application. In: Proceedings of the 2021 2nd International Conference on Intelligent Engineering and Management (ICIEM); 28–30 April 2021; Moscow, Russia; pp. 239–242.
7. Wang R, Qin JM. Fuzzy PID control method of air conditioning supply air temperature based on multi-sensing features fusion (Chinese). Chinese Journal of Sensors and Actuators. 2023; 36(6): 943–948.
8. Xu Y, Yan W, Yang G, et al. CenterFace: Joint face detection and alignment using face as point. Scientific Programming. 2020; (2): 1–8.
9. Liu Y, Deng J, Wang F, et al. DamoFD: Digging into Backbone Design on Face Detection. In: Proceedings of the Eleventh International Conference on Learning Representations (ICLR); 1–5 May 2023; Kigali, Rwanda.
10. Praveen KS, Kesava JV, Subramanya V, et al. A multiple face recognition system with dlibs resnet network using deep metric learning. Journal of Critical Reviews. 2020; 7(6): 856–859.
11. Wang X, Zhang S, Wang S, et al. Mis-classified vector guided softmax loss for face recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence; 7–12 February 2020; New York, USA. pp. 12241–12248.
12. Cai M, Cheng N, Cao C, et al. Adaptive hardness indicator softmax for deep face recognition. International Journal of Pattern Recognition and Artificial Intelligence. 2022; 36(4).
13. Kim M, Jain A K, Liu XM. Adaface: Quality adaptive margin for face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 18–24 June 2022; New Orleans, LA, USA. pp. 18750–18759.
14. El-Nabi SA, El-Shafai W, El-Rabaie ESM, et al. Machine learning and deep learning techniques for driver fatigue and drowsiness detection: A review. Multimedia Tools and Applications. 2024; 83: 9441–9477.
15. Zhou Y, Zeng CQ, Mu ZD. Optimal feature‐algorithm combination research for EEG fatigue driving detection based on functional brain network. IET Biometrics. 2023; 12(2): 65–76.
16. Akrout B, Mahdi W. A novel approach for driver fatigue detection based on visual characteristics analysis. Journal of Ambient Intelligence and Humanized Computing. 2023; 14(1): 527–552.
17. Dogan S, Tuncer I, Baygin M, Tuncer T. A new hand-modeled learning framework for driving fatigue detection using EEG signals. Neural Computing and Applications. 2023; 35(20): 14837–14854.
18. Jiao Y, Chen X, Sun Z, et al. Data-driven detection and assessment for urban railway transit driver fatigue in real work conditions. Transportation research record. 2023; 2677(1): 1367–1375.
19. Yi Y, Zhou Z, Zhang W, et al. Fatigue detection algorithm based on eye multifeature fusion. IEEE Sensors Journal. 2023; 23(7): 7949–7955.
20. Saleem AA, Siddiqui HUR, Raza MA, et al. A systematic review of physiological signals based driver drowsiness detection systems. Cognitive neurodynamics. 2023; 17(5): 1229–1259.
21. Sharma S, Kumar V. Distracted driver detection using learning representations. Multimedia Tools and Applications. 2023; 82(15): 22777–22794.
22. Hussein RM, Miften FS, George LE. Driver drowsiness detection methods using EEG signals: A systematic review. Computer methods in biomechanics and biomedical engineering. 2023; 26(11): 1237–1249.
23. Tao K, Xie K, Wen C, He J. Multi-feature fusion prediction of fatigue driving based on improved optical flow algorithm. Signal, Image and Video Processing. 2023; 17(2): 371–379.
24. Min J, Cai M, Gou C, et al. Fusion of forehead EEG with machine vision for real-time fatigue detection in an automatic processing pipeline. Neural Computing and Applications. 2023; 35(12): 8859–8872.
25. Yi Y, Zhang H, Zhang W, et al. Fatigue working detection based on facial multifeature fusion. IEEE Sensors Journal. 2023; 23(6): 5956–5961.
26. Lyu H, Yue J, Zhang W, et al. Fatigue Detection for Ship OOWs Based on input Data Features, from the Perspective of Comparison with Vehicle Drivers: A Review. IEEE Sensors Journal. 2023; 23(14): 15239–15252.
27. Sar I, Routray A, Mahanty B. A review on existing technologies for the identification and measurement of abnormal driving. International journal of intelligent transportation systems research. 2023; 21(1): 159–177.
28. Cheng W, Wang X, Mao B. A multi-feature fusion algorithm for driver fatigue detection based on a lightweight convolutional neural network. The Visual Computer. 2024; 40(4): 2419–2441.
29. Pan L, Yan C, Zheng Y, et al. Fatigue detection method for UAV remote pilot based on multi feature fusion. Electronic Research Archive. 2023; 31(1): 442–466.
30. Lu Y, Liu C, Chang F, et al. JHPFA-Net: Joint head pose and facial action network for driver yawning detection across arbitrary poses in videos. IEEE Transactions on Intelligent Transportation Systems. 2023; 24(11): 11850–11863.
31. Kumar V, Sharma S, Ranjeet. Driver drowsiness detection using modified deep learning architecture. Evolutionary Intelligence. 2023; 16(6): 1907–1916.
Copyright (c) 2025 Author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright on all articles published in this journal is retained by the author(s), while the author(s) grant the publisher as the original publisher to publish the article.
Articles published in this journal are licensed under a Creative Commons Attribution 4.0 International, which means they can be shared, adapted and distributed provided that the original published version is cited.
Submit a Paper
