Application of deep reinforcement learning under biomechanical load optimization in warehouse site selection and material transportation path
Abstract
Path optimization of logistics and transportation systems has traditionally focused on the balance between efficiency and cost, but there is a lack of systematic research on the biomechanical load of transport personnel, which leads to fatigue accumulation and increased safety hazards. To fill this gap, this paper proposes a path optimization method based on deep reinforcement learning (DRL) based on biomechanical theory, aiming to combine biomechanical load management of transport personnel with logistics efficiency improvement. Firstly, a biomechanical load assessment system for transport personnel during long-distance driving is established using human kinematics and dynamics models, with quantitative indicators including muscle fatigue index, joint load and driving posture stability. Secondly, a national logistics transportation network is constructed based on a graph theory model, with transportation distance, time and biomechanical load as constraints for multi-objective optimization, and a Deep Q Network (DQN) is designed for path planning optimization. The calculation of fatigue index is combined with driving time, road section characteristics and individual biomechanical characteristics, and verified by biomechanical simulation tools. In order to improve the optimization efficiency, the simulated annealing algorithm is used to preliminarily screen the paths, and the DRL model is combined to achieve dynamic adjustment. The experimental results show that this method significantly reduces the biomechanical load of transport personnel in nationwide logistics scheduling (the fatigue index is controlled below 0.12), and at the same time reduces the accident rate caused by fatigue (reduced by 40%), and the transportation efficiency is superior to traditional research. The research results not only deepen the application of biomechanical theory in the field of long-distance transportation, but also provide theoretical support and technical reference for building a safe, efficient and intelligent logistics and transportation system, and promote the integrated development of biomechanics and artificial intelligence in complex engineering problems.
References
1. Leng K, Li S. Distribution Path Optimization for Intelligent Logistics Vehicles of Urban Rail Transportation Using VRP Optimization Model. IEEE Transactions on Intelligent Transportation Systems. 2022; 23(2): 1661-1669. doi: 10.1109/tits.2021.3105105
2. Yu H, Murray AT, Fang Z, et al. Ship Path Optimization That Accounts for Geographical Traffic Characteristics to Increase Maritime Port Safety. IEEE Transactions on Intelligent Transportation Systems. 2022; 23(6): 5765-5776. doi: 10.1109/tits.2021.3057907
3. Zhang D, Luo R, Yin Y bo, et al. Multi-objective path planning for mobile robot in nuclear accident environment based on improved ant colony optimization with modified A∗. Nuclear Engineering and Technology. 2023; 55(5): 1838-1854. doi: 10.1016/j.net.2023.02.005
4. Yucel E, Salman FS, Erdoğan G. Optimizing two-dimensional vehicle loading and dispatching decisions in freight logistics. European Journal of Operational Research. 2022; 302(3): 954-969. doi: 10.1016/j.ejor.2022.01.021
5. Lan YL, Liu F, Ng WWY, et al. Decomposition Based Multi-Objective Variable Neighborhood Descent Algorithm for Logistics Dispatching. IEEE Transactions on Emerging Topics in Computational Intelligence. 2021; 5(5): 826-839. doi: 10.1109/tetci.2020.3002228
6. Lei N. Intelligent logistics scheduling model and algorithm based on Internet of Things technology. Alexandria Engineering Journal. 2022; 61(1): 893-903. doi: 10.1016/j.aej.2021.04.075
7. Cakmak E, Önden İ, Acar AZ, et al. Analyzing the location of city logistics centers in Istanbul by integrating Geographic Information Systems with Binary Particle Swarm Optimization algorithm. Case Studies on Transport Policy. 2021; 9(1): 59-67. doi: 10.1016/j.cstp.2020.07.004
8. Sihotang HT, Riandari F, Sihotang J. Graph-based Exploration for Mining and Optimization of Yields (GEMOY Method). Jurnal Teknik Informatika CIT Medicom. 2024; 16(2): 70-81. doi: 10.35335/cit.vol16.2024.777.pp70-81
9. Pan W, Liu SQ. Deep reinforcement learning for the dynamic and uncertain vehicle routing problem. Applied Intelligence. 2022; 53(1): 405-422. doi: 10.1007/s10489-022-03456-w
10. Zhang Y, Bai R, Qu R, et al. A deep reinforcement learning based hyper-heuristic for combinatorial optimisation with uncertainties. European Journal of Operational Research. 2022; 300(2): 418-427. doi: 10.1016/j.ejor.2021.10.032
11. Zhao J, Mao M, Zhao X, et al. A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems. IEEE Transactions on Intelligent Transportation Systems. 2021; 22(11): 7208-7218. doi: 10.1109/tits.2020.3003163
12. Liu R, Qu Z, Huang G, et al. DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network. IEEE Transactions on Intelligent Vehicles. 2023; 8(2): 1204-1218. doi: 10.1109/tiv.2022.3213703
13. Wang X, Wang S, Liang X, et al. Deep Reinforcement Learning: A Survey. IEEE Transactions on Neural Networks and Learning Systems. 2024; 35(4): 5064-5078. doi: 10.1109/tnnls.2022.3207346
14. Wang H, Liu N, Zhang Y, et al. Deep reinforcement learning: a survey. Frontiers of Information Technology & Electronic Engineering. 2020; 21(12): 1726-1744. doi: 10.1631/fitee.1900533
15. Gronauer S, Diepold K. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review. 2021; 55(2): 895-943. doi: 10.1007/s10462-021-09996-w
16. Vinstrup J, Jakobsen MD, Madeleine P, et al. Biomechanical load during patient transfer with assistive devices: Cross-sectional study. Ergonomics. 2020; 63(9): 1164-1174. doi: 10.1080/00140139.2020.1764113
17. Verheul J, Nedergaard NJ, Pogson M, et al. Biomechanical loading during running: can a two mass-spring-damper model be used to evaluate ground reaction forces for high-intensity tasks?. Sports Biomechanics. 2019; 20(5): 571-582. doi: 10.1080/14763141.2019.1584238
18. Ali SS, Kaur R, Khan S. Identification of innovative technology enablers and drone technology determinants adoption: a graph theory matrix analysis framework. Operations Management Research. 2023; 16(2): 830-852. doi: 10.1007/s12063-023-00346-3
19. Ekhlasi A, Nasrabadi AM, Mohammadi M. Analysis of EEG brain connectivity of children with ADHD using graph theory and directional information transfer. Biomedical Engineering/Biomedizinische Technik. 2022; 68(2): 133-146. doi: 10.1515/bmt-2022-0100
20. Oroojlooyjadid A, Nazari M, Snyder LV, et al. A Deep Q-Network for the Beer Game: Deep Reinforcement Learning for Inventory Optimization. Manufacturing & Service Operations Management. 2022; 24(1): 285-304. doi: 10.1287/msom.2020.0939
21. Tao X, Hafid AS. DeepSensing: A Novel Mobile Crowdsensing Framework With Double Deep Q-Network and Prioritized Experience Replay. IEEE Internet of Things Journal. 2020; 7(12): 11547-11558. doi: 10.1109/jiot.2020.3022611
22. Talaat FM. Effective deep Q-networks (EDQN) strategy for resource allocation based on optimized reinforcement learning algorithm. Multimedia Tools and Applications. 2022; 81(28): 39945-39961. doi: 10.1007/s11042-022-13000-0
23. Abdel-Basset M, Ding W, El-Shahat D. A hybrid Harris Hawks optimization algorithm with simulated annealing for feature selection. Artificial Intelligence Review. 2020; 54(1): 593-637. doi: 10.1007/s10462-020-09860-3
24. Fontes DBMM, Homayouni SM, Gonçalves JF. A hybrid particle swarm optimization and simulated annealing algorithm for the job shop scheduling problem with transport resources. European Journal of Operational Research. 2023; 306(3): 1140-1157. doi: 10.1016/j.ejor.2022.09.006
25. Shi K, Wu Z, Jiang B, et al. Dynamic path planning of mobile robot based on improved simulated annealing algorithm. Journal of the Franklin Institute. 2023; 360(6): 4378-4398. doi: 10.1016/j.jfranklin.2023.01.033
26. Wang YD, Underwood BS, Kim YR. Development of a fatigue index parameter, Sapp, for asphalt mixes using viscoelastic continuum damage theory. International Journal of Pavement Engineering. 2020; 23(2): 438-452. doi: 10.1080/10298436.2020.1751844
27. Bonetti A, Bonetti L, Čipčić O. Self-Assessment of Vocal Fatigue in Muscle Tension Dysphonia and Vocal Nodules: A Preliminary Analysis of the Discriminatory Potential of the Croatian Adaptation of the Vocal Fatigue Index (VFI-C). Journal of Voice. 2021; 35(2): 325.e1-325.e15. doi: 10.1016/j.jvoice.2019.08.028
28. Küçükakgün H, Tulek Z, Kılıçaslan K, et al. Validation of the Turkish version of the Neurological Fatigue Index for Stroke. Cognitive Neuropsychiatry. 2024; 29(2): 141-153. doi: 10.1080/13546805.2024.2337155
29. Ding Y, Jin M, Li S, et al. Smart logistics based on the internet of things technology: an overview. International Journal of Logistics Research and Applications. 2020; 24(4): 323-345. doi: 10.1080/13675567.2020.1757053
30. Cimini C, Lagorio A, Romero D, et al. Smart Logistics and The Logistics Operator 4.0. IFAC-PapersOnLine. 2020; 53(2): 10615-10620. doi: 10.1016/j.ifacol.2020.12.2818
31. Mughni MD, Putri ARD. Implementation Of Teacher Presence System Using Mobile-Based Geofencing & Haversine Formula Methods. Applied Technology and Computing Science Journal. 2023; 6(1): 31-40. doi: 10.33086/atcsj.v6i1.4119
32. Mahatmi MF, Hasanuddin T, Umar F. Implementasi Metode Haversine Formula Untuk Menentukan Jarak Terdekat Pada Pengantaran Air Galon Depot Anantama Berbasis Android. Buletin Sistem Informasi dan Teknologi Islam. 2022; 3(1): 69-78. doi: 10.33096/busiti.v3i1.1098
33. Sharma S, Kumar V. A Comprehensive Review on Multi-objective Optimization Techniques: Past, Present and Future. Archives of Computational Methods in Engineering. 2022; 29(7): 5605-5633. doi: 10.1007/s11831-022-09778-9
34. Sari IP, Fahroza M, Fahri MM. et al. Implementation of Dijkstra’s Algorithm to Determine the Shortest Route in a City. Journal of Computer Science, Information Technology and Telecommunication Engineering. 2021.
35. Wayahdi MR, Ginting SHN, Syahputra D. Greedy, A-Star, and Dijkstra’s Algorithms in Finding Shortest Path. International Journal of Advances in Data and Information Systems. 2021; 2(1): 45-52. doi: 10.25008/ijadis.v2i1.1206
36. Akram M, Habib A, Alcantud JCR. An optimization study based on Dijkstra algorithm for a network with trapezoidal picture fuzzy numbers. Neural Computing and Applications. 2020; 33(4): 1329-1342. doi: 10.1007/s00521-020-05034-y
Copyright (c) 2025 Author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright on all articles published in this journal is retained by the author(s), while the author(s) grant the publisher as the original publisher to publish the article.
Articles published in this journal are licensed under a Creative Commons Attribution 4.0 International, which means they can be shared, adapted and distributed provided that the original published version is cited.
Submit a Paper
