Adaptive HRV analysis: Reinforcement learning-driven training load monitoring in sports science
Abstract
Heart rate variability (HRV) is a widely used biomarker for assessing physiological stress, recovery, and training load in sports science. During exercise, the mechanical changes in various parts of the body, such as muscle contraction and relaxation, joint movement, and the dynamic response of the cardiovascular system, are closely related to HRV. However, traditional analysis methods face significant challenges in handling HRV’s nonlinear dynamics and noise sensitivity. These limitations reduce their effectiveness in complex sport scenarios. To address these limitations, this study proposes an innovative HRV feature extraction framework that integrates reinforcement learning (RL) with an attention-based Long Short-Term Memory (LSTM) network. The framework dynamically optimizes feature selection and weighting through RL. The integration of an attention mechanism enables the model to prioritize critical temporal segments, improving its ability to capture and interpret key physiological patterns. Additionally, the model combines time-domain, frequency-domain, biomechanical factors, and nonlinear features, providing a comprehensive and robust representation of HRV signals. The framework was validated on four publicly available datasets covering resting, exercise, stress, and recovery states. It achieved an average accuracy of 95.0% and an F1-score of 90.8%, outperforming state-of-the-art baselines by 2.7% to 3.4%. These results demonstrate the proposed method’s superior performance in stress detection, training load prediction, and recovery assessment, establishing it as a scalable and adaptive tool for HRV-based sports training monitoring and health management. The framework’s innovative design offers significant advancements in the analysis of complex HRV data, paving the way for intelligent and personalized applications in sports science and healthcare.
References
1. Falk Neto JH, Kennedy MD. The multimodal nature of high-intensity functional training: potential applications to improve sport performance. Sports. 2019; 7(2): 33.
2. Wiens L. Comparing the effect of low-load resistance training versus high intensity interval training on muscle endurance, muscle strength, muscle hypertrophy, VO₂peak and anaerobic performance [Master’s thesis]. University of British Columbia; 2024.
3. Natera AO, Cardinale M, Keogh JWL. The effect of high volume power training on repeated high-intensity performance and the assessment of repeat power ability: a systematic review. Sports Medicine. 2020; 50: 1317-1339.
4. Chu Y, Wang Q, Chu M, et al. Long-Term Effect of Vibration Therapy for Training-Induced Muscle Fatigue in Elite Athletes. International Journal of Environmental Research and Public Health. 2022; 19(12): 7531.
5. de Lima EP, Tanaka M, Lamas CB, et al. Vascular impairment, muscle atrophy, and cognitive decline: Critical age-related conditions. Biomedicines. 2024; 12(9): 2096.
6. Faust O, Hong W, Loh HW, et al. Heart rate variability for medical decision support systems: A review. Computers in Biology and Medicine. 2022; 145: 105407. doi: 10.1016/j.compbiomed.2022.105407
7. Umer W, Yu Y, Antwi-Afari MF, et al. Heart rate variability based physical exertion monitoring for manual material handling tasks. International Journal of Industrial Ergonomics. 2022; 89: 103301. doi: 10.1016/j.ergon.2022.103301
8. Grégoire JM, Gilon C, Carlier S, et al. Autonomic nervous system assessment using heart rate variability. Acta Cardiologica. 2023; 78(6): 648-662. doi: 10.1080/00015385.2023.2177371
9. Pham T, Lau ZJ, Chen SHA, et al. Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial. Sensors. 2021; 21(12): 3998. doi: 10.3390/s21123998
10. Peabody JE, Ryznar R, Ziesmann MT, et al. A Systematic Review of Heart Rate Variability as a Measure of Stress in Medical Professionals. Cureus. 2023. doi: 10.7759/cureus.34345
11. Kim SW, Park HY, Jung H, et al. Development of a Heart Rate Variability Prediction Equation Through Multiple Linear Regression Analysis Using Physical Characteristics and Heart Rate Variables. INQUIRY: The Journal of Health Care Organization, Provision, and Financing. 2023; 60. doi: 10.1177/00469580231169416
12. G S, Kp S, R V. Automated detection of diabetes using CNN and CNN-LSTM network and heart rate signals. Procedia Computer Science. 2018; 132: 1253-1262. doi: 10.1016/j.procs.2018.05.041
13. Coutts LV, Plans D, Brown AW, et al. Deep learning with wearable based heart rate variability for prediction of mental and general health. Journal of Biomedical Informatics. 2020; 112: 103610. doi: 10.1016/j.jbi.2020.103610
14. Shi K, Steigleder T, Schellenberger S, et al. Contactless analysis of heart rate variability during cold pressor test using radar interferometry and bidirectional LSTM networks. Scientific Reports. 2021; 11(1). doi: 10.1038/s41598-021-81101-1
15. Magosso E, Ricci G, Ursino M. Modulation of brain alpha rhythm and heart rate variability by attention-related mechanisms. AIMS Neuroscience. 2019; 6(1): 1-24. doi: 10.3934/neuroscience.2019.1.1
16. Matsuo Y, LeCun Y, Sahani M, et al. Deep learning, reinforcement learning, and world models. Neural Networks. 2022; 152: 267-275. doi: 10.1016/j.neunet.2022.03.037
17. Electrophysiology TF of the ES of C the NA. Heart Rate Variability. Circulation. 1996; 93(5): 1043-1065. doi: 10.1161/01.cir.93.5.1043
18. Musialik-Łydka A, Średniawa B, Pasyk S. Heart rate variability in heart failure. Polish Heart Journal (Kardiologia Polska). 2003; 58(1): 14-16.
19. DeGiorgio CM, Miller P, Meymandi S, et al. RMSSD, a measure of vagus-mediated heart rate variability, is associated with risk factors for SUDEP: The SUDEP-7 Inventory. Epilepsy & Behavior. 2010; 19(1): 78-81. doi: 10.1016/j.yebeh.2010.06.011
20. Compostella L, Lakusic N, Compostella C, et al. Does heart rate variability correlate with long-term prognosis in myocardial infarction patients treated by early revascularization? World Journal of Cardiology. 2017; 9(1): 27. doi: 10.4330/wjc.v9.i1.27
21. Bentley RF, Vecchiarelli E, Banks L, et al. Heart rate variability and recovery following maximal exercise in endurance athletes and physically active individuals. Applied Physiology, Nutrition, and Metabolism. 2020; 45(10): 1138-1144. doi: 10.1139/apnm-2020-0154
22. Thomas BL, Claassen N, Becker P, et al. Validity of Commonly Used Heart Rate Variability Markers of Autonomic Nervous System Function. Neuropsychobiology. 2019; 78(1): 14-26. doi: 10.1159/000495519
23. Posada-Quintero HF, Florian JP, Orjuela-Cañón AD, et al. Power Spectral Density Analysis of Electrodermal Activity for Sympathetic Function Assessment. Annals of Biomedical Engineering. 2016; 44(10): 3124-3135. doi: 10.1007/s10439-016-1606-6
24. Jia Y, Pei H, Liang J, et al. Preprocessing and Denoising Techniques for Electrocardiography and Magnetocardiography: A Review. Bioengineering. 2024; 11(11): 1109. doi: 10.3390/bioengineering11111109
25. Satti R, Abid NUH, Bottaro M, et al. The Application of the Extended Poincaré Plot in the Analysis of Physiological Variabilities. Frontiers in Physiology. 2019; 10. doi: 10.3389/fphys.2019.00116
26. Udhayakumar RK, Karmakar C, Palaniswami M. Understanding Irregularity Characteristics of Short-Term HRV Signals Using Sample Entropy Profile. IEEE Transactions on Biomedical Engineering. 2018; 65(11): 2569-2579. doi: 10.1109/tbme.2018.2808271
27. Wu H. Multiscale entropy with electrocardiograph, electromyography, electroencephalography, and photoplethysmography signals in healthcare: A twelve-year systematic review. Biomedical Signal Processing and Control. 2024; 93: 106124. doi: 10.1016/j.bspc.2024.106124
28. Xie L, Li Z, Zhou Y, et al. Computational Diagnostic Techniques for Electrocardiogram Signal Analysis. Sensors. 2020; 20(21): 6318. doi: 10.3390/s20216318
29. Stephenson MD, Thompson AG, Merrigan JJ, et al. Applying Heart Rate Variability to Monitor Health and Performance in Tactical Personnel: A Narrative Review. International Journal of Environmental Research and Public Health. 2021; 18(15): 8143. doi: 10.3390/ijerph18158143
30. Turcu AM, Ilie AC, Ștefăniu R, et al. The Impact of Heart Rate Variability Monitoring on Preventing Severe Cardiovascular Events. Diagnostics. 2023; 13(14): 2382. doi: 10.3390/diagnostics13142382
31. Wang Z, Luo Y, Zhang Y, et al. Heart rate variability in generalized anxiety disorder, major depressive disorder and panic disorder: A network meta-analysis and systematic review. Journal of Affective Disorders. 2023; 330: 259-266. doi: 10.1016/j.jad.2023.03.018
32. Nayak SK, Pradhan B, Mohanty B, et al. A Review of Methods and Applications for a Heart Rate Variability Analysis. Algorithms. 2023; 16(9): 433. doi: 10.3390/a16090433
33. Berrahou N, El Alami A, Mesbah A, et al. Arrhythmia detection in inter-patient ECG signals using entropy rate features and RR intervals with CNN architecture. Computer Methods in Biomechanics and Biomedical Engineering. Published online July 17, 2024: 1-20. doi: 10.1080/10255842.2024.2378105
34. Satheeswaran V, Chandrika GN, Mitra A, et al. Deep Learning based classification of ECG signals using RNN and LSTM Mechanism. Journal of Electronics, Electromedical Engineering, and Medical Informatics. 2024; 6(4): 332-342.
35. Ramteke RB, Thool VR. Heart rate variability-based mental stress detection using deep learning approach Applied Information Processing Systems: Proceedings of ICCET 2021. Springer Singapore; 2022. pp. 51-61.
36. Fan T, Qiu S, Wang Z, et al. A new deep convolutional neural network incorporating attentional mechanisms for ECG emotion recognition. Computers in Biology and Medicine. 2023; 159: 106938. doi: 10.1016/j.compbiomed.2023.106938
37. Xu J, Song C, Yue Z, et al. Facial Video-Based Non-Contact Stress Recognition Utilizing Multi-Task Learning With Peak Attention. IEEE Journal of Biomedical and Health Informatics. 2024; 28(9): 5335-5346. doi: 10.1109/jbhi.2024.3412103
38. Shah JK, Yadav A, Hopko SK, et al. Robot Adaptation Under Operator Cognitive Fatigue Using Reinforcement Learning. 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). Published online August 28, 2023: 1467-1474. doi: 10.1109/ro-man57019.2023.10309639
39. Li C, Shi Z, Zhou L, et al. TFFormer: A Time–Frequency Information Fusion-Based CNN-Transformer Model for OSA Detection With Single-Lead ECG. IEEE Transactions on Instrumentation and Measurement. 2023; 72: 1-17. doi: 10.1109/tim.2023.3312472
40. Anwar A, Khalifa Y, Coyle JL, et al. Transformers in biosignal analysis: A review. Information Fusion. 2025; 114: 102697. doi: 10.1016/j.inffus.2024.102697
Copyright (c) 2025 Author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright on all articles published in this journal is retained by the author(s), while the author(s) grant the publisher as the original publisher to publish the article.
Articles published in this journal are licensed under a Creative Commons Attribution 4.0 International, which means they can be shared, adapted and distributed provided that the original published version is cited.
Submit a Paper
