Driver Heartbeat Sensing using Millimeter-Wave Radar based on Deep Blind Source Separation
-
摘要: 对驾驶员心脏活动的监测可以有效评估其生理和心理状态,然而,现有的心跳感知方法(心电图和远程光电容积描记法)操作繁琐,易受光线影响,并不适用于车载场景。基于毫米波雷达的心跳感知技术虽然具有高精度、非接触等优势,但易受干扰影响。针对上述问题,该文结合射频信号的低频特性、长程动态敏感性、稀疏性,基于自注意力机制设计了一个射频特征提取器,通过构建深度盲源分离网络实现了驾驶员心跳射频特征和车载干扰特征的分离。此外,针对射频信号采集难的问题,该文提出了一种混合-源信号生成策略,通过少量心震描记数据和车载干扰数据合成了训练样本。最后,该文在真实的行车环境中对该方法进行了验证,实验结果表明,系统可实现心率绝对误差4.92 bpm、心搏间期中位数误差65.93 ms的感知精度。Abstract: Monitoring of drivers’ cardiac activity enables effective assessment of their physiological and psychological states. However, existing methods such as electrocardiography and remote phontoplethysmograyhy are cumbersome and sensitive to lighting conditions, limiting their applicability in vehicular settings. Despite its high accuracy and noncontact operation, millimeter-wave radar-based heartbeat sensing is inherently vulnerable to interference. To address these issues, this paper exploits the low-frequency characteristics, long-range dynamic sensitivity, and sparsity of radio-frequency (RF) signals and designs a self-attention-based RF feature extractor. On this basis, a deep blind source separation network is constructed to separate the driver’s heartbeat-related RF features from in-vehicle interference. Furthermore, to reduce the burden of RF signal acquisition, we introduce a hybrid-source signal generation strategy that synthesizes a large number of mixed and ground-truth source signals using only a small number of seismocardiogram and interference signals. Finally, extensive on-road testing demonstrates that the proposed system achieves a median heart rate error of 4.92 bpm and a median interbeat interval error of 65.93 ms.
-
Key words:
- Wireless sensing /
- Heartbeat sensing /
- Blind source separation /
- Deep learning /
- Intelligent driving
-
表 1 雷达参数
Table 1. Radar parameters
雷达配置 具体参数 起始频率$ {\mathrm{f}}_{\mathrm{c}} $ 60 GHz 带宽$ \mathrm{B} $ 3.9 GHz 啁啾持续时间$ {\mathrm{T}}_{\mathrm{c}} $ 60 ms 距离单元$ \mathrm{R} $ 30 相位信号采样率$ {\mathrm{f}}_{\mathrm{s}} $ 100 Hz 表 2 心率估计误差统计结果
Table 2. Heart rate estimation error statistics
方案 HR_RMSE HR_MAE HR_MED 基线方案1 10.46bpm 9.49bpm 10.12bpm 基线方案2 5.26bpm 4.35bpm 4.17bpm 本文方案 3.79 bpm 3.35 bpm 3.16 bpm 表 3 心搏间期估计误差统计结果
Table 3. IBI estimation error statistics
方案 IBI_RMSE IBI_MAE IBI_MED 基线方案1 148.02ms 125.94ms 120.00ms 基线方案2 97.16 ms 78.42 ms 67.29 ms 本文方案 45.13 ms 41.87 ms 39.23 ms 表 4 不同模型结构、训练方式下心率、心搏间期估计误差结果
Table 4. HR and IBI estimation errors under different model structures and training strategies
特征提取器 损失函数 叠加方式 HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED
本文结构SI-SNR 线性叠加 3.79 bpm 3.35 bpm 3.16 bpm 45.13 ms 41.87 ms 39.23 ms 非线性叠加 4.23 bpm 3.53 bpm 3.22 bpm 47.70 ms 46.90 ms 42.79 ms Cos-Sim 线性叠加 4.36 bpm 3.61 bpm 3.27 bpm 48.99 ms 46.56 ms 42.93 ms 非线性叠加 4.52 bpm 3.78 bpm 3.30 bpm 49.11 ms 46.79 ms 43.42 ms
mmformerSI-SNR 线性叠加 4.60 bpm 3.81 bpm 3.34 bpm 49.33 ms 46.93 ms 44.18 ms 非线性叠加 4.99 bpm 4.23 bpm 3.76 bpm 51.32 ms 49.04 ms 46.14 ms Cos-Sim 线性叠加 4.64 bpm 3.83 bpm 3.40 bpm 50.50 ms 48.50 ms 45.22 ms 非线性叠加 5.03 bpm 4.25 bpm 3.71 bpm 52.81 ms 50.18 ms 46.47 ms
ConvtasnetSI-SNR 线性叠加 5.04 bpm 4.30 bpm 3.80 bpm 55.03 ms 52.22 ms 48.12 ms 非线性叠加 6.23 bpm 5.36 bpm 4.79 bpm 59.13 ms 54.43 ms 52.15 ms Cos-Sim 线性叠加 5.62 bpm 5.32 bpm 4.75 bpm 57.50 ms 52.58 ms 48.46 ms 非线性叠加 6.68 bpm 5.89 bpm 4.87 bpm 67.50 ms 62.58 ms 53.27 ms 表 5 不同训练数据集下心率、心搏间期估计误差结果
Table 5. HR and IBI estimation errors under different training datasets
HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED 5000 对7.18bpm 6.39bpm 5.51bpm 87.53ms 74.11 ms 60.00ms 10000 对5.96bpm 5.53bpm 5.05bpm 63.67 ms 62.16 ms 59.47ms 25000 对4.77bpm 3.86bpm 3.72 bpm 54.36ms 49.23 ms 47.87ms 50000 对3.79 bpm 3.35 bpm 3.16 bpm 45.13 ms 41.87 ms 39.23 ms 表 6 不同受试者的心率、心搏间期估计误差结果
Table 6. HR and IBI estimation errors across different subjects
HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED 受试者1 4.52 bpm 3.92 bpm 3.07 bpm 44.00ms 43.28 ms 41.24 ms 受试者2 3.42 bpm 2.99 bpm 3.19 bpm 44.65 ms 43.82 ms 44.71 ms 受试者3 4.01 bpm 3.84 bpm 3.05 bpm 40.12 ms 38.72 ms 38.27 ms 受试者4 2.50 bpm 2.26 bpm 2.17 bpm 35.95 ms 34.31 ms 31.43 ms 受试者5 4.09 bpm 3.58 bpm 2.88 bpm 59.54 ms 53.85 ms 50.57 ms 受试者6 3.32 bpm 3.31 bpm 3.38 bpm 53.31 ms 50.37 ms 45.06 ms 受试者7 4.06 bpm 3.21 bpm 2.07 bpm 43.84 ms 41.02 ms 36.76 ms 受试者8 4.02 bpm 3.71 bpm 3.23 bpm 40.77 ms 39.60 ms 40.45 ms 表 7 不同车型条件下心率、心搏间期估计误差结果
Table 7. HR and IBI estimation errors under different vehicle types
HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED 油车 4.06 bpm 3.21 bpm 2.07 bpm 43.84 ms 41.02 ms 36.76 ms 电车 3.37 bpm 3.05 bpm 2.97 bpm 33.16 ms 31.32 ms 28.47 ms 表 8 不同路面情况下心率、心搏间期估计误差结果
Table 8. HR and IBI estimation errors under different road conditions
HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED 良好 3.42 bpm 2.99 bpm 3.19 bpm 44.65 ms 43.82 ms 44.71 ms 一般 5.13 bpm 5.09 bpm 4.94 bpm 87.25 ms 62.29 ms 61.70 ms 差 7.64 bpm 6.67 bpm 6.19 bpm 101.08 ms 94.65 ms 91.39 ms 综合 5.40 bpm 4.92 bpm 4.77 bpm 77.66 ms 66.92 ms 65.93 ms 表 9 不同驾驶环境下心率、心搏间期估计误差结果
Table 9. HR and IBI estimation errors under under different driving environments
HR_RMSE HR_MAE HR_MED IBI_RMSE IBI_MAE IBI_MED 静态(未启动) 1.28 bpm 1.17bpm 0.99bpm 30.87ms 30.73ms 29.31ms 静态(已启动) 1.90 bpm 1.59bpm 1.51bpm 47.31ms 43.36ms 40.00ms 动态 3.32 bpm 3.31 bpm 3.38 bpm 53.31 ms 50.37 ms 45.06 ms -
[1] World Health Organization. World health statistics 2025[R/OL]. Geneva: World Health Organization, 2025[2026-05-12]. https://www.who.int/publications/b/78420. [2] 国家统计局. 中国统计年鉴2024:24-5 道路交通事故情况(2023年)[M/OL]. 北京: 中国统计出版社, 2024[2026-05-12]. https://www.stats.gov.cn/sj/ndsj/2024/indexch.htm.National Bureau of Statistics of China. China Statistical Yearbook 2024: 24-5 Traffic Accidents (2023)[M/OL]. Beijing: China Statistics Press,2024[2026-05-12].https://www.stats.gov.cn/sj/ndsj/2024/indexch.htm. [3] DU Guanglong, ZHANG Linlin, SU Kang, et al. A multimodal fusion fatigue driving detection method based on heart rate and PERCLOS[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21810–21820. doi: 10.1109/TITS.2022.3176973. [4] POP G N, CHRISTODORESCU R, VELIMIROVICI D E, et al. Assessment of the impact of alcohol consumption patterns on heart rate variability by machine learning in healthy young adults[J]. Medicina, 2021, 57(9): 956. doi: 10.3390/medicina57090956. [5] HASNUL M A, AZIZ N A A, ALELYANI S, et al. Electrocardiogram-based emotion recognition systems and their applications in healthcare—a review[J]. Sensors, 2021, 21(15): 5015. doi: 10.3390/s21155015. [6] CHEN Weixuan and MCDUFF D. Deepphys: Video-based physiological measurement using convolutional attention networks[C]. 15th European Conference on Computer Vision – ECCV 2018, Munich, Germany, 2018: 356–373. doi: 10.1007/978-3-030-01216-8_22. [7] YU Zitong, SHEN Yuming, SHI Jingang, et al. PhysFormer: Facial video-based physiological measurement with temporal difference transformer[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 4176–4186. doi: 10.1109/CVPR52688.2022.00415. [8] ZOU Bochao, GUO Zizheng, CHEN Jiansheng, et al. RhythmFormer: Extracting patterned rPPG signals based on periodic sparse attention[J]. Pattern Recognition, 2025, 164: 111511. doi: 10.1016/j.patcog.2025.111511. [9] CHOI J and LEE S J. MMDrive: Multi-modal remote physiological signal measurement dataset for driver status monitoring[C]. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, USA, 2025: 5691–5698. doi: 10.1109/CVPRW67362.2025.00567. [10] DAI Tianyang, CHEN Yan, and HU Yang. rPPG-NDCL: Unsupervised remote physiological measurement via noise-disentangled contrastive learning[C]. 2025 IEEE International Conference on Image Processing (ICIP), Anchorage, USA, 2025: 2856–2861. doi: 10.1109/ICIP55913.2025.11084290. [11] LIANG Qian, CHEN Yan, and Hu Yang. Continual learning for remote physiological measurement: Minimize forgetting and simplify inference[C]. 18th European Conference on Computer Vision – ECCV 2024, Milan, Italy, 2024: 126–144. doi: 10.1007/978-3-031-72764-1_8. [12] 杨轩, 王子颖, 张力, 等. 基于盲源分离的多人呼吸信号检测方法[J]. 雷达学报(中英文), 2025, 14(1): 117–134. doi: 10.12000/JR24115.YANG Xuan, WANG Ziying, ZHANG Li, et al. Noncontact multiperson respiratory detection method based on blind source separation[J]. Journal of Radars, 2025, 14(1): 117–134. doi: 10.12000/JR24115. [13] 赵翔, 王威, 李晨洋, 等. 基于毫米波雷达微动信号和脉搏波数据融合的睡眠呼吸暂停低通气综合征筛查技术[J]. 雷达学报(中英文), 2025, 14(1): 102–116. doi: 10.12000/JR24107.ZHAO Xiang, WANG Wei, LI Chenyang, et al. Diagnosis of sleep apnea hypopnea syndrome using fusion of micro-motion signals from millimeter-wave radar and pulse wave data[J]. Journal of Radars, 2025, 14(1): 102–116. doi: 10.12000/JR24107. [14] 方震, 简璞, 张浩, 等. 基于FMCW雷达的非接触式医疗健康监测技术综述[J]. 雷达学报, 2022, 11(3): 499–516. doi: 10.12000/JR22019.FANG Zhen, JIAN Pu, ZHANG Hao, et al. Review of noncontact medical and health monitoring technologies based on FMCW radar[J]. Journal of Radars, 2022, 11(3): 499–516. doi: 10.12000/JR22019. [15] 郑学召, 丁文, 黄渊, 等. 不同领域下超宽带雷达探测呼吸心跳信号研究综述[J]. 雷达学报, 2024, 14(1): 204–228. doi: 10.12000/JR24154.ZHENG Xuezhao, DING Wen, HUANG Yuan, et al. A review of UWB radar detection of respiration and heartbeat signals in different scenarios[J]. Journal of Radars, 2025, 14(1): 204–228. doi: 10.12000/JR24154. [16] 李志, 唐成垚, 戴永鹏, 等. 多旋翼无人机载4D成像雷达生命体征感知方法[J]. 雷达学报(中英文), 2025, 14(1): 62–72. doi: 10.12000/JR24128.LI Zhi, TANG Chengyao, DAI Yongpeng, et al. Multirotor UAV-borne vital signs sensing using 4D imaging radar[J]. Journal of Radars, 2025, 14(1): 62–72. doi: 10.12000/JR24128. [17] 刘畅宇, 张浩, 耿芳琳, 等. 基于距离抽头重构的生理雷达动态解调算法[J]. 雷达学报(中英文), 2025, 14(1): 135–150. doi: 10.12000/JR24151.LIU Changyu, ZHANG Hao, GENG Fanglin, et al. Dynamic demodulation algorithm for bio-radar sensors based on range tapper[J]. Journal of Radars, 2025, 14(1): 135–150. doi: 10.12000/JR24151. [18] GUO Zehan, ZHANG Binbin, CHEN Jinbo, et al. Contactless fine-grained cardiac events detection and segmentation with radio frequency signals[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2025, 9(3): 82. doi: 10.1145/3749457. [19] ZHANG Binbin, ZHANG Dongheng, LI Yadong, et al. Monitoring long-term cardiac activity with contactless radio frequency signals[J]. Nature Communications, 2024, 15(1): 10598. doi: 10.1038/s41467-024-55061-9. [20] YUAN Yuqin, CHEN Jinbo, ZHANG Dongheng, et al. Atrial fibrillation detection via contactless radio monitoring and knowledge transfer[J]. Nature Communications, 2025, 16(1): 4317. doi: 10.1038/s41467-025-59482-y. [21] GONG Hanqin, CHEN Jinbo, XU Guixin, et al. Osense: Omni-directional heartbeat sensing with radio signal[J]. IEEE Internet of Things Journal, 2025, 12(12): 25269–25285. doi: 10.1109/JIOT.2025.3558733. [22] XU Xiaohan, ZHANG Dongheng, LU Zhi, et al. Contactless nighttime stress monitoring with mmwave radar[C]. ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 2025: 1–5. doi: 10.1109/ICASSP49660.2025.10890411. [23] ZHANG Haoyu, ZHANG Dongheng, SONG Ruiyuan, et al. UMIMO: Universal unsupervised learning for mmwave radar sensing with MIMO array synthesis[J]. IEEE Transactions on Mobile Computing, 2025, 24(8): 7042–7058. doi: 10.1109/TMC.2025.3546757. [24] LIN Jiayan, GUO Shisheng, XUE Shucheng, et al. Non-line-of-sight human vital sign detection method based on correntropy and cross-correlation with multipath exploitation[C]. 2025 IEEE Radar Conference (RadarConf25), Krakow, Poland, 2025: 1028–1033. doi: 10.1109/RadarConf2559087.2025.11205110. [25] GU Boyuan, YANG Yanhui, YOU Siyu, et al. Improved VMD based remote heartbeat estimation utilizing 60GHz mmWave radar[C]. 2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Vienna, Austria, 2025: 4635–4640. doi: 10.1109/SMC58881.2025.11342664. [26] LI Jiachen, GUO Shisheng, CUI Guolong, et al. Multidomain separation for human vital signs detection with FMCW radar in interference environment[J]. IEEE Transactions on Microwave Theory and Techniques, 2024, 72(7): 4278–4293. doi: 10.1109/TMTT.2023.3337101. [27] LIU Xingyao, CHEN Jiahui, GUO Shisheng, et al. Non-line-of-sight human vital sign detection method based on multipath signals fusion[C]. 2023 3rd International Conference on Frontiers of Electronics, Information and Computation Technologies (ICFEICT), Yangzhou, China, 2023: 489–496. doi: 10.1109/ICFEICT59519.2023.00086. [28] HA U, MADANI S, and ADIB F. WiStress: Contactless stress monitoring using wireless signals[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2021, 5(3): 103. doi: 10.1145/3478121. [29] LIANG Kun, ZHOU Anfu, ZHANG Zhan, et al. mmStress: Distilling human stress from daily activities via contact-less millimeter-wave sensing[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2023, 7(3): 110. doi: 10.1145/3610926. [30] ZHU Juncen, CAO Jiannong, YANG Yanni, et al. mmdrive: Fine-grained fatigue driving detection using mmwave radar[J]. ACM Transactions on Internet of Things, 2023, 4(4): 26. doi: 10.1145/3614437. [31] ZHENG Tianyue, CHEN Zhe, CAI Chao, et al. V2iFi: In-vehicle vital sign monitoring via compact RF sensing[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2020, 4(2): 70. doi: 10.1145/3397321. [32] XU Xiangyu, YU Jiadi, MA Chengguang, et al. mmECG: Monitoring human cardiac cycle in driving environments leveraging millimeter wave[C]. IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, London, United Kingdom, 2022: 90–99. doi: 10.1109/INFOCOM48880.2022.9796912. [33] WANG Fengyu, ZENG Xiaolu, WU Chenshu, et al. Driver vital signs monitoring using millimeter wave radio[J]. IEEE Internet of Things Journal, 2022, 9(13): 11283–11298. doi: 10.1109/JIOT.2021.3128548. [34] HU Qingyong, ZHANG Qian, LU Hao, et al. Contactless arterial blood pressure waveform monitoring with mmwave radar[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2024, 8(4): 178. doi: 10.1145/3699781. [35] KHAN U M, RIGAZIO L, and SHAHZAD M. Contactless monitoring of PPG using radar[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2022, 6(3): 123. doi: 10.1145/3550330. [36] CHEN Jinbo, ZHANG Dongheng, WU Zhi, et al. Contactless electrocardiogram monitoring with millimeter wave radar[J]. IEEE Transactions on Mobile Computing, 2024, 23(1): 270–285. doi: 10.1109/TMC.2022.3214721. [37] WANG Haoyu, CHEN Jinbo, ZHANG Dongheng, et al. Contactless radar heart rate variability monitoring via deep spatio-temporal modeling[C]. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Korea, Republic of, 2024: 111–115. doi: 10.1109/ICASSP48485.2024.10447570. [38] Wang Jiyao, YANG Xiao, HU Qingyong, et al. Physdrive: A multimodal remote physiological measurement dataset for in-vehicle driver monitoring[J]. Advances in Neural Information Processing Systems, 2026, 38. [39] ZHANG Dongheng, HU Yang, and CHEN Yan. MTrack: Tracking multiperson moving trajectories and vital signs with radio signals[J]. IEEE Internet of Things Journal, 2021, 8(5): 3904–3914. doi: 10.1109/JIOT.2020.3025820. [40] ZHAO Mingmin, ADIB F, and KATABI D. Emotion recognition using wireless signals[C]. The 22nd Annual International Conference on Mobile Computing and Networking, New York, USA, 2016: 95–108. doi: 10.1145/2973750.2973762. [41] ÖZER Y and MÜLLER M. Source separation of piano concertos using musically motivated augmentation techniques[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 1214–1225. doi: 10.1109/TASLP.2024.3356980. [42] LUO Yi and MESGARANI N. Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, 27(8): 1256–1266. doi: 10.1109/TASLP.2019.2915167. [43] STOLLER D, EWERT S, and DIXON S. Wave-U-Net: A multi-scale neural network for end-to-end audio source separation [C]. The 19th International Society for Music Information Retrieval Conference, Paris, France, 2018: 334–340. [44] DÉFOSSEZ A, USUNIER N, BOTTOU L, et al. Demucs: Deep extractor for music sources with extra unlabeled data remixed[J]. arXiv preprint arXiv:1909.01174, 2019. [45] HA U, ASSANA S, and ADIB F. Contactless seismocardiography via deep learning radars [C]. The 26th Annual International Conference on Mobile Computing and Networking, London, United Kingdom, 2020: 62. doi: 10.1145/3372224.3419982. [46] PARLATO S, CENTRACCHIO J, CINOTTI E, et al. A Forcecardiography dataset with simultaneous SCG, Heart Sounds, ECG, and Respiratory signals[J]. Scientific Data, 2025, 12(1): 1370. doi: 10.1038/s41597-025-05694-2. [47] CHEN Jiayu, SONG Yingjian, ZHANG Yida, et al. SelfDenoiser: Self-supervised seismic signal denoiser for continuous and contactless cardiac monitoring[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2025, 9(4): 165. doi: 10.1145/3770701. [48] DROITCOUR A D, BORIC-LUBECKE O, LUBECKE V M, et al. Range correlation and I/Q performance benefits in single-chip silicon Doppler radars for noncontact cardiopulmonary monitoring[J]. IEEE Transactions on Microwave Theory and Techniques, 2004, 52(3): 838–848. doi: 10.1109/TMTT.2004.823552. -
作者中心
专家审稿
责编办公
编辑办公
下载: