A Semisupervised Radar-Based Human Action Recognition Method via Multidomain Collaborative Training
-
摘要: 针对雷达人体动作识别任务中标注数据不足的问题,该文提出了一种基于多域协同训练的半监督学习方法。该方法融合慢时间-距离域、慢时间-多普勒频率域和距离-多普勒频率域的动作特征,构建决策层集成框架,通过域间一致性评估机制动态调整各域在集成预测中的权重,并设计了分层置信度动态伪标签策略,通过多层次质量评估和动态阈值校准实现伪标签质量与利用率的平衡。此外,该方法引入了特征对齐约束机制,利用快速主成分分析方法提取多域特征的主成分,引导深度网络学习紧凑的特征表示,增强模型的判别能力。在基于随机码雷达的穿墙人体动作数据集上,该文所提方法在5%标注比例下平均识别准确率达到(93.6±1.6)%。在基于调频连续波雷达的室内人体动作数据集上,5%标注比例下平均识别准确率达到(91.3±1.9)%,既高于包括Bi-LSTM, LH-ViT和MFAFN的监督学习方法,也高于包括FixMatch, C-TGAN, MF-Match和LW-HGR的半监督学习方法。实验结果证明该方法在随机码雷达和调频连续波雷达两种不同雷达体制下,以及在穿墙和室内两种不同探测场景下,均表现稳定,验证了其跨体制和跨场景的适应性。此外,基于多域协同训练的半监督学习模型的参数量为1.30 M,浮点计算量为26.16 M,模型大小为5.01 MB,展现出较高的计算效率。Abstract: To address the problem of insufficient labeled data in radar-based human action recognition, a semisupervised learning method based on multidomain collaborative training is proposed herein. This method fuses action features from the slow time–range, slow time–Doppler frequency, and range–Doppler frequency domains to construct a decision-level ensemble framework. An interdomain consistency evaluation mechanism is employed to dynamically adjust the contribution of each domain in ensemble prediction. Furthermore, a stratified confidence dynamic pseudo-labeling strategy is designed to balance pseudo-label quality and utilization rate through multilevel quality assessment and dynamic threshold calibration. In addition, a feature alignment constraint mechanism is introduced, where fast principal component analysis is utilized to extract the principal components of multidomain features. This mechanism guides the deep network to learn compact feature representations and enhances model discriminability. On the through-wall human action dataset based on a random code radar, an average recognition accuracy of (93.6±1.6)% is achieved with 5% labeled data; meanwhile, the corresponding accuracy on the indoor human action dataset based on a frequency modulated continuous-wave (FMCW) radar is (91.3±1.9)%. The proposed method outperforms supervised learning methods, including the use of Bi-LSTM, LH-ViT, and MFAFN, as well as semisupervised learning methods, including the use of FixMatch, C-TGAN, MF-Match, and LW-HGR. Experimental results demonstrate that the proposed method exhibits stable performance across two different radar systems (random code and FMCW radars) and two detection scenarios (through-wall and indoor scenarios), validating its cross-system and cross-scenario adaptability. Finally, the semisupervised learning model based on multidomain collaborative training has 1.30M parameters, requires 26.16M floating-point operations, and exhibits a size of 5.01 MB, demonstrating high computational efficiency.
-
表 1 CNN分类器的结构参数
Table 1. Structural parameters of CNN classifiers
层 配置 输出维度 Conv1 7×7, 16通道, BN, ReLU, MaxPool, Dropout 16×H×W Conv2 5×5, 32通道, BN, ReLU, MaxPool, Dropout 32×H×W FC1 128, BN, ReLU, Dropout 128 FC2 64, BN, ReLU, Dropout 64 FC3 K 10/6 表 2 两个数据集样本的统计信息
Table 2. Statistical information of two dataset samples
数据集 动作类别 原始样本 增强后样本 数据分布 增强倍数 穿墙人体动作数据集 10 1200 10800 均衡 9倍 室内人体动作数据集 6 1754 15786 不均衡 9倍 表 3 基于穿墙人体动作数据集的融合策略对比实验 (%)
Table 3. Comparison experiment of fusion strategies based on through-wall human action dataset (%)
融合方式 标注比例 3% 5% 10% 15% 特征级融合 80.2 84.6 88.9 92.0 本文方法 84.8 93.6 94.8 98.2 注:表中加粗数值表示最优。 表 4 基于穿墙人体动作数据集的权重机制对比实验 (%)
Table 4. Comparison experiment of weighting mechanisms based on through-wall human action dataset (%)
权重机制 标注比例 3% 5% 10% 15% 均匀权重 79.6 87.6 89.3 91.2 自适应加权 82.1 89.1 91.5 94.9 本文方法 84.8 93.6 94.8 98.2 注:表中加粗数值表示最优。 表 5 基于穿墙人体动作数据集的多域协同方法消融实验 (%)
Table 5. Ablation experiment of multi-domain collaborative method based on through-wall human action dataset (%)
实验配置 标注比例 3% 5% 10% 15% 单域 ST-DF 68.8±3.7 75.6±3.1 77.3±2.5 83.9±2.2 R-DF 72.1±3.2 78.3±2.8 83.5±2.3 85.9±1.9 ST-R 74.5±3.3 80.6±2.6 84.3±2.1 86.7±1.7 双域 ST-DF+R-DF 75.4±3.2 79.7±2.7 84.9±2.2 87.5±1.8 ST-R+ST-DF 75.2±2.9 82.3±2.4 86.3±2.0 88.9±1.8 ST-R+R-DF 77.6±2.8 83.1±2.3 86.8±1.9 89.6±1.6 多域 多域 79.2±2.9 84.9±2.5 89.6±1.9 92.6±1.3 多域+特征对齐 83.1±2.6 87.4±2.3 91.7±1.7 95.6±1.5 多域+动态阈值 80.4±2.8 85.2±2.3 90.7±1.8 94.4±1.3 本文方法 84.8±2.5 93.6±1.6 94.8±1.5 98.2±1.2 注:表中加粗数值表示最优。 表 6 基于穿墙人体动作数据集的HAR方法性能对比 (%)
Table 6. Performance comparison of various HAR methods based on through-wall human action dataset (%)
类型 方法 输入 标注比例 3% 5% 10% 15% 监督 Bi-LSTM[13] ST-DF 41.9±5.8 44.2±5.3 54.7±4.5 64.4±3.6 ST-R 69.3±4.2 73.0±3.7 79.5±3.1 85.3±2.5 R-DF 59.5±4.9 68.2±4.1 68.5±4.2 76.8±3.3 LH-ViT[18] ST-DF 36.8±6.5 48.2±5.7 53.9±4.8 62.2±3.8 ST-R 81.0±3.0 84.3±2.6 88.2±2.2 91.1±2.0 R-DF 85.1±2.5 91.3±2.1 91.7±2.0 93.2±1.7 MFAFN[29] ST-DF+
ST-R81.3±2.3 86.5±2.4 92.2±1.7 94.5±1.4 半监督 FixMatch[32] ST-DF 64.3±4.0 74.7±3.3 83.6±2.6 87.7±2.1 ST-R 83.7±2.6 90.3±2.2 91.9±2.2 93.0±1.6 R-DF 84.5±2.7 88.7±2.3 90.4±2.0 92.7±1.7 C-TGAN[21] ST-DF 49.5±5.1 60.0±4.4 68.7±3.6 76.0±3.0 ST-R 78.2±3.2 83.9±2.7 87.4±2.7 92.0±2.1 R-DF 84.4±2.5 89.3±2.2 91.6±1.9 93.3±1.9 MF-Match[22] ST-DF 71.6±3.7 75.9±3.2 86.9±2.3 91.9±1.8 ST-R 85.6±2.9 91.4±2.1 93.7±1.9 95.4±1.3 R-DF 84.7±2.9 89.1±2.3 91.5±1.9 93.1±1.7 LW-HGR[30] ST-DF
+ST-R80.0±2.0 83.2±2.1 87.8±2.0 91.3±1.8 本文方法 多域 84.8±2.5 93.6±1.6 94.8±1.5 98.2±1.2 注:表中加粗数值表示最优。 表 7 基于室内人体动作数据集的HAR方法性能与计算效率对比 (%)
Table 7. Performance and computational efficiency comparison of various HAR methods based on indoor human action dataset (%)
类型 方法 输入 标注比例 计算效率 3% 5% 10% 15% 参数量 浮点计算量 模型大小 推理时间 监
督Bi-LSTM[13] ST-DF 64.3±3.3 71.5±3.0 73.2±2.9 83.4±2.4 0.37M 0.13G 1.27 MB 3.46 ms LH-ViT[18] ST-DF 79.9±2.3 81.2±2.0 84.5±2.2 88.9±2.1 1.41M 5.44G 5.38 MB 5.79 ms MFAFN[29] ST-DF
+ST-R78.5±2.5 81.9±2.2 83.7±1.9 91.2±1.6 21.79M 5.14G 83.21 MB 7.57 ms 半
监
督FixMatch[32] ST-DF 80.8±2.4 82.3±2.3 85.7±1.8 89.8±1.8 11.17M 8.95G 42.70 MB 6.65 ms C-TGAN[21] ST-DF 82.2±2.8 84.5±2.6 89.9±2.3 92.8±1.9 1.55M 3.58G 5.93 MB 2.69 ms MF-Match[22] ST-DF 82.9±2.3 85.7±2.2 90.5±1.8 93.8±1.6 22.52M 8.96G 86.06 MB 6.89 ms LW-HGR[30] ST-DF
+ST-R77.8±2.2 82.1±2.1 85.6±1.9 90.6±1.8 0.21M 9.62M 0.81 MB 2.51 ms 本文方法 多域 83.8±2.4 91.3±1.9 93.2±1.6 95.6±1.4 1.30M 26.16M 5.01 MB 3.09 ms 注:表中加粗数值表示最优。 -
[1] LI Xinyu, HE Yuan, and JING Xiaojun. A survey of deep learning-based human activity recognition in radar[J]. Remote Sensing, 2019, 11(9): 1068. doi: 10.3390/rs11091068. [2] 丁一鹏, 厍彦龙. 穿墙雷达人体动作识别技术的研究现状与展望[J]. 电子与信息学报, 2022, 44(4): 1156–1175. doi: 10.11999/JEIT211051.DING Yipeng and SHE Yanlong. Research status and prospect of human movement recognition technique using through-wall radar[J]. Journal of Electronics & Information Technology, 2022, 44(4): 1156–1175. doi: 10.11999/JEIT211051. [3] JARAMILLO I E, JEONG J G, LOPEZ P R, et al. Real-time human activity recognition with IMU and encoder sensors in wearable exoskeleton robot via deep learning networks[J]. Sensors, 2022, 22(24): 9690. doi: 10.3390/s22249690. [4] WANG Zhengwei, SHE Qi, and SMOLIC A. ACTION-Net: Multipath excitation for action recognition[C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 13214–13223. doi: 10.1109/CVPR46437.2021.01301. [5] GUAN Qiuju, YIN Xuguang, GUO Xuemei, et al. A novel infrared motion sensing system for compressive classification of physical activity[J]. IEEE Sensors Journal, 2016, 16(8): 2251–2259. doi: 10.1109/JSEN.2016.2514606. [6] 杨小鹏, 高炜程, 渠晓东. 基于微多普勒角点特征与Non-Local机制的穿墙雷达人体步态异常终止行为辨识技术[J]. 雷达学报(中英文), 2024, 13(1): 68–86. doi: 10.12000/JR23181.YANG Xiaopeng, GAO Weicheng, and QU Xiaodong. Human anomalous gait termination recognition via through-the-wall radar based on micro-Doppler corner features and Non-Local mechanism[J]. Journal of Radars, 2024, 13(1): 68–86. doi: 10.12000/JR23181. [7] SONG Yongkun, DAI Yongpeng, JIN Tian, et al. Dual-task human activity sensing for pose reconstruction and action recognition using 4-D imaging radar[J]. IEEE Sensors Journal, 2023, 23(19): 23927–23940. doi: 10.1109/JSEN.2023.3308788. [8] AMIN M G, ZHANG Y D, AHMAD F, et al. Radar signal processing for elderly fall detection: The future for in-home monitoring[J]. IEEE Signal Processing Magazine, 2016, 33(2): 71–80. doi: 10.1109/MSP.2015.2502784. [9] 金添, 李志, 戴永鹏, 等. RIS-4D生物雷达多人体定位与生命体征监测[J]. 信号处理, 2024, 40(2): 225–235. doi: 10.16798/j.issn.1003-0530.2024.02.001.JIN Tian, LI Zhi, DAI Yongpeng, et al. Multi-subject localization and vital-sign monitoring with RIS-4D bioradar[J]. Journal of Signal Processing, 2024, 40(2): 225–235. doi: 10.16798/j.issn.1003-0530.2024.02.001. [10] KIM Y and LING Hao. Human activity classification based on micro-Doppler signatures using a support vector machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(5): 1328–1337. doi: 10.1109/TGRS.2009.2012849. [11] CHOWDHURY A, DAS T, RANI S, et al. Activity recognition using ultra wide band range-time scan[C]. 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, Netherlands, 2021: 1338–1342. doi: 10.23919/Eusipco47968.2020.9287598. [12] WANG Mingyang, ZHANG Y D, and CUI Guolong. Human motion recognition exploiting radar with stacked recurrent neural network[J]. Digital Signal Processing, 2019, 87: 125–131. doi: 10.1016/j.dsp.2019.01.013. [13] SHRESTHA A, LI Haobo, LE KERNEC J, et al. Continuous human activity classification from FMCW radar with Bi-LSTM networks[J]. IEEE Sensors Journal, 2020, 20(22): 13607–13619. doi: 10.1109/JSEN.2020.3006386. [14] NGUYEN N, NGUYEN T, PHAM M, et al. Improving human activity classification based on micro-Doppler signatures separation of FMCW radar[C]. 2023 12th International Conference on Control, Automation and Information Sciences (ICCAIS), Hanoi, Vietnam, 2023: 454–459. doi: 10.1109/ICCAIS59597.2023.10382332. [15] 蒋留兵, 魏光萌, 车俐. 基于卷积神经网络的雷达人体动作识别方法[J]. 计算机应用与软件, 2019, 36(11): 168–174,234. doi: 10.3969/j.issn.1000-386x.2019.11.028.JIANG Liubing, WEI Guangmeng, and CHE Li. Human motion recognition method by radar based on CNN[J]. Computer Applications and Software, 2019, 36(11): 168–174,234. doi: 10.3969/j.issn.1000-386x.2019.11.028. [16] ZHENG Zhijie, PAN Jun, NI Zhikang, et al. Human posture reconstruction for through-the-wall radar imaging using convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 3505205. doi: 10.1109/LGRS.2021.3073073. [17] 渠晓东, 王文远, 孟昊宇, 等. 基于鲁棒主成分分析及YOLOv8的穿墙雷达运动人员检测方法[J]. 信号处理, 2025, 41(8): 1390–1403. doi: 10.12466/xhcl.2025.08.008.QU Xiaodong, WANG Wenyuan, MENG Haoyu, et al. Moving human detection method in through-the-wall radar based on robust principal component analysis and YOLOv8[J]. Journal of Signal Processing, 2025, 41(8): 1390–1403. doi: 10.12466/xhcl.2025.08.008. [18] HUAN Sha, WANG Zhaoyue, WANG Xiaoqiang, et al. A lightweight hybrid vision transformer network for radar-based human activity recognition[J]. Scientific Reports, 2023, 13(1): 17996. doi: 10.1038/s41598-023-45149-5. [19] WU Yizhuo, FIORANELLI F, and GAO Chang. RadMamba: Efficient human activity recognition through radar-based micro-Doppler-oriented Mamba state-space model[EB/OL]. https://arxiv.org/abs/2504.12039, 2025. [20] LI Xinyu, HE Yuan, FIORANELLI F, et al. Semisupervised human activity recognition with radar micro-Doppler signatures[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5103112. doi: 10.1109/TGRS.2021.3090106. [21] LIU Li, WANG Shengyao, SONG Chenyan, et al. Radar-based human motion recognition using semisupervised triple-GAN[J]. IEEE Sensors Journal, 2023, 23(24): 30691–30702. doi: 10.1109/JSEN.2023.3327963. [22] YUN Tianhe and WANG Zhangang. MF-Match: A semi-supervised model for human action recognition[J]. Sensors, 2024, 24(15): 4940. doi: 10.3390/s24154940. [23] SADEGHI-ADL Z and AHMAD F. Semi-supervised convolutional autoencoder with attention mechanism for activity recognition[C]. 2023 31st European Signal Processing Conference, Helsinki, Finland, 2023: 785–789. doi: 10.23919/EUSIPCO58844.2023.10289719. [24] PINYOANUNTAPONG E, ALI A, JAKKALA K, et al. GaitSADA: Self-aligned domain adaptation for mmWave gait recognition[C]. 2023 IEEE 20th International Conference on Mobile Ad Hoc and Smart Systems (MASS), Toronto, Canada, 2023: 218–226. doi: 10.1109/MASS58611.2023.00034. [25] XU Hang, LI Yong, LI Yingxin, et al. Through-wall human motion recognition using random code radar sensor with multi-domain feature fusion[J]. IEEE Sensors Journal, 2022, 22(15): 15123–15132. doi: 10.1109/JSEN.2022.3183292. [26] XU Hang, LI Yong, DONG Qingran, et al. Random code radar with range-time-frequency points and improved PointConv network for through-wall human action recognition[J]. IEEE Sensors Journal, 2025, 25(8): 13719–13728. doi: 10.1109/JSEN.2025.3548121. [27] DING Chuanwei, ZHANG Li, CHEN Haoyu, et al. Sparsity-based human activity recognition with PointNet using a portable FMCW radar[J]. IEEE Internet of Things Journal, 2023, 10(11): 10024–10037. doi: 10.1109/JIOT.2023.3235808. [28] DING Wen, GUO Xuemei, and WANG Guoli. Radar-based human activity recognition using hybrid neural network model with multidomain fusion[J]. IEEE Transactions on Aerospace and Electronic Systems, 2021, 57(5): 2889–2898. doi: 10.1109/TAES.2021.3068436. [29] CAO Lin, LIANG Song, ZHAO Zongmin, et al. Human activity recognition method based on FMCW radar sensor with multi-domain feature attention fusion network[J]. Sensors, 2023, 23(11): 5100. doi: 10.3390/s23115100. [30] WU Yajie, WANG Xiang, GUO Shisheng, et al. A lightweight network with multifeature fusion for mmWave radar-based hand gesture recognition[J]. IEEE Sensors Journal, 2024, 24(12): 19553–19561. doi: 10.1109/JSEN.2024.3395638. [31] BU Yuqing, WANG Xiang, ZHANG Bo, et al. Multidomain fusion method for human head movement recognition[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 2504608. doi: 10.1109/TIM.2023.3238750. [32] SOHN K, BERTHELOT D, LI Chunliang, et al. FixMatch: Simplifying semi-supervised learning with consistency and confidence[C]. The 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 51. [33] BLUM A and MITCHELL T. Combining labeled and unlabeled data with co-training[C]. The Eleventh Annual Conference on Computational Learning Theory, Madison, USA, 1998: 92–100. doi: 10.1145/279943.279962. [34] XU Yi, SHANG Lei, YE Jinxing, et al. Dash: Semi-supervised learning with dynamic thresholding[C]. The 38th International Conference on Machine Learning, Virtual Event, 2021: 11525–11536. [35] ZHU Rui, LV Songlin, WANG Zikang, et al. Bi-CoG: Bi-consistency-guided self-training for vision-language models[EB/OL]. https://arxiv.org/abs/2510.20477, 2025. [36] FIORANELLI F, SHAH S A, LI H, et al. Radar signatures of human activities[DS/OL]. University of Glasgow. https://doi.org/10.5525/gla.researchdata.848, 2019. -
作者中心
专家审稿
责编办公
编辑办公
下载: