Turn off MathJax
Article Contents
Article Navigation >  Journal of Radars  > 2024  > Online First
DING Chuanwei, LIU Zhilin, ZHANG Li, et al. Tangential human posture recognition with sequential images based on MIMO radar[J]. Journal of Radars, in press. doi: 10.12000/JR24116
Citation: DING Chuanwei, LIU Zhilin, ZHANG Li, et al. Tangential human posture recognition with sequential images based on MIMO radar[J]. Journal of Radars, in press. doi: 10.12000/JR24116
shu

Tangential Human Posture Recognition with Sequential Images Based on MIMO Radar

DOI: 10.12000/JR24116
  • 1.

    Nanjing University of Science and Technology, Nanjing 210000, China

  • 2.

    Shanghai Aerospace Electronic Technology Institute, Shanghai 201109, China

Funds:  The National Natural Science Foundation of China (62201259, 62301255), Fundamental Research Funds for the Central Universities (30923011006, 30923011026)
More Information
    Abstract
  • Recent research on radar-based human activity recognition has typically focused on activities that move toward or away from radar in radial directions. Conventional Doppler-based methods can barely describe the true characteristics of nonradial activities, especially static postures or tangential activities, resulting in a considerable decline in recognition performance. To address this issue, a method for recognizing tangential human postures based on sequential images of a Multiple-Input Multiple-Output (MIMO) radar system is proposed. A time sequence of high-quality images is achieved to describe the structure of the human body and corresponding dynamic changes, where spatial and temporal features are extracted to enhance the recognition performance. First, a Constant False Alarm Rate (CFAR) algorithm is applied to locate the human target. A sliding window along the slow time axis is then utilized to divide the received signal into sequential frames. Next, a fast Fourier transform and the 2D Capon algorithm are performed on each frame to estimate range, pitch angle, and azimuth angle information, which are fused to create a tangential posture image. They are connected to form a time sequence of tangential posture images. To improve image quality, a modified joint multidomain adaptive threshold–based denoising algorithm is applied to improve the image quality by suppressing noises and enhancing human body outline and structure. Finally, a Spatio-Temporal-Convolution Long Short Term Memory (ST-ConvLSTM) network is designed to process the sequential images. In particular, the ConvLSTM cell is used to extract continuous image features by combining convolution operation with the LSTM cell. Moreover, spatial and temporal attention modules are utilized to emphasize intraframe and interframe focus for improving recognition performance. Extensive experiments show that our proposed method can achieve an accuracy rate of 96.9% in classifying eight typical tangential human postures, demonstrating its feasibility and superiority in tangential human posture recognition.

     

    • FullText(HTML)
  • loading
    • References(42)
  • [1]
    金添, 宋永坤, 戴永鹏, 等. UWB-HA4D-1.0: 超宽带雷达人体动作四维成像数据集[J]. 雷达学报, 2022, 11(1): 27–39. doi: 10.12000/JR22008.

    JIN Tian, SONG Yongkun, DAI Yongpeng, et al. UWB-HA4D-1.0: An ultra-wideband radar human activity 4D imaging dataset[J]. Journal of Radars, 2022, 11(1): 27–39. doi: 10.12000/JR22008.
    [2]
    LE KERNEC J, FIORANELLI F, DING Chuanwei, et al. Radar signal processing for sensing in assisted living: The challenges associated with real-time implementation of emerging algorithms[J]. IEEE Signal Processing Magazine, 2019, 36(4): 29–41. doi: 10.1109/MSP.2019.2903715.
    [3]
    QI Fugui, LV Hao, WANG Jianqi, et al. Quantitative evaluation of channel micro-Doppler capacity for MIMO UWB radar human activity signals based on time-frequency signatures[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(9): 6138–6151. doi: 10.1109/TGRS.2020.2974749.
    [4]
    TANG Longzhen, GUO Shisheng, JIAN Qiang, et al. Through-wall human activity recognition with complex-valued range-time-Doppler feature and region-vectorization ConvGRU[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5111014. doi: 10.1109/TGRS.2023.3329561.
    [5]
    LI Zhi, JIN Tian, LI Lianlin, et al. Spatiotemporal processing for remote sensing of trapped victims using 4-D imaging radar[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5103412. doi: 10.1109/TGRS.2023.3266039.
    [6]
    YANG Shufan, LE KERNEC J, ROMAIN O, et al. The human activity radar challenge: Benchmarking based on the ‘radar signatures of human activities’ dataset from Glasgow university[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(4): 1813–1824. doi: 10.1109/JBHI.2023.3240895.
    [7]
    BAI Xueru, HUI Ye, WANG Li, et al. Radar-based human gait recognition using dual-channel deep convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(12): 9767–9778. doi: 10.1109/TGRS.2019.2929096.
    [8]
    LI Changzhi, PENG Zhengyu, HUANG T Y, et al. A review on recent progress of portable short-range noncontact microwave radar systems[J]. IEEE Transactions on Microwave Theory and Techniques, 2017, 65(5): 1692–1706. doi: 10.1109/TMTT.2017.2650911.
    [9]
    KIM Y and LING Hao. Human activity classification based on micro-Doppler signatures using a support vector machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(5): 1328–1337. doi: 10.1109/TGRS.2009.2012849.
    [10]
    DING Chuanwei, HONG Hong, ZOU Yu, et al. Continuous human motion recognition with a dynamic range-Doppler trajectory method based on FMCW radar[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6821–6831. doi: 10.1109/TGRS.2019.2908758.
    [11]
    EROL B and AMIN M G. Radar data cube processing for human activity recognition using multisubspace learning[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(6): 3617–3628. doi: 10.1109/TAES.2019.2910980.
    [12]
    DING Chuanwei, ZHANG Li, CHEN Haoyu, et al. Sparsity-based human activity recognition with pointnet using a portable FMCW radar[J]. IEEE Internet of Things Journal, 2023, 10(11): 10024–10037. doi: 10.1109/JIOT.2023.3235808.
    [13]
    LI Xinyu, HE Yuan, FIORANELLI F, et al. Semisupervised human activity recognition with radar micro-Doppler signatures[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5103112. doi: 10.1109/TGRS.2021.3090106.
    [14]
    KIM W Y and SEO D H. Radar-based human activity recognition combining range-time-Doppler maps and range-distributed-convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1002311. doi: 10.1109/TGRS.2022.3162833.
    [15]
    QIAO Xingshuai, AMIN M G, SHAN Tao, et al. Human activity classification based on micro-Doppler signatures separation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5105014. doi: 10.1109/TGRS.2021.3105124.
    [16]
    LUO Fei, BODANESE E, KHAN S, et al. Spectro-temporal modeling for human activity recognition using a radar sensor network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5103913. doi: 10.1109/TGRS.2023.3270365.
    [17]
    DING Chuanwei, CHAE R, WANG Jing, et al. Inattentive driving behavior detection based on portable FMCW radar[J]. IEEE Transactions on Microwave Theory and Techniques, 2019, 67(10): 4031–4041. doi: 10.1109/TMTT.2019.2934413.
    [18]
    WANG Bo, ZHANG Hao, and GUO Yongxin. Radar-based soft fall detection using pattern contour vector[J]. IEEE Internet of Things Journal, 2023, 10(3): 2519–2527. doi: 10.1109/JIOT.2022.3213693.
    [19]
    元志安, 周笑宇, 刘心溥, 等. 基于RDSNet的毫米波雷达人体跌倒检测方法[J]. 雷达学报, 2021, 10(4): 656–664. doi: 10.12000/JR21015.

    YUAN Zhi’an, ZHOU Xiaoyu, LIU Xinpu, et al. Human fall detection method using millimeter-wave radar based on RDSNet[J]. Journal of Radars, 2021, 10(4): 656–664. doi: 10.12000/JR21015.
    [20]
    TAHMOUSH D and SILVIOUS J. Radar micro-Doppler for long range front-view gait recognition[C]. 3rd IEEE International Conference on Biometrics: Theory, Applications, and Systems (BTAS), Washington, USA, 2009: 1–6. doi: 10.1109/BTAS.2009.5339049.
    [21]
    ALNUJAIM I, RAM S S, OH D, et al. Synthesis of micro-Doppler signatures of human activities from different aspect angles using generative adversarial networks[J]. IEEE Access, 2021, 9: 46422–46429. doi: 10.1109/ACCESS.2021.3068075.
    [22]
    QI Fugui, LI Zhao, MA Yangyang, et al. Generalization of channel micro-Doppler capacity evaluation for improved finer-grained human activity classification using MIMO UWB radar[J]. IEEE Transactions on Microwave Theory and Techniques, 2021, 69(11): 4748–4761. doi: 10.1109/TMTT.2021.3076055.
    [23]
    FIORANELLI F, PATEL J, GÜRBÜZ S Z, et al. Multistatic human micro-Doppler classification with degraded/jammed radar data[C]. 2019 IEEE Radar Conference (RadarConf), Boston, USA, 2019: 1–6. doi: 10.1109/RADAR.2019.8835618.
    [24]
    THIEL M and SARABANDI K. Ultrawideband multi-static scattering analysis of human movement within buildings for the purpose of stand-off detection and localization[J]. IEEE Transactions on Antennas and Propagation, 2011, 59(4): 1261–1268. doi: 10.1109/TAP.2011.2109349.
    [25]
    FIORANELLI F, RITCHIE M, and GRIFFITHS H. Aspect angle dependence and multistatic data fusion for micro-Doppler classification of armed/unarmed personnel[J]. IET Radar, Sonar & Navigation, 2015, 9(9): 1231–1239. doi: 10.1049/iet-rsn.2015.0058.
    [26]
    FAIRCHILD D P and NARAYANAN R M. Multistatic micro-Doppler radar for determining target orientation and activity classification[J]. IEEE Transactions on Aerospace and Electronic Systems, 2016, 52(1): 512–521. doi: 10.1109/TAES.2015.130595.
    [27]
    FIORANELLI F, RITCHIE M, GÜRBÜZ S Z, et al. Feature diversity for optimized human micro-Doppler classification using multistatic radar[J]. IEEE Transactions on Aerospace and Electronic Systems, 2017, 53(2): 640–654. doi: 10.1109/TAES.2017.2651678.
    [28]
    QIAO Xingshuai, LI Gang, SHAN Tao, et al. Human activity classification based on moving orientation determining using multistatic micro-Doppler radar signals[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5104415. doi: 10.1109/TGRS.2021.3100482.
    [29]
    YU J T, YEN Li, and TSENG P H. mmWave radar-based hand gesture recognition using range-angle image[C]. 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), Antwerp, Belgium, 2020: 1–5. doi: 10.1109/VTC2020-Spring48590.2020.9128573.
    [30]
    ZHAO Yubin, YAROVOY A, and FIORANELLI F. Angle-insensitive human motion and posture recognition based on 4D imaging radar and deep learning classifiers[J]. IEEE Sensors Journal, 2022, 22(12): 12173–12182. doi: 10.1109/JSEN.2022.3175618.
    [31]
    YANG Yang, HOU Chunping, LANG Yue, et al. Omnidirectional motion classification with monostatic radar system using micro-Doppler signatures[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(5): 3574–3587. doi: 10.1109/TGRS.2019.2958178.
    [32]
    YANG Yang, ZHANG Yutong, SONG Chunying, et al. Omnidirectional spectrogram generation for radar-based omnidirectional human activity recognition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5105513. doi: 10.1109/TGRS.2023.3278409.
    [33]
    YANG Yang, LI Junhan, LI Beichen, et al. Few-shot omnidirectional human motion recognition using monostatic radar system[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 2531414. doi: 10.1109/TIM.2023.3328079.
    [34]
    张哲熙. 基于BM3D的图像去噪算法研究[D]. [硕士论文], 西安电子科技大学, 2017.

    ZHANG Zhexi. Research on image denoising algorithm based on BM3D[D]. [Master dissertation], Xidian University, 2017.
    [35]
    DABOV K, FOI A, KATKOVNIK V, et al. Image denoising with block-matching and 3D filtering[C]. Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, San Jose, USA, 2006: 606414. doi: 10.1117/12.643267.
    [36]
    WANG Xinheng, ISTEPANIAN R S H, and SONG Yonghua. Microarray image enhancement by denoising using stationary wavelet transform[J]. IEEE Transactions on Nanobioscience, 2003, 2(4): 184–189. doi: 10.1109/TNB.2003.816225.
    [37]
    王安义, 战金龙, 卢建军. 一种新的二维Capon算法的研究[J]. 西安科技学院学报, 2003, 23(4): 437–440. doi: 10.3969/j.issn.1672-9315.2003.04.023.

    WANG Anyi, ZHAN Jinlong, and LU Jianjun. A new 2-D Capon algorithm[J]. Journal of Xi’an University of Science and Technology, 2003, 23(4): 437–440. doi: 10.3969/j.issn.1672-9315.2003.04.023.
    [38]
    RAHMAN S A and ADJEROH D A. Deep learning using convolutional LSTM estimates biological age from physical activity[J]. Scientific Reports, 2019, 9(1): 11425. doi: 10.1038/s41598-019-46850-0.
    [39]
    SUDHAKARAN S and LANZ O. Convolutional long short-term memory networks for recognizing first person interactions[C]. The IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy, 2017: 2339–2346. doi: 10.1109/ICCVW.2017.276.
    [40]
    KIM Y and MOON T. Human detection and activity classification based on micro-Doppler signatures using deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(1): 8–12. doi: 10.1109/LGRS.2015.2491329.
    [41]
    LI Wenxuan, ZHANG Dongheng, LI Yadong, et al. Real-time fall detection using mmWave radar[C]. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 2022: 16–20. doi: 10.1109/ICASSP43922.2022.9747153.
    [42]
    KIM Y, ALNUJAIM I, and OH D. Human activity classification based on point clouds measured by millimeter wave MIMO radar with deep recurrent neural networks[J]. IEEE Sensors Journal, 2021, 21(12): 13522–13529. doi: 10.1109/JSEN.2021.3068388.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML
    Article views(403) PDF downloads(80) Cited by()
    Proportional views
    Related

    京ICP备20021838号-8   Supported by: Beijing Renhe Information Technology Co. Ltd     百度统计

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return