RM Operator Learning-driven Non-line-of-sight 3D Imaging Method for Millimeter Wave Radar
-
摘要: 非视距(NLOS)毫米波雷达三维成像利用电磁波反射、衍射、散射、穿透等传播特性,实现对隐蔽环境目标的探测、定位和成像,在无人驾驶、灾害救援、城市作战等领域具有重要应用潜力。然而,受实际非视距场景中反射面、遮挡面等不确定性引入的相位误差、孔径遮蔽、多径效应影响,雷达成像出现分辨率差、伪影增多等问题。针对上述问题,结合深度展开网络和环境先验感知,本文提出了一种基于距离徙动(RM)算子学习驱动的非视距毫米波雷达三维成像方法。首先,建立了拐角(LAC)场景下非视距毫米波雷达三维成像模型,引入RM核算子提高成像效率,降低计算复杂度;其次,构建了一种基于快速迭代收缩阈值(FISTA)框架的高精度非视距三维成像网络,利用非视距场景特性,将算法参数设计为网络权重的函数,实现非视距目标高精度、高效率三维重构;最后,搭建了近场非视距毫米波雷达成像平台,完成了理想与非理想反射面场景下金属字母“O”,“S”以及埃菲尔铁塔模型、人造卫星模型等目标的实验验证,结果表明所提方法在提升三维成像精度的同时,运行速度较传统稀疏成像算法提升了两个数量级。Abstract: Non-Line-Of-Sight (NLOS) millimeter wave radar 3D imaging leverages electromagnetic wave propagation characteristics such as reflection, diffraction, scattering, and penetration to detect, locate, and image hidden targets in occluded environments. It holds significant potential for applications in autonomous driving, disaster rescue, and urban warfare. However, uncertainties introduced by reflection surfaces and occluding objects in practical NLOS scenarios, such as phase errors, ghost targets, and aperture shadowing, lead to issues such as blurred imaging and increased artifacts. To address these challenges, this studty proposes a 3D imaging method for NLOS millimeter wave radar based on Range Migration (RM) operator learning, leveraging the adaptive optimization properties of deep unfolding networks and prior environmental perception. First, a 3D imaging model for NLOS millimeter wave radar in Looking Around Corner (LAC) scenarios is established. An RM kernel operator is introduced to enhance imaging efficiency and reduce computational complexity. Second, a high-precision NLOS 3D imaging network is constructed based on the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) framework. Utilizing features specific to NLOS scenes and designing algorithm parameters as functions of network weights, the method achieves high-precision, high-efficiency 3D reconstruction of NLOS targets. Finally, a near-field NLOS millimeter wave radar imaging platform is developed. Experimental validations are performed on targets, including metal letters “O” and “S,” an Eiffel Tower model, and an artificial satellite model, under both ideal and non-ideal reflection surface conditions. The results demonstrate that the proposed method significantly improves 3D imaging precision, achieving a two-orders-of-magnitude increase in computational speed over traditional sparse imaging algorithms.
-
1 基于RM核算子的FISTA稀疏成像算法
1. FISTA sparse imaging algorithm based on RM kernel operator
输入:稀疏降采样非视距回波$ \boldsymbol{Y} $,迭代步长$ \rho $,迭代次数T,正
则化参数$ \eta $,迭代误差$ \varepsilon $输出:非视距目标重构结果$ {\boldsymbol{X}}^{\left(T\right)}\in {\mathbb{C}}^{W\times H} $ 初始化:$ t=1 $,$ {\boldsymbol{Z}}^{\left(1\right)}={\boldsymbol{X}}^{\left(0\right)}={\text{RM}}^{\dagger }\left(\boldsymbol{Y}\right) $,$ {c}^{\left(1\right)}=1 $; 循环开始 (1) 更新动量修正量残差:$ {\boldsymbol{V}}^{\left(t\right)}=\boldsymbol{Y}-\text{RM}\left({\boldsymbol{Z}}^{\left(t\right)}\right) $; (2) 梯度下降粗估计:$ {\boldsymbol{R}}^{\left(t\right)}={\boldsymbol{Z}}^{\left(t\right)}+\rho {\text{RM}}^{\dagger }\left({\boldsymbol{V}}^{\left(t\right)}\right) $; (3) 软阈值收缩:$ {\boldsymbol{X}}^{\left(t\right)}=\text{soft}\left({\boldsymbol{R}}^{\left(t\right)},\eta \right) $; (4) 更新动量修正系数:$ {c}^{\left(t+1\right)}=\dfrac{1+\sqrt{1+4{\left({c}^{\left(t\right)}\right)}^{2}}}{2} $; (5) 更新动量修正量:
$ {\boldsymbol{Z}}^{\left(t+1\right)}={\boldsymbol{X}}^{\left(t\right)}+\left(\dfrac{{c}^{\left(t\right)}-1}{{c}^{\left(t+1\right)}}\right)\left({\boldsymbol{X}}^{\left(t\right)}-{\boldsymbol{X}}^{\left(t-1\right)}\right) $;(6) 终止准则判定:若$ \dfrac{\left|\left|{\boldsymbol{X}}^{\left(t+1\right)}-{\boldsymbol{X}}^{\left(t\right)}\right|\right|}{\left|\left|{\boldsymbol{X}}^{\left(t+1\right)}\right|\right|} > \varepsilon $则$ t=t+1 $;否
则,结束循环;(7) 迭代判定:若$ t\leq T $,则重复步骤(1)–步骤(7);否则,结束
循环。循环结束 表 1 非视距毫米波雷达成像试验系统主要参数
Table 1. Main parameters of the experimental NLOS millimeter wave radar imaging system
参数 实测系统值 载频(GHz)
调频斜率(MHz/μs)79
70.295带宽(GHz) 3.998 X轴合成孔径长度(mm) 400 X轴天线阵元间距(mm) 1 Z轴合成孔径长度(mm) 400 Z轴天线阵元间距(mm) 2 脉冲发射间隔(ms) 25 表 2 4个目标的非视距毫米波三维成像试验数据数值评估结果
Table 2. Numerical evaluation results of experimental four-target NLOS 3D imaging data using millimeter wave radar
目标 反射面 采样率 BP RMA FISTA NLFIST-Net ENT IC Time (s) ENT IC Time (s) ENT IC Time (s) ENT IC Time (s) O 金属板 70% 11.11 7.70 460.83 13.06 7.46 1.15 11.17 12.59 4.13 10.79 15.68 0.017 30% 11.73 6.49 495.48 14.13 3.77 1.14 11.02 14.59 4.67 9.86 25.09 0.016 墙面 70% 11.21 7.59 487.46 13.64 6.39 1.31 11.28 12.45 4.37 11.03 14.35 0.017 30% 11.91 6.28 494.76 14.51 3.32 2.07 11.28 13.66 4.31 10.40 20.20 0.015 S 金属板 70% 11.21 7.04 488.84 13.15 6.94 1.14 11.36 11.43 4.27 10.97 14.28 0.019 30% 11.84 5.90 495.43 14.15 3.66 1.14 11.36 12.61 3.68 10.14 21.78 0.016 墙面 70% 11.24 7.26 498.36 13.52 6.59 1.20 11.86 10.63 4.14 11.07 13.90 0.018 30% 11.92 6.00 490.90 14.42 3.42 1.21 11.39 11.50 4.62 10.53 18.77 0.016 铁塔 金属板 70% 10.68 15.91 543.18 13.93 9.90 0.74 10.90 22.83 4.03 10.08 28.24 0.016 30% 11.93 11.26 540.94 14.89 4.26 0.74 11.09 23.24 4.19 9.57 35.87 0.015 墙面 70% 12.03 10.84 458.58 15.03 3.49 0.82 12.04 16.58 4.24 10.94 23.18 0.016 30% 13.39 4.89 469.50 15.32 1.57 0.74 12.42 12.50 4.29 10.84 24.22 0.017 卫星 金属板 70% 11.00 8.47 542.64 13.22 7.44 0.84 11.32 12.95 4.31 10.79 15.95 0.018 30% 11.71 6.99 479.34 14.25 3.76 0.77 11.42 13.46 4.33 9.92 25.19 0.016 墙面 70% 11.89 6.55 479.34 14.79 3.22 0.79 11.88 11.24 4.46 11.08 14.61 0.018 30% 12.94 4.28 473.34 15.1 1.86 0.73 11.67 12.75 4.08 9.99 25.55 0.017 注:加粗数值代表数值评估最优的算法。 表 3 各成像算法计算复杂度
Table 3. Computational complexity of different imaging algorithms
算法 FLOPs 数值实例 BP $ {N}_{x}{N}_{z}{N}_{\rm r}\times 17{N}_{l}{N}_{\rm r} $ $ 3.65\times {10}^{14} $ RMA $ {N}_{\rm r}{N}_{l}\left(10{\log }_{2}{N}_{l}+6\right) $ $ 1.56\times {10}^{9} $ FISTA $ {N}_{\text{iter}}{N}_{\rm r}{N}_{l}\left(20{\log }_{2}{N}_{l}+24\right) $ $ 4.83\times {10}^{10} $ NLFIST-Net $ T{N}_{\rm r}{N}_{l}\left(20{\log }_{2}{N}_{l}+24\right) $ $ 2.89\times {10}^{10} $ -
[1] 杨建宇. 雷达技术发展规律和宏观趋势分析[J]. 雷达学报, 2012, 1(1): 19–27. doi: 10.3724/SP.J.1300.2012.20010.YANG Jianyu. Development laws and macro trends analysis of radar technology[J]. Journal of Radars, 2012, 1(1): 19–27. doi: 10.3724/SP.J.1300.2012.20010. [2] 丁赤飚, 仇晓兰, 吴一戎. 全息合成孔径雷达的概念、体制和方法[J]. 雷达学报, 2020, 9(3): 399–408. doi: 10.12000/JR20063.DING Chibiao, QIU Xiaolan, and WU Yirong. Concept, system, and method of holographic synthetic aperture radar[J]. Journal of Radars, 2020, 9(3): 399–408. doi: 10.12000/JR20063. [3] 丁赤飚, 仇晓兰, 徐丰, 等. 合成孔径雷达三维成像——从层析、阵列到微波视觉[J]. 雷达学报, 2019, 8(6): 693–709. doi: 10.12000/JR19090.DING Chibiao, QIU Xiaolan, XU Feng, et al. Synthetic aperture radar three-dimensional imaging—from TomoSAR and array InSAR to microwave vision[J]. Journal of Radars, 2019, 8(6): 693–709. doi: 10.12000/JR19090. [4] 孔令讲, 郭世盛, 陈家辉, 等. 多径利用雷达目标探测技术综述与展望[J]. 雷达学报(中英文), 2024, 13(1): 23–45. doi: 10.12000/JR23134.KONG Lingjiang, GUO Shisheng, CHEN Jiahui, et al. Overview and prospects of multipath exploitation radar target detection technology[J]. Journal of Radars, 2024, 13(1): 23–45. doi: 10.12000/JR23134. [5] SMITH G E and MOBASSERI B G. Multipath exploitation for radar target classification[C]. 2012 IEEE Radar Conference, Atlanta, USA, 2012: 623–628. doi: 10.1109/RADAR.2012.6212215. [6] RASKAR R and DAVIS J. 5d time-light transport matrix: What can we reason about scene properties?[OL]. https://dspace.mit.edu/bitstream/handle/1721.1/67888/TransientLightReportRaskarDavis.pdf?sequence=1. [7] KIRMANI A, HUTCHISON T, DAVIS J, et al. Looking around the corner using transient imaging[C]. 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 2009: 159–166. doi: 10.1109/ICCV.2009.5459160. [8] VELTEN A, WILLWACHER T, GUPTA O, et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging[J]. Nature communications, 2012, 3(1): 745. doi: 10.1038/ncomms1747. [9] KATZ O, HEIDMANN P, FINK M, et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations[J]. Nature photonics, 2014, 8(10): 784–790. doi: 10.1038/nphoton.2014.189. [10] YEDIDIA A B, BARADAD M, THRAMPOULIDIS C, et al. Using unknown occluders to recover hidden scenes[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 12223–12231. doi: 10.1109/CVPR.2019.01251. [11] LIU Xintong, WANG Jianyu, LI Zhupeng, et al. Non-line-of-sight reconstruction with signal-object collaborative regularization[J]. Light: Science & Applications, 2021, 10(1): 198. doi: 10.1038/s41377-021-00633-3. [12] HAO Mingyang, NING Fangli, WANG Ke et al. Acoustic non-line-of-sight vehicle approaching and leaving detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8): 9979–9991. doi: 10.1109/TITS.2024.3353749. [13] MAEDA T, WANG Yiqin, RASKAR R et al. Thermal non-line-of-sight imaging[C]. 2019 IEEE International Conference on Computational Photography (ICCP), Tokyo, Japan, 2019: 1–11. doi: 10.1109/ICCPHOT.2019.8747343. [14] ZHOU Zimu, YANG Zheng, WU Chenshu et al. WiFi-based indoor line-of-sight identification[J]. IEEE Transactions on Wireless Communications, 2015, 14(11): 6125–6136. doi: 10.1109/TWC.2015.2448540. [15] PAULI M, GÖTTEL B, SCHERR S, et al. Miniaturized millimeter-wave radar sensor for high-accuracy applications[J]. IEEE Transactions on Microwave Theory and Techniques, 2017, 65(5): 1707–1715. doi: 10.1109/TMTT.2017.2677910. [16] WANG Xiao, XU Linhai, SUN Hongbin, et al. On-road vehicle detection and tracking using MMW radar and monovision fusion[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(7): 2075–2084. doi: 10.1109/TITS.2016.2533542. [17] GUO Shisheng, ZHAO Qingsong, CUI Guolong, et al. Behind corner targets location using small aperture millimeter wave radar in nlos urban environment[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 460–470. doi: 10.1109/JSTARS.2020.2963924. [18] CHEN Jiahui, ZHANG Yang, GUO Shisheng, et al. Joint estimation of nlos building layout and targets via sparsity-driven approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5114513. doi: 10.1109/TGRS.2022.3182429. [19] WEI Shunjun, WEI Jinshan, LIU Xinyuan, et al. Nonline-of-sight 3-D imaging using millimeter-wave radar[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5106518. doi: 10.1109/TGRS.2021.3112579. [20] 林雨青, 仇晓兰, 彭凌霄, 等. 基于多径模型的建筑区SAR三维成像中非视距目标重定位方法[J]. 雷达学报(中英文), 2024, 13(4): 777–790. doi: 10.12000/JR24057.LIN Yuqing, QIU Xiaolan, PENG Lingxiao, et al. Non-line-of-sight target relocation by multipath model in SAR 3D urban area imaging[J]. Journal of Radars, 2024, 13(4): 777–790. doi: 10.12000/JR24057. [21] WEN Yanbo, WEI Shunjun, WEI Jinshan, et al. Non-line-of-sight imaging of hidden moving target using millimeter-wave inverse synthetic aperture radar[C]. IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 2022: 555–558. doi: 10.1109/IGARSS46834.2022.9883939. [22] TAN Xiaoheng, YANG Zhijun, LI Dong, et al. An efficient range-Doppler domain ISAR imaging approach for rapidly spinning targets[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4): 2670–2681. doi: 10.1109/TGRS.2019.2953303. [23] 邢孟道, 马鹏辉, 楼屹杉, 等. 合成孔径雷达快速后向投影算法综述[J]. 雷达学报(中英文), 2024, 13(1): 1–22. doi: 10.12000/JR23183.XING Mengdao, MA Penghui, LOU Yishan, et al. Review of fast back projection algorithms in synthetic aperture radar[J]. Journal of Radars, 2024, 13(1): 1–22. doi: 10.12000/JR23183. [24] DONOHO D L. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306. doi: 10.1109/TIT.2006.871582. [25] CETIN M and KARL W C. Feature-enhanced synthetic aperture radar image formation based on nonquadratic regularization[J]. IEEE Transactions on Image Processing, 2001, 10(4): 623–631. doi: 10.1109/83.913596. [26] ÁLVAREZ Y, RODRIGUEZ-VAQUEIRO Y, GONZALEZ-VALDES B, et al. Three-dimensional compressed sensing-based millimeter-wave imaging[J]. IEEE Transactions on Antennas and Propagation, 2015, 63(12): 5868–5873. doi: 10.1109/TAP.2015.2481487. [27] BARANIUK R and STEEGHS P. Compressive radar imaging[C]. 2007 IEEE Radar Conference, Waltham, USA, 2007: 128–133. doi: 10.1109/RADAR.2007.374203. [28] ZHU Xiaoxiang and BAMLER R. Tomographic SAR Inversion by L1 -norm regularization—the compressive sensing approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(10): 3839–3846. doi: 10.1109/TGRS.2010.2048117. [29] WEI Shunjun, ZHANG Xiaoling, SHI Jun, et al. Sparse reconstruction for SAR imaging based on compressed sensing[J]. Progress in Electromagnetics Research, 2010, 109: 63–81. doi: 10.2528/PIER10080805. [30] LIU Xinyuan, WEI Shunjun, WEI Jinshan, et al. Non-line-of-sight millimeter-wave radar 3-D sparse reconstruct via MSSTV method[C]. 2022 IEEE 9th International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications (MAPE), Chengdu, China, 2022: 424–427. doi: 10.1109/MAPE53743.2022.9935195. [31] 蔡响, 韦顺军, 文彦博, 等. 基于非视距雷达三维成像的隐藏目标精确重构方法[J]. 雷达学报(中英文), 2024, 13(4): 791–806. doi: 10.12000/JR24060.CAI Xiang, WEI Shunjun, WEN Yanbo, et al. Precise reconstruction method for hidden targets based on non-line-of-sight radar 3D imaging[J]. Journal of Radars, 2024, 13(4): 791–806. doi: 10.12000/JR24060. [32] CAI Xiang, WEI Shunjun, WEN Yanbo, et al. Bayesian-based 3-D MMW radar imaging of non-line-of-sight environments[C]. 2023 Cross Strait Radio Science and Wireless Technology Conference (CSRSWTC), Guilin, China, 2023: 1–3. doi: 10.1109/CSRSWTC60855.2023.10427181. [33] WEN Yanbo, WEI Shunjun, CAI Xiang, et al. NMTCS: Non-line-of-sight sparse reconstruct for hidden motion targets[C]. 2023 Cross Strait Radio Science and Wireless Technology Conference (CSRSWTC), Guilin, China, 2023: 1–3. doi: 10.1109/CSRSWTC 60855.2023.10426822. [34] WEN Yanbo, WEI Shunjun, CAI xiang, et al. CMTI: Non-line-of-sight radar imaging for non-cooperative corner motion target[J]. IEEE Transactions on Vehicular Technology, 2025, 74(1): 179–190. doi: 10.1109/TVT.2024.3398218. [35] WEN Yanbo, WEI Shunjun, CAI xiang, et al. Non-line-of-sight sparse aperture ISAR imaging via a novel detail-aware regularization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5220218. doi: 10.1109/TGRS.2024.3447900. [36] 董小舒, 朱伟, 刘羽, 等. 毫米波雷达与视觉融合的车辆目标检测系统[J]. 指挥信息系统与技术, 2021, 12(1): 91–96. doi: 10.15908/j.cnki.cist.2021.01.017.DONG Xiaoshu, ZHU Wei, LIU Yu, et al. Vehicle object detection system combined of millimeter-wave radar and vision[J]. Command Information System And Technology, 2021, 12(1): 91–96. doi: 10.15908/j.cnki.cist.2021.01.017. [37] 张春杰, 陈奇, 赵佳琦. 基于注意力机制CNN-LSTM的毫米波雷达点云特征数据预测生成[J]. 电讯技术, 2024, 64(11): 1718–1725. doi: 10.20079/j.issn.1001-893x.240118005.ZHANG Chunjie, CHEN Qi, and ZHAO Jiaqi. Enhancement of mmwave radar point cloud feature data based on self-attention echanism CNN-LSTM[J]. Telecommunication Engineering, 2024, 64(11): 1718–1725. doi: 10.20079/j.issn.1001-893x.240118005. [38] GREGOR K and LECUN Y. Learning fast approximations of sparse coding[C]. The 27th International Conference on International Conference on Machine Learning, Haifa, Israel, 2010: 399–406. [39] LIU Risheng, ZHANG Yuxi, CHENG Shichao, et al. A deep framework assembling principled modules for CS-MRI: Unrolling perspective, convergence behaviors, and practical modeling[J]. IEEE Transactions on Medical Imaging, 2020, 39(12): 4150–4163. doi: 10.1109/TMI.2020.3014193. [40] 王谋, 韦顺军, 沈蓉, 等. 基于自学习稀疏先验的三维SAR成像方法[J]. 雷达学报, 2023, 12(1): 36–52. doi: 10.12000/JR22101.WANG Mou, WEI Shunjun, SHEN Rong, et al. 3D SAR imaging method based on learned sparse prior[J]. Journal of Radars, 2023, 12(1): 36–52. doi: 10.12000/JR22101. [41] WANG Mou, WEI Shunjun, LIANG Jiadian, et al. RMIST-Net: Joint range migration and sparse reconstruction network for 3-D mmW imaging[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5205117. doi: 10.1109/TGRS.2021.3068405. [42] WANG Mou, WEI Shunjun, ZHOU Zichen, et al. CTV-Net: Complex-valued TV-driven network with nested topology for 3-D SAR imaging[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(4): 5588–5602. doi: 10.1109/TNNLS.2022.3208252. [43] WANG Mou, WEI Shunjun, SHI Jun, et al. CSR-Net: A novel complex-valued network for fast and precise 3-D microwave sparse reconstruction[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 4476–4492. doi: 10.1109/JSTARS.2020.3014696. [44] PU Wei. SAE-Net: A deep neural network for SAR autofocus[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5220714. doi: 10.1109/TGRS.2021.3139914. [45] WANG Yingzhou, LI Lijun, and GONG Ke. Narrowband experimental study on millimeter-wave indoor propagation[C]. 1998 International Conference on Communication Technology. Proceedings (IEEE Cat. No.98EX243), Beijing, China, 1998: 5. doi: 10.1109/ICCT.1998.741269. [46] BECK A and TEBOULLE M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J]. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202. doi: 10.1137/080716542. [47] XIANG Jinxi, DONG Yonggui, and YANG Yunjie. FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging[J]. IEEE Transactions on Medical Imaging, 2021, 40(5): 1329–1339. doi: 10.1109/TMI.2021.3054167. [48] ZHOU Yulong, ZHONG Yu, WEI Zhun, et al. An improved deep learning scheme for solving 2-D and 3-D inverse scattering problem[J]. IEEE Transactions on Antennas and Propagation, 2021, 69(5): 2853–2863. doi: 10.1109/TAP.2020.3027898. -
作者中心
专家审稿
责编办公
编辑办公
下载: