Volume 14 Issue 4
Aug.  2025
Turn off MathJax
Article Contents
Article Navigation >  Journal of Radars  > 2025  >  14(4): 1005-1018
WEI Zhiqiang, ZHANG Jiashuo, LIU Fan, et al. Low-altitude secure communication driven by deep reinforcement learning: An integrated sensing and communication design[J]. Journal of Radars, 2025, 14(4): 1005–1018. doi: 10.12000/JR25025
Citation: WEI Zhiqiang, ZHANG Jiashuo, LIU Fan, et al. Low-altitude secure communication driven by deep reinforcement learning: An integrated sensing and communication design[J]. Journal of Radars, 2025, 14(4): 1005–1018. doi: 10.12000/JR25025
shu

Low-altitude Secure Communication Driven by Deep Reinforcement Learning: An Integrated Sensing and Communication Design

DOI: 10.12000/JR25025 CSTR: 32380.14.JR25025
  • 1.

    Xi’an Jiaotong University, Xi’an 710049, China

  • 2.

    Southeast University, Nanjing 211189, China

  • 3.

    Beijing Institute of Technology, Beijing 100081, China

Funds:  The National Natural Science Foundation of China (12371464, 62331023, 62471039), Qin Chuang Yuan High-Level Innovation and Entrepreneurship Talent Program (QCYRCXM-2023-094), Guangdong Basic and Applied Basic Research Foundation (2024A1515011218)
More Information
  • Corresponding author: YANG Zai, yangzai@xjtu.edu.cn
  • Received Date: 2025-02-04
  • Rev Recd Date: 2025-06-30
    • Available Online: 2025-07-02
  • Publish Date: 2025-07-09
    Abstract
  • To address the physical layer security challenges in low-altitude Unmanned Aerial Vehicle (UAV) communications, this paper proposes an Integrated Sensing And Communication (ISAC) scheme. For the proposed ISAC scheme, an online optimization framework for UAV trajectory and communication resource allocation is developed using Deep Reinforcement Learning (DRL). In the proposed scheme, artificial noise transmitted by a communication UAV is reused to simultaneously sense and jam a potential eavesdropping UAV, thereby enhancing secure communications for ground users. By estimating and predicting the state of the eavesdropping UAV, the trajectory and resource allocation design problem is reformulated as a Markov decision process. Using the Deep Deterministic Policy Gradient (DDPG) algorithm, the optimal framework is learned over time, dynamically optimizing the communication UAV’s trajectory and resource allocation to maximize long-term sensing and secure communication performance. Simulation results demonstrate that the proposed scheme achieves a superior trade-off between sensing and security without degrading sensing performance and outperforms baseline methods in terms of secure communication performance. This validates the performance gains achieved through sensing and online trajectory design, as well as the potential and superior performance of applying DRL to the integrated design of sensing, communication, and trajectory.

     

    • FullText(HTML)
  • loading
    • References(31)
  • [1]
    MUKHERJEE A, FAKOORIAN S A A, HUANG Jing, et al. Principles of physical layer security in multiuser wireless networks: A survey[J]. IEEE Communications Surveys & Tutorials, 2014, 16(3): 1550–1573. doi: 10.1109/SURV.2014.012314.00178.
    [2]
    LIU Chenxi, LEE J, and QUEK T Q S. Safeguarding UAV communications against full-duplex active eavesdropper[J]. IEEE Transactions on Wireless Communications, 2019, 18(6): 2919–2931. doi: 10.1109/TWC.2019.2906177.
    [3]
    WANG Qian, CHEN Zhi, MEI Weidong, et al. Improving physical layer security using UAV-enabled mobile relaying[J]. IEEE Wireless Communications Letters, 2017, 6(3): 310–313. doi: 10.1109/LWC.2017.2680449.
    [4]
    ZHANG Guangchi, WU Qingqing, CUI Miao, et al. Securing UAV communications via joint trajectory and power control[J]. IEEE Transactions on Wireless Communications, 2019, 18(2): 1376–1389. doi: 10.1109/TWC.2019.2892461.
    [5]
    ZHOU Xiaobo, WU Qingqing, YAN Shihao, et al. UAV-enabled secure communications: Joint trajectory and transmit power optimization[J]. IEEE Transactions on Vehicular Technology, 2019, 68(4): 4069–4073. doi: 10.1109/TVT.2019.2900157.
    [6]
    CUI Miao, ZHANG Guangchi, WU Qingqing, et al. Robust trajectory and transmit power design for secure UAV communications[J]. IEEE Transactions on Vehicular Technology, 2018, 67(9): 9042–9046. doi: 10.1109/TVT.2018.2849644.
    [7]
    CAI Yuanxin, WEI Zhiqiang, LI Ruide, et al. Joint trajectory and resource allocation design for energy-efficient secure UAV communication systems[J]. IEEE Transactions on Communications, 2020, 68(7): 4536–4553. doi: 10.1109/TCOMM.2020.2982152.
    [8]
    LIU Fan, MASOUROS C, PETROPULU A P, et al. Joint radar and communication design: Applications, state-of-the-art, and the road ahead[J]. IEEE Transactions on Communications, 2020, 68(6): 3834–3862. doi: 10.1109/TCOMM.2020.2973976.
    [9]
    ZHANG J A, RAHMAN M L, WU Kai, et al. Enabling joint communication and radar sensing in mobile networks-A survey[J]. IEEE Communications Surveys & Tutorials, 2022, 24(1): 306–345. doi: 10.1109/COMST.2021.3122519.
    [10]
    WEI Zhongxiang, LIU Fan, MASOUROS C, et al. Toward multi-functional 6G wireless networks: Integrating sensing, communication, and security[J]. IEEE Communications Magazine, 2022, 60(4): 65–71. doi: 10.1109/MCOM.002.2100972.
    [11]
    SU Nanchi, LIU Fan, WEI Zhongxiang, et al. Secure dual-functional radar-communication transmission: Exploiting interference for resilience against target eavesdropping[J]. IEEE Transactions on Wireless Communications, 2022, 21(9): 7238–7252. doi: 10.1109/TWC.2022.3156893.
    [12]
    SU Nanchi, LIU Fan, and MASOUROS C. Secure radar-communication systems with malicious targets: Integrating radar, communications and jamming functionalities[J]. IEEE Transactions on Wireless Communications, 2021, 20(1): 83–95. doi: 10.1109/TWC.2020.3023164.
    [13]
    HUANG Chongwen, YANG Zhaohui, ALEXANDROPOULOS G C, et al. Multi-Hop RIS-empowered terahertz communications: A DRL-based hybrid beamforming design[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(6): 1663–1677. doi: 10.1109/JSAC.2021.3071836.
    [14]
    SUN Yan, XU Dongfang, NG D W K, et al. Optimal 3D-trajectory design and resource allocation for solar-powered UAV communication systems[J]. IEEE Transactions on Communications, 2019, 67(6): 4281–4298. doi: 10.1109/TCOMM.2019.2900630.
    [15]
    YOU Changsheng and ZHANG Rui. Hybrid offline-online design for UAV-enabled data harvesting in probabilistic LoS channels[J]. IEEE Transactions on Wireless Communications, 2020, 19(6): 3753–3768. doi: 10.1109/TWC.2020.2978073.
    [16]
    WEI Zhiqiang, LIU Fan, LIU Chang, et al. Integrated sensing, navigation, and communication for secure UAV networks with a mobile eavesdropper[J]. IEEE Transactions on Wireless Communications, 2024, 23(7): 7060–7078. doi: 10.1109/TWC.2023.3337148.
    [17]
    MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing Atari with deep reinforcement learning[J]. arXiv preprint arXiv: 1312.5602, 2013. doi: 10.48550/arXiv.1312.5602.
    [18]
    LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. arXiv preprint arXiv: 1509.02971, 2015. doi: 10.48550/arXiv.1509.02971.
    [19]
    SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv: 1707.06347, 2017. doi: 10.48550/arXiv.1707.06347.
    [20]
    HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor[J]. arXiv preprint arXiv: 1801.01290, 2018. doi: 10.48550/arXiv.1801.01290.
    [21]
    GAN Xu, HUANG Chongwen, YANG Zhaohui, et al. Bayesian learning for double-RIS aided ISAC systems with superimposed pilots and data[J]. IEEE Journal of Selected Topics in Signal Processing, 2024, 18(5): 766–781. doi: 10.1109/JSTSP.2024.3408071.
    [22]
    QIN Yunhui, ZHANG Zhongshan, LI Xulong, et al. Deep reinforcement learning based resource allocation and trajectory planning in integrated sensing and communications UAV network[J]. IEEE Transactions on Wireless Communications, 2023, 22(11): 8158–8169. doi: 10.1109/TWC.2023.3260304.
    [23]
    ZHU Yuqian, LI Ming, LIU Yang, et al. DRL-based joint beamforming and BS-RIS-UE association design for RIS-assisted mmWave networks[C]. 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, USA, 2022: 345–350. doi: 10.1109/WCNC51071.2022.9771607.
    [24]
    DONG Runze, WANG Buhong, TIAN Jiwei, et al. Deep reinforcement learning based UAV for securing mmWave communications[J]. IEEE Transactions on Vehicular Technology, 2023, 72(4): 5429–5434. doi: 10.1109/TVT.2022.3224959.
    [25]
    ZENG Yong, WU Qingqing, and ZHANG Rui. Accessing from the sky: A tutorial on UAV communications for 5G and beyond[J]. Proceedings of the IEEE, 2019, 107(12): 2327–2375. doi: 10.1109/JPROC.2019.2952892.
    [26]
    YOU Changsheng and ZHANG Rui. 3D trajectory optimization in Rician fading for UAV-enabled data harvesting[J]. IEEE Transactions on Wireless Communications, 2019, 18(6): 3192–3207. doi: 10.1109/TWC.2019.2911939.
    [27]
    NASIR A A, DURRANI S, MEHRPOUYAN H, et al. Timing and carrier synchronization in wireless communication systems: A survey and classification of research in the last 5 years[J]. EURASIP Journal on Wireless Communications and Networking, 2016, 180(2016): 1–38. doi: 10.1186/s13638-016-0670-9.
    [28]
    SKOLNIK M I. Introduction to Radar Systems[M]. New York: McGraw-Hill, 1980: 581.
    [29]
    LIU Fan, YUAN Weijie, MASOUROS C, et al. Radar-assisted predictive beamforming for vehicular links: Communication served by sensing[J]. IEEE Transactions on Wireless Communications, 2020, 19(11): 7704–7719. doi: 10.1109/TWC.2020.3015735.
    [30]
    KAY S M. Fundamentals of Statistical Signal Processing[M]. Englewood Cliffs, NJ: PTR Prentice-Hall, 1993.
    [31]
    MONTAVON G, ORR G B, and MÜLLER K R. Neural Networks: Tricks of the Trade[M]. 2nd ed. Berlin: Springer, 2012: 769. doi: 10.1007/978-3-642-35289-8.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML
    Article views(307) PDF downloads(67) Cited by()
    Proportional views
    Related

    京ICP备20021838号-14   Supported by: Beijing Renhe Information Technology Co. Ltd     百度统计

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return