Turn off MathJax
Article Contents
Article Navigation >  Journal of Radars  > 2024  > Online First
ZHANG Qiang, WANG Zhihao, WANG Xueqian, et al. Cooperative detection of ships in optical and SAR remote sensing images based on neighborhood saliency[J]. Journal of Radars, in press. doi: 10.12000/JR24037
Citation: ZHANG Qiang, WANG Zhihao, WANG Xueqian, et al. Cooperative detection of ships in optical and SAR remote sensing images based on neighborhood saliency[J]. Journal of Radars, in press. doi: 10.12000/JR24037
shu

Cooperative Detection of Ships in Optical and SAR Remote Sensing Images Based on Neighborhood Saliency

doi: 10.12000/JR24037
  • 1.

    Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

  • 2.

    Beijing Institute of Remote Sensing, Beijing 100854, China

  • 3.

    Zhejiang Key Laboratory of Space Information Sensing and Transmission, Hangzhou Dianzi University, Hangzhou 310018, China

Funds:  The National Key R&D Program of China (2021YFA0715201), The National Natural Science Foundation of China (62101303, 62341130), Autonomous Research Program of the Department of Electronic Engineering, Tsinghua University
More Information
    Abstract
  • In ship detection through remote sensing images, optical images often provide rich details and texture information; however, the quality of such optical images can be affected by cloud and fog interferences. In contrast, Synthetic Aperture Radar (SAR) provides all-weather and all-day imaging capabilities; however, SAR images are susceptible to interference from complex sea clutter. Cooperative ship detection combining the advantages of optical and SAR images can enhance the detection performance of ships. In this paper, by focusing on the slight shift of ships in a small neighborhood range in the prior and later temporal images, we propose a method for cooperative ship detection based on neighborhood saliency in multisource heterogeneous remote sensing images, including optical and SAR data. Initially, a sea-land segmentation algorithm of optical and SAR images is applied to reduce interference from land regions. Next, single-source ship detection from optical and SAR images is performed using the RetinaNet and YOLOv5s models, respectively. Then, we introduce a multisource cooperative ship target detection strategy based on the neighborhood window opening of single-source detection results in remote sensing images and secondary detection of neighborhood salient ships. This strategy further leverages the complementary advantages of both optical and SAR heterogeneous images, reducing the possibility of missing ship and false alarms to improve overall detection performance. The performance of the proposed method has been validated using optical and SAR remote sensing data measured from Yantai, China, in 2022. Compared with existing ship detection methods, our method improves detection accuracy AP50 by ≥1.9%, demonstrating its effectiveness and superiority.

     

    • FullText(HTML)
  • loading
    • References(61)
  • [1]
    JIANG Xiao, LI Gang, LIU Yu, et al. Change detection in heterogeneous optical and SAR remote sensing images via deep homogeneous feature fusion[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 1551–1566. doi: 10.1109/jstars.2020.2983993.
    [2]
    王志豪, 李刚, 蒋骁. 基于光学和SAR遥感图像融合的洪灾区域检测方法[J]. 雷达学报, 2020, 9(3): 539–553. doi: 10.12000/JR19095.

    WANG Zhihao, LI Gang, and JIANG Xiao. Flooded area detection method based on fusion of optical and SAR remote sensing images[J]. Journal of Radars, 2020, 9(3): 539–553. doi: 10.12000/JR19095.
    [3]
    ZHANG Qiang, WANG Xueqian, WANG Zhihao, et al. Heterogeneous remote sensing image fusion based on homogeneous transformation and target enhancement[C]. 2022 IEEE International Conference on Unmanned Systems (ICUS), Guangzhou, China, 2022: 688–693. doi: 10.1109/ICUS55513.2022.9987218.
    [4]
    WANG Xueqian, ZHU Dong, LI Gang, et al. Proposal-copula-based fusion of spaceborne and airborne SAR images for ship target detection[J]. Information Fusion, 2022, 77: 247–260. doi: 10.1016/j.inffus.2021.07.019.
    [5]
    ZHANG Yu, WANG Xueqian, JIANG Zhizhuo, et al. An efficient center-based method with multilevel auxiliary supervision for multiscale SAR ship detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 7065–7075. doi: 10.1109/jstars.2022.3197210.
    [6]
    刘泽宇, 柳彬, 郭炜炜, 等. 高分三号NSC模式SAR图像舰船目标检测初探[J]. 雷达学报, 2017, 6(5): 473–482. doi: 10.12000/JR17059.

    LIU Zeyu, LIU Bin, GUO Weiwei, et al. Ship detection in GF-3 NSC mode SAR images[J]. Journal of Radars, 2017, 6(5): 473–482. doi: 10.12000/JR17059.
    [7]
    BRUSCH S, LEHNER S, FRITZ T, et al. Ship surveillance with TerraSAR-X[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(3): 1092–1103. doi: 10.1109/tgrs.2010.2071879.
    [8]
    WANG Xueqian, LI Gang, ZHANG Xiaoping, et al. A fast CFAR algorithm based on density-censoring operation for ship detection in SAR images[J]. IEEE Signal Processing Letters, 2021, 28: 1085–1089. doi: 10.1109/lsp.2021.3082034.
    [9]
    张帆, 陆圣涛, 项德良, 等. 一种改进的高分辨率SAR图像超像素CFAR舰船检测算法[J]. 雷达学报, 2023, 12(1): 120–139. doi: 10.12000/JR22067.

    ZHANG Fan, LU Shengtao, XIANG Deliang, et al. An improved superpixel-based CFAR method for high-resolution SAR image ship target detection[J]. Journal of Radars, 2023, 12(1): 120–139. doi: 10.12000/JR22067.
    [10]
    ZHANG Linping, LIU Yu, ZHAO Wenda, et al. Frequency-adaptive learning for SAR ship detection in clutter scenes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5215514. doi: 10.1109/TGRS.2023.3249349.
    [11]
    QIN Chuan, WANG Xueqian, LI Gang, et al. A semi-soft label-guided network with self-distillation for SAR inshore ship detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5211814. doi: 10.1109/TGRS.2023.3293535.
    [12]
    刘方坚, 李媛. 基于视觉显著性的SAR遥感图像NanoDet舰船检测方法[J]. 雷达学报, 2021, 10(6): 885–894. doi: 10.12000/JR21105.

    LIU Fangjian and LI Yuan. SAR remote sensing image ship detection method NanoDet based on visual saliency[J]. Journal of Radars, 2021, 10(6): 885–894. doi: 10.12000/JR21105.
    [13]
    胥小我, 张晓玲, 张天文, 等. 基于自适应锚框分配与IOU监督的复杂场景SAR舰船检测[J]. 雷达学报, 2023, 12(5): 1097–1111. doi: 10.12000/JR23059.

    XU Xiaowo, ZHANG Xiaoling, ZHANG Tianwen, et al. SAR ship detection in complex scenes based on adaptive anchor assignment and IOU supervise[J]. Journal of Radars, 2023, 12(5): 1097–1111. doi: 10.12000/JR23059.
    [14]
    WANG Wensheng, ZHANG Xinbo, SUN Wu, et al. A novel method of ship detection under cloud interference for optical remote sensing images[J]. Remote Sensing, 2022, 14(15): 3731. doi: 10.3390/rs14153731.
    [15]
    TIAN Yang, LIU Jinghong, ZHU Shengjie, et al. Ship detection in visible remote sensing image based on saliency extraction and modified channel features[J]. Remote Sensing, 2022, 14(14): 3347. doi: 10.3390/rs14143347.
    [16]
    ZHUANG Yin, LI Lianlin, and CHEN He. Small sample set inshore ship detection from VHR optical remote sensing images based on structured sparse representation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 2145–2160. doi: 10.1109/JSTARS.2020.2987827.
    [17]
    HU Jianming, ZHI Xiyang, ZHANG Wei, et al. Salient ship detection via background prior and foreground constraint in remote sensing images[J]. Remote Sensing, 2020, 12(20): 3370. doi: 10.3390/rs12203370.
    [18]
    QIN Chuan, WANG Xueqian, LI Gang, et al. An improved attention-guided network for arbitrary-oriented ship detection in optical remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 6514805. doi: 10.1109/LGRS.2022.3198681.
    [19]
    REN Zhida, TANG Yongqiang, HE Zewen, et al. Ship detection in high-resolution optical remote sensing images aided by saliency information[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5623616. doi: 10.1109/TGRS.2022.3173610.
    [20]
    SI Jihao, SONG Binbin, WU Jixuan, et al. Maritime ship detection method for satellite images based on multiscale feature fusion[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 6642–6655. doi: 10.1109/JSTARS.2023.3296898.
    [21]
    BENDER E J, REESE C E, and VAN DER WAL G S. Comparison of additive image fusion vs. feature-level image fusion techniques for enhanced night driving[C]. SPIE 4796, Low-Light-Level and Real-Time Imaging Systems, Components, and Applications, Seattle, USA, 2003: 140–151. doi: 10.1117/12.450867.
    [22]
    JIA Yong, KONG Lingjiang, YANG Xiaobo, et al. Multi-channel through-wall-radar imaging based on image fusion[C]. 2011 IEEE RadarCon (RADAR), Kansas City, USA, 2011: 103–105. doi: 10.1109/RADAR.2011.5960508.
    [23]
    JIN Yue, YANG Ruliang, and HUAN Ruohong. Pixel level fusion for multiple SAR images using PCA and wavelet transform[C]. 2006 CIE International Conference on Radar, Shanghai, China, 2006: 1–4. doi: 10.1109/ICR.2006.343209.
    [24]
    FASANO L, LATINI D, MACHIDON A, et al. SAR data fusion using nonlinear principal component analysis[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(9): 1543–1547. doi: 10.1109/LGRS.2019.2951292.
    [25]
    ZHU Dong, WANG Xueqian, LI Gang, et al. Vessel detection via multi-order saliency-based fuzzy fusion of spaceborne and airborne SAR images[J]. Information Fusion, 2023, 89: 473–485. doi: 10.1016/j.inffus.2022.08.022.
    [26]
    张良培, 何江, 杨倩倩, 等. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317–1337. doi: 10.11947/j.AGCS.2022.20220171.

    ZHANG Liangpei, HE Jiang, YANG Qianqian, et al. Data-driven multi-source remote sensing data fusion: Progress and challenges[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1317–1337. doi: 10.11947/j.AGCS.2022.20220171.
    [27]
    童莹萍, 全英汇, 冯伟, 等. 基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法[J]. 系统工程与电子技术, 2022, 44(7): 2074–2083. doi: 10.12305/j.issn.1001-506X.2022.07.02.

    TONG Yingping, QUAN Yinghui, FENG Wei, et al. Multi-source remote sensing image fusion method based on spatial-spectrum information collaboration and Gram-Schmidt transform[J]. Systems Engineering and Electronics, 2022, 44(7): 2074–2083. doi: 10.12305/j.issn.1001-506X.2022.07.02.
    [28]
    QUANG N H, TUAN V A, HAO N T P, et al. Synthetic aperture radar and optical remote sensing image fusion for flood monitoring in the Vietnam Lower Mekong Basin: A prototype application for the Vietnam open data cube[J]. European Journal of Remote Sensing, 2019, 52(1): 599–612. doi: 10.1080/22797254.2019.1698319.
    [29]
    KAUR H, KOUNDAL D, and KADYAN V. Image fusion techniques: A survey[J]. Archives of Computational Methods in Engineering, 2021, 28(7): 4425–4447. doi: 10.1007/s11831-021-09540-7.
    [30]
    FUENTES REYES M, AUER S, MERKLE N, et al. SAR-to-optical image translation based on conditional generative adversarial networks—optimization, opportunities and limits[J]. Remote Sensing, 2019, 11(17): 2067. doi: 10.3390/rs11172067.
    [31]
    LEWIS J J, O’CALLAGHAN R J, NIKOLOV S G, et al. Pixel- and region-based image fusion with complex wavelets[J]. Information Fusion, 2007, 8(2): 119–130. doi: 10.1016/j.inffus.2005.09.006.
    [32]
    JIANG Xiao, HE You, LI Gang, et al. Building damage detection via superpixel-based belief fusion of space-borne SAR and optical images[J]. IEEE Sensors Journal, 2020, 20(4): 2008–2022. doi: 10.1109/jsen.2019.2948582.
    [33]
    KHELIFI L and MIGNOTTE M. Deep learning for change detection in remote sensing images: Comprehensive review and meta-analysis[J]. IEEE Access, 2020, 8: 126385–126400. doi: 10.1109/access.2020.3008036.
    [34]
    JIANG Xiao, LI Gang, ZHANG Xiaoping, et al. A semisupervised siamese network for efficient change detection in heterogeneous remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4700718. doi: 10.1109/TGRS.2021.3061686.
    [35]
    LI Chengxi, LI Gang, WANG Xueqian, et al. A copula-based method for change detection with multisensor optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5620015. doi: 10.1109/TGRS.2023.3312344.
    [36]
    YU Ruikun, WANG Guanghui, SHI Tongguang, et al. Potential of land cover classification based on GF-1 and GF-3 data[C]. 2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, USA, 2020: 2747–2750. doi: 10.1109/IGARSS39084.2020.9324435.
    [37]
    MA Yanbiao, LI Yuxin, FENG Kexin, et al. Multisource data fusion for the detection of settlements without electricity[C]. 2021 IEEE International Geoscience and Remote Sensing Symposium, Brussels, Belgium, 2021: 1839–1842. doi: 10.1109/IGARSS47720.2021.9553860.
    [38]
    KANG Wenchao, XIANG Yuming, WANG Feng, et al. CFNet: A cross fusion network for joint land cover classification using optical and SAR images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 1562–1574. doi: 10.1109/JSTARS.2022.3144587.
    [39]
    YU Yongtao, LIU Chao, GUAN Haiyan, et al. Land cover classification of multispectral LiDAR data with an efficient self-attention capsule network[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 6501505. doi: 10.1109/LGRS.2021.3071252.
    [40]
    WU Xin, LI Wei, HONG Danfeng, et al. Vehicle detection of multi-source remote sensing data using active fine-tuning network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 167: 39–53. doi: 10.1016/j.isprsjprs.2020.06.016.
    [41]
    FANG Qingyun and WANG Zhaokui. Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery[J]. Pattern Recognition, 2022, 130: 108786. doi: 10.1016/j.patcog.2022.108786.
    [42]
    FANG Qingyun and WANG Zhaokui. Fusion detection via distance-decay intersection over union and weighted dempster–shafer evidence theory[J]. Journal of Aerospace Information Systems, 2023, 20(3): 114–125. doi: 10.2514/1.I011031.
    [43]
    焦洪臣, 张庆君, 刘杰, 等. 基于光电通路耦合的光SAR一体化卫星探测系统[P]. 中国, 115639553B, 2023.

    JIAO Hongchen, ZHANG Qingjun, LIU Jie, et al. Optical and SAR integrated satellite detection system based on photoelectric path coupling[P]. CN, 115639553B, 2023.
    [44]
    焦洪臣, 刘杰, 张庆君, 等. 一种基于光SAR共口径集成的多源一体化探测方法[P]. 中国, 115616561B, 2023.

    JIAO Hongchen, LIU Jie, ZHANG Qingjun, et al. A multi-source integrated detection method based on optical and SAR co-aperture integration[P]. CN, 115616561B, 2023.
    [45]
    ZHANG Lu, ZHU Xiangyu, CHEN Xiangyu, et al. Weakly aligned cross-modal learning for multispectral pedestrian detection[C]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019: 5126–5136. doi: 10.1109/ICCV.2019.00523.
    [46]
    陈俊. 基于R-YOLO的多源遥感图像海面目标融合检测算法研究[D]. [硕士论文], 华中科技大学, 2019. doi: 10.27157/d.cnki.ghzku.2019.002510.

    CHEN Jun. Research on maritime target fusion detection in multi-source remote sensing images based on R-YOLO[D]. [Master dissertation], Huazhong University of Science and Technology, 2019. doi: 10.27157/d.cnki.ghzku.2019.002510.
    [47]
    OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62–66. doi: 10.1109/TSMC.1979.4310076.
    [48]
    HARTIGAN J A and WONG M A. Algorithm AS 136: A K-means clustering algorithm[J]. Journal of the Royal Statistical Society. Series C (Applied Statistics), 1979, 28(1): 100–108. doi: 10.2307/2346830.
    [49]
    LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 2999–3007. doi: 10.1109/ICCV.2017.324.
    [50]
    JOCHER G, STOKEN A, BOROVEC J, et al. YOLOv5[EB/OL]. https://github.com/ultralytics/yolov5, 2020.
    [51]
    XIONG Hongqiang, LI Jing, LI Zhilian, et al. GPR-GAN: A ground-penetrating radar data generative adversarial network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5200114. doi: 10.1109/TGRS.2023.3337172.
    [52]
    WANG Zhixu, HOU Guangyu, XIN Zhihui, et al. Detection of SAR image multiscale ship targets in complex inshore scenes based on improved YOLOv5[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 5804–5823. doi: 10.1109/JSTARS.2024.3370722.
    [53]
    SHI Jingye, ZHI Ruicong, ZHAO Jingru, et al. A double-head global reasoning network for object detection of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5402216. doi: 10.1109/TGRS.2023.3347798.
    [54]
    LI Tianhua, SUN Meng, HE Qinghai, et al. Tomato recognition and location algorithm based on improved YOLOv5[J]. Computers and Electronics in Agriculture, 2023, 208: 107759. doi: 10.1016/j.compag.2023.107759.
    [55]
    LIU Wei, QUIJANO K, and CRAWFORD M M. YOLOv5-tassel: Detecting tassels in RGB UAV imagery with improved YOLOv5 based on transfer learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 8085–8094. doi: 10.1109/JSTARS.2022.3206399.
    [56]
    XIA Guisong, BAI Xiang, DING Jian, et al. DOTA: A large-scale dataset for object detection in aerial images[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3974–3983. doi: 10.1109/CVPR.2018.00418.
    [57]
    LEI Songlin, LU Dongdong, QIU Xiaolan, et al. SRSDD-v1.0: A high-resolution SAR rotation ship detection dataset[J]. Remote Sensing, 2021, 13(24): 5104. doi: 10.3390/rs13245104.
    [58]
    ACHANTA R and SÜSSTRUNK S. Saliency detection using maximum symmetric surround[C]. 2010 IEEE International Conference on Image Processing, Hong Kong, China, 2010: 2653–2656. doi: 10.1109/ICIP.2010.5652636.
    [59]
    WANG Wensheng, REN Jianxin, SU Chang, et al. Ship detection in multispectral remote sensing images via saliency analysis[J]. Applied Ocean Research, 2021, 106: 102448. doi: 10.1016/j.apor.2020.102448.
    [60]
    李志远, 郭嘉逸, 张月婷, 等. 基于自适应动量估计优化器与空变最小熵准则的SAR图像船舶目标自聚焦算法[J]. 雷达学报, 2022, 11(1): 83–94. doi: 10.12000/JR21159.

    LI Zhiyuan, GUO Jiayi, ZHANG Yueting, et al. A novel autofocus algorithm for ship targets in SAR images based on the adaptive momentum estimation optimizer and space-variant minimum entropy criteria[J]. Journal of Radars, 2022, 11(1): 83–94. doi: 10.12000/JR21159.
    [61]
    罗汝, 赵凌君, 何奇山, 等. SAR图像飞机目标智能检测识别技术研究进展与展望[J]. 雷达学报, 2024, 13(2): 307–330. doi: 10.12000/JR23056.

    LUO Ru, ZHAO Lingjun, HE Qishan, et al. Intelligent technology for aircraft detection and recognition through SAR imagery: Advancements and prospects[J]. Journal of Radars, 2024, 13(2): 307–330. doi: 10.12000/JR23056.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML
    Article views(188) PDF downloads(33) Cited by()
    Proportional views
    Related

    京ICP备20021838号-8   Supported by: Beijing Renhe Information Technology Co. Ltd     百度统计

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return