特征图知识蒸馏引导的轻量化任意方向SAR舰船目标检测器

陈诗琪; 王威; 占荣辉; 张军; 刘盛启

doi:10.12000/JR21209

特征图知识蒸馏引导的轻量化任意方向SAR舰船目标检测器

DOI: 10.12000/JR21209

国防科技大学自动目标识别重点实验室长沙 410073

基金项目: 国家自然科学基金(61901500)，湖南省科技创新计划(2020RC2044)，中国博士后科学基金(2020TQ0082)

详细信息

作者简介:
陈诗琪，博士生，主要研究方向为SAR目标检测识别等

王　威，博士，副研究员，主要研究方向为SAR/极化SAR信息处理、雷达成像、自动目标识别

占荣辉，博士，副研究员，主要研究方向为雷达目标识别、目标跟踪技术

张　军，博士，研究员，主要研究方向为雷达智能信号处理、制导雷达应用技术

刘盛启，博士，助理研究员，主要研究方向为雷达信号处理与目标识别

通讯作者:
王威 wangwei_nudt@hotmail.com

占荣辉 zhanrh@nudt.edu.cn

责任主编：孙显 Corresponding Editor: SUN Xian
中图分类号: TN959.72
计量
- 文章访问数:
- HTML全文浏览量:
- PDF下载量:
- 被引次数: 0
出版历程
- 收稿日期: 2021-12-25
- 修回日期: 2022-02-24
- 网络出版日期: 2022-03-29
- 刊出日期: 2023-02-28

A Lightweight, Arbitrary-oriented SAR Ship Detector via Feature Map-based Knowledge Distillation（in English）

National Key Laboratory of Science and Technology on Automatic Target Recognition, National University of Defense Technology, Changsha 410073, China

Funds: The National Natural Science Foundation of China (61901500), The Science and Technology Innovation Program of Hunan Province (2020RC2044), China Postdoctoral Science Foundation (2020TQ0082)

More Information

Corresponding author: WANG Wei, wangwei_nudt@hotmail.com; ZHAN Ronghui, zhanrh@nudt.edu.cn

摘要

摘要: 基于有向边界框的合成孔径雷达(SAR)舰船目标检测器能输出精准的边界框，但仍存在模型计算复杂度高、推理速度慢、存储消耗大等问题，导致其难以在星载平台上部署。基于此该文提出了结合特征图和检测头分支知识蒸馏的无锚框轻量化旋转检测方法。首先，结合目标的长宽比和方向角信息提出改进高斯核，使生成的热度图能更好地刻画目标形状。然后在检测器预测头部引入前景区域增强分支，使网络更关注前景特征且抑制背景杂波的干扰。在训练轻量化网络时，将像素点间的相似度构建为热度图蒸馏知识。为解决特征蒸馏中正负样本不平衡问题，将前景注意力区域作为掩模引导网络蒸馏与目标相关的特征。另外，该文提出全局语义模块对像素进行上下文信息建模，能够结合背景知识加强目标精确表征。基于HRSID数据集的实验结果表明所提方法在模型参数仅有9.07 M的轻量化条件下，mAP能达到80.71%，且检测帧率满足实时应用需求。
- 合成孔径雷达舰船目标检测 /
- 轻量化旋转框检测 /
- 改进高斯核 /
- 前景区域增强 /
- 知识蒸馏
Abstract: In the Synthetic Aperture Radar (SAR) ship target detection task, the targets have a large aspect ratio and dense distribution, and they are arranged in arbitrary directions. The oriented bounding box-based detection methods can output accurate detection results. However, these methods are strongly restricted by high computational complexity, slow inference speed, and large storage consumption, which complicate their deployment on space-borne platforms. To solve the above issues, a lightweight oriented anchor-free-based detection method is proposed by combining feature map and prediction head knowledge distillation. First, we propose an improved Gaussian kernel based on the aspect ratio and angle information so that the generated heatmaps can better describe the shape of the targets. Second, the foreground region enhancement branch is introduced to make the network focus more on foreground features while suppressing the background interference. When training the lightweight student network, the similarity between pixels is treated as transferred knowledge in heatmap distillation. To tackle the imbalance between positive and negative samples in feature distillation, the foreground attention region is applied as a mask to guide the feature distillation process. In addition, a global semantic module is proposed to model the contextual information around pixels, and the background knowledge is combined to further strengthen the feature representation. Experimental results based on HRSID show that our method can achieve 80.71% mAP with only 9.07 M model parameters, and the detection frame rate meets the needs of real-time applications.
- SAR ship detection /
- Lightweight oriented detection /
- Improved Gaussian kernel /
- Strengthened foreground region /
- Knowledge distillation

HTML全文

图 1 基于关键点的旋转框检测器整体框架

Figure 1. Overall framework of keypoint-based oriented detector

下载: 全尺寸图片幻灯片

图 2 HRNet骨干网络结构图

Figure 2. Structure of HRNet backbone network

下载: 全尺寸图片幻灯片

图 3 蒸馏结构示意图

Figure 3. Schematic diagram of distillation structure

下载: 全尺寸图片幻灯片

图 4 热度图可视化

Figure 4. Heatmap visualization

下载: 全尺寸图片幻灯片

图 5 不同蒸馏策略下PR曲线比较

Figure 5. Precision-Recall curves under different distillation strategies

下载: 全尺寸图片幻灯片

图 6 不同蒸馏策略下预测热度图比较

Figure 6. Comparison of predicted heatmaps under different distillation strategies

下载: 全尺寸图片幻灯片

图 7 不同蒸馏策略下不同IoU阈值下的检测定位精度变化图

Figure 7. Changes of detection performance under different IoU thresholds of different distillation strategies

下载: 全尺寸图片幻灯片

图 8 HRSID上不同旋转框检测方法的PR曲线比较

Figure 8. Precision-Recall curves comparison of different oriented detection methods on HRSID

下载: 全尺寸图片幻灯片

图 9 不同旋转框检测方法下不同场景下的检测结果比较

Figure 9. Detection results of different oriented detection methods under different scenes

下载: 全尺寸图片幻灯片

图 10 大场景图像迁移检测结果图

Figure 10. Migration detection results on large scene images

下载: 全尺寸图片幻灯片

图 1 Overall framework of a keypoint-based oriented detector

下载: 全尺寸图片幻灯片

图 2 Structure of the HRNet backbone network

下载: 全尺寸图片幻灯片

图 3 Schematic diagram of distillation structure

下载: 全尺寸图片幻灯片

图 4 Heatmap visualization

下载: 全尺寸图片幻灯片

图 5 Precision-Recall curves under different distillation strategies

下载: 全尺寸图片幻灯片

图 6 Comparison of predicted heatmaps under different distillation strategies

下载: 全尺寸图片幻灯片

图 7 Changes in detection performance under different IoU thresholds of different distillation strategies

下载: 全尺寸图片幻灯片

图 8 Precision-Recall curves comparison of different oriented detection methods on HRSID

下载: 全尺寸图片幻灯片

图 9 Detection results of different oriented detection methods under different scenes

下载: 全尺寸图片幻灯片

图 10 Migration detection results on large scene images

下载: 全尺寸图片幻灯片

表 1 HRSID数据集上的消融实验

Table 1. Ablation experiments on HRSID dataset

基线	旋转高斯核	前景注意力引导	准确率	召回率	平均精度	F1分数
√			0.8540	0.7234	0.7833	0.7759
√	√		0.8591	0.7579	0.7927	0.8053
√		√	0.8448	0.7746	0.8033	0.8082
√	√	√	0.8877	0.7543	0.8120	0.8156

下载: 导出CSV

表 2 教师和学生检测网络的性能比较

Table 2. Performance comparison of teacher and student detection network

检测网络	骨干网络	参数量(M)	浮点计算量(G)	精度
教师网络	HRNet32	30.53	104.06	0.8120
未蒸馏学生网络	HRNet16	9.07	59.05	0.7402
蒸馏后学生网络	HRNet16	9.07	59.05	0.7596

下载: 导出CSV

表 3 不同蒸馏方法在近岸和远海场景下的检测性能比较

Table 3. Detection performance comparison of different distillation methods under inshore and offsihore scenes

蒸馏方法	近岸				远海
蒸馏方法	P	R	F1	AP	P	R	F1	AP
Baseline	0.6241	0.5231	0.5691	0.4819	0.9370	0.9282	0.9326	0.9190
Mimic fea	0.6801	0.4924	0.5712	0.4912	0.9382	0.9279	0.9330	0.9194
L₂热度图	0.6539	0.5153	0.5764	0.5061	0.9440	0.9321	0.9380	0.9228
改进热度图蒸馏	0.6665	0.5685	0.6136	0.5389	0.9512	0.9321	0.9416	0.9269
AT fea+改进热度图蒸馏	0.7114	0.5491	0.6198	0.5454	0.9533	0.9347	0.9439	0.9272
改进特征+热度图蒸馏	0.7473	0.5664	0.6443	0.5778	0.9571	0.9279	0.9422	0.9255

下载: 导出CSV

表 4 典型旋转检测器上的性能比较

Table 4. Performance comparison on typical oriented detectors

检测器	准确率	召回率	F1值	平均精度	参数量	帧率
RoI Transformer	0.8524	0.7198	0.7805	0.7681	55.26	11.34
YOLOv3-R	0.8225	0.6531	0.7281	0.6907	59.68	9.13
BBAV	0.8462	0.7332	0.7857	0.7720	71.83	19.08
Oriented-RCNN	0.8369	0.7271	0.7781	0.7582	41.82	15.13
DAL	0.8517	0.7603	0.8034	0.7896	36.34	8.06
CenterNet-R	0.8628	0.7381	0.7956	0.7319	34.04	18.30
RetinaNet-R	0.8301	0.6638	0.7377	0.7070	32.33	16.10
本文方法	0.8475	0.7736	0.8089	0.8071	9.07	28.76

下载: 导出CSV

表 1 Ablation experiments on the HRSID dataset

Baseline	Oriented Gaussian kernel	Foreground attention	Precision	Recall	mAP	F1 score
√			0.8540	0.7234	0.7833	0.7759
√	√		0.8591	0.7579	0.7927	0.8053
√		√	0.8448	0.7746	0.8033	0.8082
√	√	√	0.8877	0.7543	0.8120	0.8156

下载: 导出CSV

表 2 Performance comparison of teacher and student detection networks

Detection network	Backbone	Parameter (M)	FLOP (G)	AP
Teacher network	HRNet32	30.53	104.06	0.8120
Undistilled student network	HRNet16	9.07	59.05	0.7402
Distilled student network	HRNet16	9.07	59.05	0.7596

下载: 导出CSV

表 3 Detection performance comparison of different distillation methods under inshore and offshore scenes

Method	Inshore				Offshore
Method	P	R	F1	AP	P	R	F1	AP
Baseline	0.6241	0.5231	0.5691	0.4819	0.9370	0.9282	0.9326	0.9190
Mimic fea	0.6801	0.4924	0.5712	0.4912	0.9382	0.9279	0.9330	0.9194
L ₂ heatmap	0.6539	0.5153	0.5764	0.5061	0.9440	0.9321	0.9380	0.9228
Improved heatmap distillation	0.6665	0.5685	0.6136	0.5389	0.9512	0.9321	0.9416	0.9269
AT fea+ Improved heatmap distillation	0.7114	0.5491	0.6198	0.5454	0.9533	0.9347	0.9439	0.9272
Improved feature map and heatmap distillation	0.7473	0.5664	0.6443	0.5778	0.9571	0.9279	0.9422	0.9255

下载: 导出CSV

表 4 Performance comparison on typical oriented detectors

Detector	Precision	Recall	F1 score	mAP	Parameter	FPS
RoI Transformer	0.8524	0.7198	0.7805	0.7681	55.26	11.34
Yolov3-R	0.8225	0.6531	0.7281	0.6907	59.68	9.13
BBAV	0.8462	0.7332	0.7857	0.7720	71.83	19.08
Oriented-RCNN	0.8369	0.7271	0.7781	0.7582	41.82	15.13
DAL	0.8517	0.7603	0.8034	0.7896	36.34	8.06
CenterNet-R	0.8628	0.7381	0.7956	0.7319	34.04	18.30
RetinaNet-R	0.8301	0.6638	0.7377	0.7070	32.33	16.10
Our method	0.8475	0.7736	0.8089	0.8071	9.07	28.76

下载: 导出CSV

参考文献(34)

[1]	DU Lan, DAI Hui, WANG Yan, et al. Target discrimination based on weakly supervised learning for high-resolution SAR images in complex scenes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(1): 461–472. doi: 10.1109/TGRS.2019.2937175.
[2]	CHEN Jianlai, ZHANG Junchao, JIN Yanghao, et al. Real-time processing of spaceborne SAR data with nonlinear trajectory based on variable PRF[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5205212. doi: 10.1109/TGRS.2021.3067945.
[3]	GAO Gui, LIU Li, ZHAO Lingjun, et al. An adaptive and fast CFAR algorithm based on automatic censoring for target detection in high-resolution SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(6): 1685–1697. doi: 10.1109/TGRS.2008.2006504.
[4]	CRISP D J. The state-of-the-art in ship detection in synthetic aperture radar imagery[R]. DSTO-RR-0272, 2004.
[5]	李健伟, 曲长文, 彭书娟, 等. 基于卷积神经网络的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2018, 40(9): 1953–1959. doi: 10.3969/j.issn.1001-506X.2018.09.09. LI Jianwei, QU Changwen, PENG Shujuan, et al. Ship detection in SAR images based on convolutional neural network[J]. Systems Engineering and Electronics, 2018, 40(9): 1953–1959. doi: 10.3969/j.issn.1001-506X.2018.09.09.
[6]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. doi: 10.1109/TPAMI.2016.2577031.
[7]	ZHANG Xiaohan, WANG Haipeng, XU Congan, et al. A lightweight feature optimizing network for ship detection in SAR image[J]. IEEE Access, 2019, 7: 141662–141678. doi: 10.1109/ACCESS.2019.2943241.
[8]	LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 21–37.
[9]	张晓玲, 张天文, 师君, 等. 基于深度分离卷积神经网络的高速高精度SAR舰船检测[J]. 雷达学报, 2019, 8(6): 841–851. doi: 10.12000/JR19111. ZHANG Xiaoling, ZHANG Tianwen, SHI Jun, et al. High-speed and high-accurate SAR ship detection based on a depthwise separable convolution neural network[J]. Journal of Radars, 2019, 8(6): 841–851. doi: 10.12000/JR19111.
[10]	REDMON J and FARHADI A. YOLOv3: An incremental improvement[J]. arXiv: 1804.02767, 2018.
[11]	GAO S, LIU Jianming, MIAO Yuhao, et al. A high-effective implementation of ship detector for SAR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 4019005. doi: 10.1109/LGRS.2021.3115121.
[12]	ZHU Mingming, HU Guoping, ZHOU Hao, et al. H2Det: A high-speed and high-accurate ship detector in SAR images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 12455–12466. doi: 10.1109/JSTARS.2021.313116.
[13]	FU Jiamei, SUN Xian, WANG Zhirui, et al. An anchor-free method based on feature balancing and refinement network for multiscale ship detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(2): 1331–1344. doi: 10.1109/TGRS.2020.3005151.
[14]	TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: Fully convolutional one-stage object detection[C]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, 2020: 9626–9635.
[15]	孙忠镇, 戴牧宸, 雷禹, 等. 基于级联网络的复杂大场景SAR图像舰船目标快速检测[J]. 信号处理, 2021, 37(6): 941–951. doi: 10.16798/j.issn.1003-0530.2021.06.005. SUN Zhongzhen, DAI Muchen, LEI Yu, et al. Fast detection of ship targets for complex large-scene SAR images based on a cascade network[J]. Journal of Signal Processing, 2021, 37(6): 941–951. doi: 10.16798/j.issn.1003-0530.2021.06.005.
[16]	GUO Haoyuan, YANG Xi, WANG Nannan, et al. A CenterNet++ model for ship detection in SAR images[J]. Pattern Recognition, 2021, 112: 107787. doi: 10.1016/j.patcog.2020.107787.
[17]	ZHOU Xingyi, WANG Dequan, and KRHENBÜHL P. Objects as points[EB/OL]. http://arxiv.org/abs/1904.07850, 2019.
[18]	AN Quanzhi, PAN Zongxu, LIU Lei, et al. DRBox-v2: An improved detector with rotatable boxes for target detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 8333–8349. doi: 10.1109/TGRS.2019.2920534.
[19]	YANG Rong, PAN Zhenru, JIA Xiaoxue, et al. A novel CNN-based detector for ship detection based on rotatable bounding box in SAR images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 1938–1958. doi: 10.1109/JSTARS.2021.3049851.
[20]	LIN Tsung-Yi, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 2999–3007.
[21]	徐英, 谷雨, 彭冬亮, 等. 面向合成孔径雷达图像任意方向舰船检测的改进YOLOv3模型[J]. 兵工学报, 2021, 42(8): 1698–1707. doi: 10.3969/j.issn.1000-1093.2021.08.014. XU Ying, GU Yu, PENG Dongliang, et al. An improved YOLOv3 model for arbitrary-oriented ship detection in SAR image[J]. Acta Armamentarii, 2021, 42(8): 1698–1707. doi: 10.3969/j.issn.1000-1093.2021.08.014.
[22]	FU Kun, FU Jiamei, WANG Zhihui, et al. Scattering-keypoint-guided network for oriented ship detection in high-resolution and large-scale SAR images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 11162–11178. doi: 10.1109/JSTARS.2021.3109469.
[23]	XU Yongchao, FU Mingtao, WANG Qimeng, et al. Gliding vertex on the horizontal bounding box for multi-oriented object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1452–1459. doi: 10.1109/TPAMI.2020.2974745.
[24]	WANG Jingdong, SUN Ke, CHENG Tianheng, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3349–3364. doi: 10.1109/TPAMI.2020.2983686.
[25]	ZHANG Linfeng and MA Kaisheng. Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors[C]. International Conference on Learning Representations, 2020.
[26]	CAO Yue, XU Jiarui, LIN Stephen, et al. GCNet: Non-local networks meet squeeze-excitation networks and beyond[C]. The IEEE/CVF International Conference on Computer Vision Workshop, Seoul, Korea, 2019: 1971–1980.
[27]	WEI Shunjun, ZENG Xiangfeng, QU Qizhe, et al. HRSID: A high-resolution SAR images dataset for ship detection and instance segmentation[J]. IEEE Access, 2020, 8: 120234–120254. doi: 10.1109/ACCESS.2020.3005861.
[28]	ZAGORUYKO S and KOMODAKIS N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer[EB/OL]. http://arxiv.org/abs/1612.03928, 2016.
[29]	ROMERO A, BALLAS N, KAHOU S E, et al. FitNets: Hints for thin deep nets[EB/OL]. http://arxiv.org/abs/1412.6550, 2014.
[30]	YI Jingru, WU Pengxiang, LIU Bo, et al. Oriented object detection in aerial images with box boundary-aware vectors[C]. The IEEE Winter Conference on Applications of Computer Vision, Waikoloa, USA, 2021: 2149–2158.
[31]	MING Qi, ZHOU Zhiqiang, MIAO Lingjuan, et al. Dynamic anchor learning for arbitrary-oriented object detection[EB/OL]. http://arxiv.org/abs/2012.04150, 2020.
[32]	DING Jian, XUE Nan, LONG Yang, et al. Learning RoI transformer for oriented object detection in aerial images[C]. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 2844–2853.
[33]	XIE Xingxing, CHENG Gong, WANG Jiabao, et al. Oriented R-CNN for object detection[C]. The IEEE/CVF International Conference on Computer Vision. 2021: 3520–3529.
[34]	孙显, 王智睿, 孙元睿, 等. AIR-SARShip-1.0: 高分辨率SAR舰船检测数据集[J]. 雷达学报, 2019, 8(6): 852–862. doi: 10.12000/JR19097. SUN Xian, WANG Zhirui, SUN Yuanrui, et al. AIR-SARShip-1.0: High-resolution SAR ship detection dataset[J]. Journal of Radars, 2019, 8(6): 852–862. doi: 10.12000/JR19097.