SARMV3D-1.0: SAR微波视觉三维成像数据集

仇晓兰 焦泽坤 彭凌霄 陈健堃 郭嘉逸 周良将 陈龙永 丁赤飚 徐丰 董秋雷 吕守业

仇晓兰, 焦泽坤, 彭凌霄, 等. SARMV3D-1.0: SAR微波视觉三维成像数据集[J]. 雷达学报, 2021, 10(4): 485–498. doi: 10.12000/JR21112
引用本文: 仇晓兰, 焦泽坤, 彭凌霄, 等. SARMV3D-1.0: SAR微波视觉三维成像数据集[J]. 雷达学报, 2021, 10(4): 485–498. doi: 10.12000/JR21112
QIU Xiaolan, JIAO Zekun, PENG Lingxiao, et al. SARMV3D-1.0: Synthetic aperture radar microwave vision 3D imaging dataset[J]. Journal of Radars, 2021, 10(4): 485–498. doi: 10.12000/JR21112
Citation: QIU Xiaolan, JIAO Zekun, PENG Lingxiao, et al. SARMV3D-1.0: Synthetic aperture radar microwave vision 3D imaging dataset[J]. Journal of Radars, 2021, 10(4): 485–498. doi: 10.12000/JR21112

SARMV3D-1.0: SAR微波视觉三维成像数据集

DOI: 10.12000/JR21112
基金项目: 国家自然科学基金重大项目(61991420, 61991421, 61991424)
详细信息
    作者简介:

    仇晓兰(1982–),女,中国科学院空天信息创新研究院研究员,博士生导师。主要研究领域为SAR成像处理、SAR图像理解,IEEE高级会员、IEEE地球科学与遥感快报副主编、雷达学报青年编委

    丁赤飚(1969–),男,研究员,博士生导师。主要从事合成孔径雷达、遥感信息处理和应用系统等领域的研究工作,先后主持多项国家重点项目和国家级遥感卫星地面系统工程建设等项目,曾获国家科技进步奖一等奖、二等奖,国家发明奖二等奖等奖励

    通讯作者:

    仇晓兰 xlqiu@mail.ie.ac.cn

    丁赤飚 cbding@mail.ie.ac.cn

  • 责任主编:张群 Corresponding Editor: ZHANG Qun
  • 中图分类号: TN957.52

SARMV3D-1.0: Synthetic Aperture Radar Microwave Vision 3D Imaging Dataset(in English)

Funds: The National Natural Science Foundation of China (NSFC)(61991420, 61991421, 61991424)
More Information
  • 摘要: 三维成像是合成孔径雷达技术发展的前沿趋势之一,目前的SAR三维成像体制主要包括层析和阵列干涉,但面临数据采集周期长或系统过于复杂的问题,为此该文提出了SAR微波视觉三维成像的新技术思路,即充分挖掘利用SAR微波散射机制和图像视觉语义中蕴含的三维线索,并将其与SAR成像模型有效结合,以显著降低SAR三维成像的系统复杂度,实现高效能、低成本的SAR三维成像。为推动SAR微波视觉三维成像理论技术的发展,在国家自然科学基金重大项目支持下,拟构建一个比较完整的SAR微波视觉三维成像数据集。该文概述了该数据集的构成和构建规划,并给出了第一批发布数据(SARMV3D-1.0)的组成和信息描述方式、数据集制作的方法,为该数据集的共享和应用提供支撑。

     

  • 图  1  SARMV3D数据集构成示意图

    Figure  1.  Composition of SARMV3D dataset

    图  2  建筑物语义分割数据集构成示意图

    Figure  2.  Composition of SARMV3D-BIS dataset

    图  3  数据集构建流程图

    Figure  3.  Flow chart of construction of SARMV3D-BIS dataset

    图  4  建筑足迹信息提取

    Figure  4.  Extraction of building footprint

    图  5  建筑物屋顶区域示意图

    Figure  5.  Schematic diagram of building roof

    图  6  SAR图像中建筑物立面投影示意图

    Figure  6.  Projection of building elevation

    图  7  等效镜像点示意图

    Figure  7.  Schematic diagram of equivalent mirror point

    图  8  SAR图像中建筑物阴影投影示意图

    Figure  8.  Projection of building shadow in SAR image

    图  9  SAR图像中多个建筑物的投影示意图

    Figure  9.  Projection of multiple buildings in SAR image

    图  10  使用Mask RCNN在SARMV3D-BIS 1.0(S)上的训练过程中验证集损失变化曲线

    Figure  10.  Loss curve in the train process on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    图  11  使用Mask RCNN在SARMV3D-BIS 1.0(S)验证集上的实例分割结果

    Figure  11.  Results of instance segmentation on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    图  12  GOTCHA数据集幅度图像

    Figure  12.  Amplitude image of GOTCHA dataset

    图  13  阵列SAR飞行平台及天线

    Figure  13.  Array InSAR

    图  14  运城区域数据影像

    Figure  14.  Data of Yuncheng area

    图  15  峨眉山区域SAR图像

    Figure  15.  Data of mount Emei area

    图  16  幅相误差补偿前后的三维点云结果对比

    Figure  16.  Comparison of 3D point cloud results

    图  17  通道间相位误差估计结果

    Figure  17.  Results of phase error estimation between channels

    图  18  两种相位估计结果的比较

    Figure  18.  Comparison results of two phase estimation methods

    图  19  SARMV3D Imaging数据集三维成像结果

    Figure  19.  3D imaging results of SARMV3D Imaging dataset

    图  20  SARMV3D Imaging数据集区域叠掩次数图

    Figure  20.  Overlay times map of SARMV3D Imaging dataset

    1  SAR微波视觉三维成像数据集发布网页

    1.  Release webpage of Synthetic Aperture Radar Microwave Vision 3D Imaging Dataset

    图  1  Composition of SARMV3D dataset

    图  2  Composition of SARMV3D-BIS dataset

    图  3  Flow chart of construction of SARMV3D-BIS dataset

    图  4  Extraction of building footprint

    图  5  Schematic diagram of building roof

    图  6  Projection of building elevation

    图  7  Schematic diagram of equivalent mirror point

    图  8  Projection of building shadow in SAR image

    图  9  Projection of multiple buildings in SAR image

    图  10  Loss curve in the train process on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    图  11  Results of instance segmentation on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    图  12  Amplitude image of GOTCHA dataset

    图  13  Array InSAR

    图  14  Data of Yuncheng area

    图  15  Data of mount Emei area

    图  16  Comparison of 3D point cloud results

    图  17  Results of phase error estimation between channels

    图  18  Comparison results of the two phase estimation methods

    图  19  3D imaging results of SARMV3D Imaging dataset

    图  20  Overlay times map of SARMV3D Imaging dataset

    1  Release webpage of Synthetic Aperture Radar Microwave Vision 3D Imaging Dataset

    表  1  SAR建筑物语义分割数据集构成

    Table  1.   Composition of SARMV3D-BIS dataset

    序号 内容 文件后缀 说明
    1 1B级图像切片 *L1B.jpgf 切片尺寸 1024× 1024
    uint8量化
    2 SLC图像切片 *L1A.dat 切片尺寸 1024× 1024,与1B图像严格对应,int16量化,实部虚部交替存储
    3 叠掩次数图 *Layover.jpgf 切片每个像素对应的叠掩次数,uint8量化
    4 Mask图像 *Mask.jpgf 每个切片对应的建筑物分割可视化结果图像,黑色表示地面、红色表示立面、白色表示屋顶、蓝色表示阴影,如 图2所示
    5 标注文件 *.json 训练、测试和验证各一个JSON文件,详见3.2.3小节
    下载: 导出CSV

    表  2  SARMV3D-BIS数据集的标注文件

    Table  2.   Annotation file of SARMV3D-BIS dataset

    层级1字段 层级2字段 说明
    info description 描述数据集基本信息
    url 数据集的网络链接
    version 数据集版本
    contributor 数据集贡献的单位和团队等信息
    images file_name 对应图像切片的文件名,包含切片编号
    height 图像切片的高度尺寸
    width 图像切片的宽度尺寸
    date_captured 标注信息生成时间
    annotations instance_id 建筑实例的唯一编号
    segmentation 实例分割信息,见 表3
    area 建筑物投影于图像上所占的像素面积(不含阴影区域)
    image_id 该建筑实例对应的图像切片编号
    bbox 包括[ X, Y, W, H]四个数值,[ X, Y]为该建筑实例的外接矩形在对应图像切片中的左上角像素,[ W, H]为外接矩形的宽度及高度
    下载: 导出CSV

    表  3  segmentation字段信息

    Table  3.   Information of ‘segmentation’ field

    层级2字段 层级3字段 说明
    segmentation category_id 1;表示立面
    mask [ X1, Y1], [ X2, Y2], ···, [ X n, Y n],为立面多边形的角点像素坐标
    category_id 2;表示屋顶
    mask [ X1, Y1], [ X2, Y2], ···,[ X n, Y n],为屋顶多边形的角点像素坐标
    category_id 3;表示阴影
    mask [ X1, Y1], [ X2, Y2], ···, [ X n, Y n],为阴影多边形的角点像素坐标
    下载: 导出CSV

    表  4  使用Mask RCNN在SARMV3D-BIS 1.0(S)验证集上的实例分割损失

    Table  4.   Loss of instance segmentation on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone LOSS $ {\rm{LOSS}}_{{\rm{mrcnn}}}^{{\rm{bb}}} $ $ {\rm{LOSS}}_{{\rm{mrcnn}}}^{{\rm{cls}}} $ $ {\rm{LOSS}}_{{\rm{mrcnn}}}^{\rm{m}} $ $ {\rm{LOSS}}_{{\rm{RPN}}}^{{\rm{bb}}}$ ${\rm{LOSS}}_{{\rm{RPN}}}^{{\rm{cls}}}$
    ResNet-50-FPN 0.3785 0.0478 0.0405 0.1890 0.0667 0.0346
    ResNet-101-FPN 0.3473 0.0422 0.0359 0.1856 0.0498 0.0338
    下载: 导出CSV

    表  5  使用Mask RCNN在SARMV3D-BIS 1.0(S)验证集上的检测框mAP

    Table  5.   mAP of bounding box on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone AP bb $ {\rm{AP}}_{{\rm{50}}}^{{\rm{bb}}} $ $ {\rm{AP}}_{{\rm{75}}}^{{\rm{bb}}} $ $ {\rm{AP}}_{{\rm{S}}}^{{\rm{bb}}} $ $ {\rm{AP}}_{{\rm{M}}}^{{\rm{bb}}} $ $ {\rm{AP}}_{{\rm{L}}}^{{\rm{bb}}} $
    ResNet-50-FPN 16.9 29.2 17.4 23.9 23.2 15.9
    ResNet-101-FPN 19.1 30.4 22.5 27.4 28.2 16.1
    下载: 导出CSV

    表  6  使用Mask RCNN在SARMV3D-BIS 1.0(S)验证集上的掩膜mAP

    Table  6.   mAP of mask on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone AP m $ {\rm{AP}}_{{\rm{50}}}^{\rm{m}} $ ${\rm{AP}}_{{\rm{75}}}^{\rm{m}} $ $ {\rm{AP}}_{{\rm{S}}}^{\rm{m}} $ $ {\rm{AP}}_{{\rm{M}}}^{\rm{m}} $ $ {\rm{AP}}_{{\rm{L}}}^{\rm{m}} $
    ResNet-50-FPN 14.4 27.0 14.7 23.7 18.8 13.7
    ResNet-101-FPN 16.8 29.5 18.6 28.6 24.1 14.5
    下载: 导出CSV

    表  7  SARMV3D Imaging数据集信息

    Table  7.   Information of SARMV3D Imaging dataset

    地区 通道数 带宽 图像尺寸 极化
    山西运城 8 500 MHz 3100(方位)× 1220(距离) HH
    四川峨眉山 12 810 MHz 3600(方位)× 1800(距离) HH
    下载: 导出CSV

    表  8  SARMV3D Imaging数据集构成

    Table  8.   Composition of SARMV3D Imaging dataset

    文件夹名 文件名后缀 说明
    运城、峨眉山 *.jpg 1B级SAR图像,uint16量化
    *ch1~chN.dat SLC数据,float32格式,实部虚部交替存放
    *3Dresult.dat 三维成像结果数据,float32格式,5个一组,记录每个散射点( X, Y、高度、散射系数实部及散射系数虚部)
    *Layover.jpg 叠掩次数图,uint8格式
    *AuxPara.dat 三维成像所需辅助数据,数据格式见辅助数据说明文件
    *readme.pdf 说明文件,包括各个通道天线相位中心相对位置等处理所需要的信息
    下载: 导出CSV

    表  1  Composition of SARMV3D-BIS dataset

    No. Content File Suffix Description
    1 $ 1\mathrm{B} $ level image slice *L1B.jpgf Slice size 1024×1024, uint 8 quantization
    2 SLC image slice *L1A.dat Slice size 1024×1024, strictly corresponding to the 1B image, int16 quantization, with real and imaginary parts stored alternately
    3 Stacked masks *Layover.jpgf Layover tiff for each pixel of the slice, uint 8 quantization
    4 Mask image *Mask.jpgf Building segmentation visualization corresponding to each slice; the black color indicates the ground, the red color indicates the facade, the white color indicates the roof, and the blue color indicates the shadow, as shown in Fig. 2
    5 Annotation file *.json One each for training, testing and validation JSON file; see Section 3.2.3 for details
    下载: 导出CSV

    表  2  Annotation file of SARMV3D-BIS dataset

    Layer 1 fields Layer 2 fields Description
    info description Description of the basic information of the dataset
    url Web link to the dataset
    version Version of the dataset
    contributor Information about the organization and team contributing to the dataset
    images file_name File name of the corresponding image slice, including slice number
    height Height of the image slice
    width Width of the image slice
    date_captured The date when the annotation information was generated
    annotations instance_id Unique number of the building instance
    segmentation Instance segmentation information, see Tab. 3
    area The area of pixels occupied by the building projected on the image (excluding shadow areas)
    image_id The image slice number of the building instance
    bbox Includes four values $ [X,Y,W,H] $, $ [X,Y] $ is the value of the pixel of the outer rectangle of the building instance in the corresponding image slice, is the pixel of the upper left corner of the outer rectangle
    in the corresponding image slice, and $ [W,H] $ is the width and height of the outer
    rectangle of the external rectangle
    下载: 导出CSV

    表  3  Information of ‘segmentation’ field

    Layer 2 field Layer 3 fields Description
    segmentation category_id 1: indicates the facade
    mask $ [X1,Y1],[X2,Y2],\cdots ,[Xn,Yn] $, are the pixel coordinates of the corner points of the facade polygon
    category_id 2: indicates the roof
    mask $ [X1,Y1],[X2,Y2],\cdots ,[Xn,Yn] $, are the pixel coordinates of the corner points of the roof polygon
    category_id 3: indicates the shadow
    mask $ [X1,Y1],[X2,Y2],\cdots ,[Xn,Yn] $, are the pixel coordinates of the corner points of the shadow polygon
    下载: 导出CSV

    表  4  Loss of instance segmentation on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone LOSS LOSS $ {}_{\text{mrcnn}}^{\text{bb}} $ LOSS $ {}_{\text{mrcnn}\text{}}^{\text{cls}} $ LOSS $ {}_{\text{mrcnn}}^{\mathrm{m}} $ LOSS $ {}_{\text{RPN}}^{\text{bb}} $ LOSS $ {}_{\text{RPN}}^{\text{cls}} $
    ResNet-50-FPN 0.3785 0.0478 0.0405 0.1890 0.0667 0.0346
    ResNet-101-FPN 0.3473 0.0422 0.0359 0.1856 0.0498 0.0338
    下载: 导出CSV

    表  5  mAP of bounding box on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone $ {\mathrm{A}\mathrm{P}}^{\mathrm{b}\mathrm{b}} $ $ {\mathrm{A}\mathrm{P}}_{50}^{\mathrm{b}\mathrm{b}} $ $ {\mathrm{A}\mathrm{P}}_{75}^{\mathrm{b}\mathrm{b}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{S}}^{\mathrm{b}\mathrm{b}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{M}}^{\mathrm{b}\mathrm{b}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{L}}^{\mathrm{b}\mathrm{b}} $
    ResNet-50-FPN 16.9 29.2 17.4 23.9 23.2 15.9
    ResNet-101-FPN 19.1 30.4 22.5 27.4 28.2 16.1
    下载: 导出CSV

    表  6  mAP of mask on the SARMV3D-BIS 1.0(S) validation set by using Mask RCNN

    backbone $ {\mathrm{A}\mathrm{P}}^{\mathrm{m}} $ $ {\mathrm{A}\mathrm{P}}_{50}^{\mathrm{m}} $ $ {\mathrm{A}\mathrm{P}}_{75}^{\mathrm{m}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{S}}^{\mathrm{m}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{M}}^{\mathrm{m}} $ $ {\mathrm{A}\mathrm{P}}_{\mathrm{L}}^{\mathrm{m}} $
    ResNet-50-FPN 14.4 27.0 14.7 23.7 18.8 13.7
    ResNet-101-FPN 16.8 29.5 18.6 28.6 24.1 14.5
    下载: 导出CSV

    表  7  Information of SARMV3D Imaging dataset

    Area Number of channels Bandwidth Image size Polarization
    Yuncheng, Shanxi 8 $ 500\;\mathrm{M}\mathrm{H}\mathrm{z} $ 3100 (Azimuth $ )\times 1220 $ (Range) $ \mathrm{H}\mathrm{H} $
    Emei mountain, Sichuan 12 $ 810\;\mathrm{M}\mathrm{H}\mathrm{z} $ 3600 (Azimuth $ )\times 1800 $ (Range) $ \mathrm{H}\mathrm{H} $
    下载: 导出CSV

    表  8  Composition of SARMV3D Imaging dataset

    Folder name File Name Suffix Description
    Yuncheng, Mount Emei *.jpg Class 1B SAR image, uint 16 quantization
    *ch1~ chN.dat SLC data, float32 format, real part imaginary part alternate storage
    *3Dresult.dat 3D imaging result data, float32 format, in groups of 5, record each scattering point ( X, Y), height, scattering coefficient real and scattering coefficient imaginary
    *Layover.jpg Layover count map, uint 8 format
    *AuxPara.dat Auxiliary data for 3D imaging, data format see auxiliary data description file
    *readme.pdf Description file, including information required for processing, such as the relative position of the antenna phase center for each channel
    下载: 导出CSV
  • [1] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark[C]. IEEE International Geoscience and Remote Sensing Symposium, Fort Worth, USA, 2017: 3226–3229.
    [2] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4): 448–459. doi: 10.11947/j.AGCS.2019.20180206.

    JI Shunping and WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4): 448–459. doi: 10.11947/j.AGCS.2019.20180206.
    [3] https://spacenet.ai/sn6-challenge/.
    [4] LE S B, YOKOYA N, HAENSCH R, et al. 2019 IEEE GRSS data fusion contest: Large-scale semantic 3D reconstruction [technical committees][J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(4): 33–36. doi: 10.1109/MGRS.2019.2949679.
    [5] ZHANG Guo, QIANG Qiang, LUO Ying, et al. Application of RPC model in orthorectification of spaceborne SAR imagery[J]. The Photogrammetric Record, 2012, 27(137): 94–110. doi: 10.1111/j.1477-9730.2011.00667.x.
    [6] ZHANG Guo, FEI Wenbo, LI Zhen, et al. Evaluation of the RPC model for spaceborne SAR imagery[J]. Photogrammetric Engineering& Remote Sensing, 2010, 76(6): 727–733.
    [7] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 740–755.
    [8] HE Kaiming, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]. IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2980–2988.
    [9] WANG Xinlong, ZHANG Rufeng, KONG Tao, et al. SOLOv2: Dynamic, faster and stronger[OL]. https://arxiv.org/abs/2003.10152v2. 2020.
    [10] BOLYA D, ZHOU Chong, XIAO Fanyi, et al. YOLACT: Real-time instance segmentation[C]. IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019: 9156–9165.
    [11] https://cocodataset.org/#detection-eval.
    [12] Everingham M, Gool L V, Williams C K I, et al. The pascal visual object classes (VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2): 303–338.
    [13] ERTIN E, AUSTIN C D, SHARMA S, et al. GOTCHA experience report: Three-dimensional SAR imaging with complete circular apertures[C]. Proceedings of SPIE, Algorithms for Synthetic Aperture Radar Imagery XIV, Orlando, USA, 2007: 656802.
    [14] 丁赤飚, 仇晓兰, 徐丰, 等. 合成孔径雷达三维成像——从层析、阵列到微波视觉[J]. 雷达学报, 2019, 8(6): 693–709. doi: 10.12000/JR19090.

    DING Chibiao, QIU Xiaolan, XU Feng, et al. Synthetic aperture radar three-dimensional imaging—from TomoSAR and array InSAR to microwave vision[J]. Journal of Radars, 2019, 8(6): 693–709. doi: 10.12000/JR19090.
    [15] 卜运成. 阵列干涉SAR定标技术研究[D]. [博士论文], 中国科学院大学, 2018: 71–95.

    BU Yuncheng. Research on calibration technology of array synthetic aperture radar interferometry[D]. [Ph.D. dissertation], University of Chinese Academy of Sciences, 2018: 71–95.
    [16] 崔磊, 仇晓兰, 郭嘉逸, 等. 一种基于误差反向传播优化的多通道SAR相位误差估计方法[J]. 雷达学报, 2020, 9(5): 878–885. doi: 10.12000/JR20096.

    CUI Lei, QIU Xiaolan, GUO Jiayi, et al. Multi-channel phase error estimation method based on an error backpropagation algorithm for a multichannel SAR[J]. Journal of Radars, 2020, 9(5): 878–885. doi: 10.12000/JR20096.
    [17] JIAO Zekun, DING Chibiao, QIU Xiaolan, et al. Urban 3D imaging using airborne TomoSAR: Contextual information-based approach in the statistical way[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 170: 127–141. doi: 10.1016/j.isprsjprs.2020.10.013.
  • 加载中
图(42) / 表(16)
计量
  • 文章访问数:  4956
  • HTML全文浏览量:  2938
  • PDF下载量:  676
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-08-20
  • 修回日期:  2021-08-24
  • 网络出版日期:  2021-08-28

目录

    /

    返回文章
    返回