Target Segmentation Method in SAR Images Based on Appearance Conversion Machine

1. Introduction

Today, the research and application of Artificial Intelligence (AI) has become a major area of scientific and technological development. Developing AI is a major strategy for enhancing national core competitiveness and maintaining national security.

The Massachusetts Institute of Technology (MIT) has not established a new college for decades. However, in October 2018, MIT announced a new facility, the Schwarzman College of Computing^[1], and the construction of the Stata Science Center (see Fig. 1) for computer science, AI, data science, and related intersections. Its purpose is to harness the powerful role of AI and big data computing in science and technology of the future. From Fig. 2, the SCR-615B radar built by MIT during World War II is on display in the Stata Science Center lobby. The MIT president also published an article in this year’s MIT newsletter^[2] emphasizing the competition and challenges brought by AI.

Figure  1.  MIT Stata Science Center

DownLoad: Full-Size Img PowerPoint

Figure  2.  SCR-615B radar displayed in the hall

DownLoad: Full-Size Img PowerPoint

In 2016, the United States (U.S.) White House released three important reports titled Preparing for the Future of Artificial Intelligence, National Artificial Intelligence Research and Development Strategic Plan, and Artificial Intelligence, and Automation and Economic Report, which promoted the establishment of a Machine Learning and Artificial Intelligence (MLAI) subcommittee that would actively plan for the future development of AI^[3]. In January 2018, the United States Department of Defense released a new version of the National Defense Strategy report, stating that the development of advanced computing, big data analysis, and robotics are important factors affecting national security. In June 2018, the U.S. Defense Advanced Research Projects Agency (DARPA) discussed for the first time the preliminary details of the U.S. Electronic Revival Plan. The implementation of this Electronic Revival Plan will accelerate the development of AI hardware. In September of the same year, DARPA announced its commitment to building a system based on common sense, contextual awareness, and higher energy efficiency^[4]. In February 2019, U.S. President Trump signed an executive order titled To Maintain U.S. Artificial Intelligence Leadership, which aims to maintain U.S. global leadership in AI. On February 12, 2019, the U.S. Department of Defense website published a Summary of the 2018 Department of Defense Artificial Intelligence Strategy—Harnessing AI to Advance Our Security and Prosperity, which clarified the U.S. military’s strategic initiatives and key areas for deploying AI^[5]. The U.S. Department of Defense plans to use DARPA’s Next Generation Artificial Intelligence (AI Next) and Artificial Intelligence Exploration (AIE) projects as benchmarks for exploring and applying AI technologies to enhance military strength. The AI Next project, which was announced in September 2018, is based on the two generations of AI technology that were led by DARPA over the past 60 years. It emphasizes the environmentally adaptive capability of AI. The main areas of this project are to explore new technologies that promote the Department of Defense’s automation of key business processes, improve the robustness and reliability of AI systems, enhance the security and flexibility of machine learning and AI technologies, reduce power consumption and avoid inefficient data collection and performance, and create the next generation of AI algorithms and applications^[6]. The AIE program will focus on Third Wave applications and theories of AI and aim to adapt machines to changing conditions. It will streamline proposals, contracts, and funding processes. The goal is to accelerate the research and development of AI platforms to help the U.S. maintain its technical advantages in the field of AI.

In March 2017, France released its Artificial Intelligence Strategy, built a new AI center, and developed data storage and processing platforms, automatic learning technology platforms, and network security platforms^[7]. The German Brain Science strategy focuses on robotics and digitization. In 2012, the Max Planck Institute for Scientific Research in Germany cooperated with the U.S. in computational neuroscience^[8]. Japan also attaches great importance to the development of AI technology. In 2017, the Japanese government issued the Next Generation Artificial Intelligence Promotion Strategy to clarify its focus on AI development and to promote the extension of AI technology to strong AI and super AI levels^[9].

China released the New Generation Artificial Intelligence Development Plan in July 2017 and formulated a three-step goal for the national AI strategy. By 2030, China’s AI theory, technology, and applications will generally reach world-leading levels and become the world’s major AI innovation center^[10]. Currently, China is showing very strong scientific research mobilization in the research and application of AI. For example, in August 2017, the National Natural Science Foundation of China (NSFC) released Guidelines for Emergency Management of Basic Research in Artificial Intelligence, which outlines plans to fund research in 25 research directions in three foundational aspects of the AI frontier, including intelligent autonomous movement bodies, intelligent decision-making theory, and key technologies of complex manufacturing processes^[11]. We believe that, driven by innovation, China will achieve significant development in the research, application, and industrial fields of AI and AI technology, occupying an important territory in the world of AI.

In this paper, we propose the development of AI technology in the field of space remote sensing and target recognition. In 2017, we hosted the Institute of Electrical and Electronics Engineers’ (IEEE) Remote Sensing Intelligent Processing Conference^[12] and published some papers in the IEEE Transactions on Geoscience and Remote Sensing/Geoscience and Remote Sensing Letters^[13-16]. We have also published several discussions in the Science & Technology Review ^[17,18], highlighting concepts regarding physical intelligence and microwave vision. Here we focus on Synthetic Aperture Radar (SAR) target monitoring and information perception and discuss the research on AI information technology against the physical background of the interaction between electromagnetic waves and targets, i.e., the use of this physical intelligence to develop microwave visions that can perceive target information on the electromagnetic spectrum that cannot be recognized by the human eye.

2. Multisource Multimode SAR Information Perception

In the 1950s, SAR images were only single-mode RCS grayscale images used for monitoring military targets. Later, in the 1970s, the development and application of this technology began to make great strides in civilian fields of study, such as ocean wind fields, terrestrial hydrology, vegetation, snow, precipitation, drought, the monitoring and evaluation of natural disasters, and the identification of surface changes, to name a few. Various applications have various needs, and the theoretical and technical issues associated with different scientific connotations have strongly promoted the comprehensive development of SAR technology. Since the beginning of the 21st century, SAR satellite technologies have developed rapidly, with the realization of full polarization, interference, and high- resolution to produce a multisource multimode full-polarization high-resolution SAR (hereinafter referred to as multimode SAR) information technology (see Fig. 3).

Figure  3.  Overview of SAR development in various countries

DownLoad: Full-Size Img PowerPoint

With the improvement in spatial resolution to meters and decimeters, the perception of multimode SAR remote sensing information has produced a field of science and technology that has great significance for civilian and national defense technology. SAR in the 21st century promotes the research and application of Automatic Target Recognition (ATR). Based on the presence or absence of a one-dimensional to a two-dimensional object map, three-dimensional object feature recognition is achieved, along with identification of multi-dimensional object morphology.

However, SAR information perception and target feature inversion and reconstruction are not accomplished by human vision. The interaction between electromagnetic waves and complex targets and their image-scattering mechanisms provide the physical basis for SAR imaging. We have studied the theoretical parameter modeling, numerical simulation, and physical and numerical characteristics in the frequency, spatial, time, and polarization domains, and have developed polarized SAR parametric simulation software, techniques for scattering and imaging calculations, and target classification, recognition, and feature reconstruction^[19].

Multimode SAR remote sensing produces a many series of images with multiple temporal and physical characteristics and rich and multiple types of complex data. Driven by remote sensing big data, remote sensing application technology has progressed in a broad range of areas. However, most of these are limited to traditional data statistical analysis and image processing technologies, which cannot meet the needs of multimode SAR technology and applications. In particular, it is difficult to realize the automatic recognition of various types of targets in the sky, land, and sea, as well as the perception and inversion reconstruction of fine-scale multi-dimensional information.

3. Big Data-driven AI Technology

In recent years, AI technology has attracted considerable attention from science and industry. Based on the recognition of local structure-features-whole target in the eye-retina-brain V1–V4 area, a simple perception rule was established to obtain visual perception ability. Using the method of computational neuroscience and driven by the fitting of big data, multi-layer convolution networks are constructed from the local structure and feature-vector space for large overall network calculations to realize the ability to perceive internal information, which is the basic idea of AI and deep learning.

Similarly, we must determine how to develop a new smart brain-like function suitable for the perception of SAR information from electromagnetic wave image scattering, which differs from computer vision processing that is usually based on optical vision. To do so, it is necessary to construct an intelligent information technology that can perceive SAR information from the microwave spectrum. We call this the electromagnetic AI–new scientific technology, i.e., from optical vision by the human brain to humanoid brain electromagnetic waves–microwave vision, which is driven by remote sensing big data under the guidance of the physics mechanism of multi-source multimode full-polarimetric high-resolution SAR.

Fig. 4 and Fig. 5 illustrate the physical basis of multimode SAR as a forward problem of electromagnetic-wave-scattering modeling simulation and an inverse problem of multi-dimensional information inversion and reconstruction. AI deep learning based on a brain-like computing neural algorithm is driven by various types of big data constrained by the physical background of multimode SAR remote sensing for processing perceptions of AI information for application in various fields.

Figure  4.  Research and application of multisource and multimode SAR remote sensing information perception for space-ground-sea targets

DownLoad: Full-Size Img PowerPoint

Figure  5.  Physical intelligence to application of remotely sensed big data

DownLoad: Full-Size Img PowerPoint

4. Microwave Vision to Realize ATR Based on Physical Intelligence

Based on the SAR image-scattering mechanism, we developed a brain-like intelligent function for processing this type of big data to perceive SAR information. This is like seeing microwaves, i.e., microwave vision. Eventually, this technology will be able to perform automatic interpretations online and produce easy-to-accept visual representations and visual semantics. Known as microwave consciousness, this technology plays an important role in the technical methods of visual semantics, reasoning, decision-making, interactive detection, identification, interference, confrontation, and the attack of SAR scattered radiation fields.

In Fig. 6, we propose a combined forward and inverse theory for the creation of electromagnetic-wave-scattering and brain-like AI research to generate a new intelligent algorithm. This cross-discipline electromagnetic AI (EM AI) has important applications in Earth remote sensing, ATR, electronic countermeasures, and satellite navigation communications. Therefore, this proposal represents remote sensing-communication-navigation technology in electromagnetic space.

Figure  6.  Artificial intelligence of space electromagnetics

DownLoad: Full-Size Img PowerPoint

We have recently edited a book series titled Spaceborne Microwave Remote Sensing^[20], whereby 14 monographs will be published by Science Press in the next two years, eight monographs of which deal with the acquisition of SAR information (Fig. 7). These include the monograph Intelligent Interpretation of Radar Image Information, written by our laboratory team^[21]. Based on the background and research status of SAR image interpretation, this monograph summarizes our laboratory’s latest research progress using deep learning intelligent technology in SAR ATR and polarized SAR feature classification, and provides sample data and program code for relevant chapters.

Figure  7.  Spaceborne microwave remote sensing research and application series

DownLoad: Full-Size Img PowerPoint

Some of the research conducted at our laboratory on intelligent information perception can be summarized as follows:

• We proposed an intelligent recognition algorithm for SAR targets^[15]. The full convolutional network we proposed reduces the number of independent parameters by removing fully connected layers. It achieved a classification accuracy of 99% for a 10-class task when applied to the SAR target classification dataset MSTAR^[22]. In addition, an end-to-end target detection–discrimination–recognition method for SAR images was implemented. Furthermore, we proposed a fast-detection algorithm for surface ship targets, established an SAR image ship target data set, and performed a ship target classification experiment based on transfer learning.

• We proposed a deep-learning training network algorithm in a complex domain^[16], whereby we can train a Convolutional Neural Network (CNN) of a polarized SAR surface classification with complex multi-dimensional images in a polarized coherence matrix. This algorithm achieved state-of-the-art accuracy of 95% for a 15-class task on the Flevoland benchmark dataset^[22].

• We proposed a CNN using few samples for target ATR, which has good network generalization ability. We also studied the target recognition and classification ability of CNN feature-vector distribution under the condition of no samples^[14]. Zero-sample learning is important for SAR ATR because training samples are not always suitable for all targets and scenarios. In this paper, we proposed a new generation-based deep neural network framework, the key aspect of which is a generative deconvolutional neural network, called a generator that automatically constructs a continuous SAR target feature space composed of direction-invariant features and direction angles while learning the target hierarchical representation. This framework is then used as a reference for designing and initializing the interpreter CNN, which is antisymmetric to the generator network. The interpreter network is then trained to map any input SAR image to the target feature space.

• We proposed a deep neural network structure for CNN processing to despeckle SAR-image noise^[23]. This process uses a CNN to extract image features and reconstruct a discrete RCS Probability Density Function (PDF). The network is trained by a mixed loss function that measures the distance between the actual and estimated SAR image intensity PDFs, which is obtained by the convolution between the reconstructed RCS PDF and the prior speckled PDF. The network can be trained using either simulated or real SAR images. Experimental results on both simulated SAR images and real NASA/JPL AIRSAR images confirm the effectiveness of the proposed noise-despeckling deep neural network.

• Lastly, we proposed a colorized CNN processing method from single-polarized SAR images to polarized SAR images for scene analysis and processing^[24]. This paper proposed a deep neural network that converts a single-polarized SAR image into a fully polarized SAR image. This network has two parts, a feature extraction network and a feature translation network that is used to match spatial and polarized features. Using this method, the polarization covariance matrix of each pixel can be reconstructed. The resulting fully polarized SAR image is very close to the real fully polarized SAR image not only visually but also in real PolSAR applications.

In addition, part of the work of our laboratory is to do the SAR-AI-ATR identification of—base on domestic and foreign SAR data including China’s GF-3 SAR data. do the SAR-AI-ATR identification of ground vehicles, airport aircraft, and sea surface ships. In addition, we proposed a CNN method for the inversion of forest tree heights by interferometric SAR, i.e., INSAR, and a method for constructing the reciprocal generation of optical images and microwave radar images by the contrast training of optical and microwave images. The above work can be found in related monographs^[21].

5. Conclusion

Data is not synonymous with information. Big data is just material and a driver, and different data have different scientific connotations. Therefore, the use of simple and direct statistics in the analysis of big data cannot realize the perception of connotative information, especially in the imaging of multi-dimensional vectorized complex data of multimode microwave SAR, which is difficult to intuitively perceive by the human eye. In this paper, we proposed the use of AI driven by big data under the guidance of physics to retrieve information and develop new AI models and algorithms to meet the needs of SAR remote sensing physics and applications. Interdisciplinary AI research is very important. The realization of new EM AI technology will drive the development of multiple industries and applications.

At present, research on multimode remote sensing intelligent information and target recognition is still in the exploratory stage, and further research is needed to continue to develop new theories, methods, and applications of microwave vision.