深度学习在光学和SAR影像融合研究进展

Review of deep learning in optical and SAR image fusion

遥感影像融合作为影像处理领域中最具有挑战的工作,一直是学术界研究的热点。合成孔径雷达SAR(Synthetic Aperture Radar)具备全天时、全天候、穿透云雾等多种特点,却因存在相干斑噪声等问题,使得影像难以解译。相比之下,光学影像可以反映地物的光谱和空间信息,易于解译,但容易受到云雾干扰,造成信息丢失,将光学与SAR影像数据融合可以实现不同类型传感器成像之间的信息互补,能够更好地为后续的影像分析与解译提供方便。本文首先对光学和SAR影像融合进行了系统性回顾,包括传统融合方法和基于深度学习方法在影像融合方面的最新工作,重点阐述了卷积神经网络CNN (Convolutional Neural Network)、生成式对抗网络GAN (Generative Adversarial Networks)等框架在光学和SAR影像融合中的进展;然后总结了光学和SAR影像融合在深度学习领域开发的数据集,并做了简单介绍和说明;最后,从数据集、时间序列影像融合、融合评价体系和算法轻量化等4个方面对光学和SAR影像融合的未来发展趋势进行了展望。

Remote sensing image fusion,as the most challenging work in the field of image processing,has been an academic research hotspot.SAR has various features,such as all-weather service,and penetrating clouds.However,the image is difficult to interpret due to the speckle noise problem.In contrast,optical images can reflect the spectral information of ground objects,which are easy to interpret,but interference by clouds and fog can easily occur,resulting in information loss.The fusion of optical and SAR image data can realize the complementary information between different types of sensor imaging,which can facilitate the subsequent image analysis and interpretation.In this paper,the research status and future development trend of optical and SAR remote sensing fusion is reviewed.The introduction part presents the motivation of this paper by explaining the importance of image fusion.The second part outlines the classification of optical and SAR image fusion from traditional methods to deep learning methods.The third part presents the datasets in terms of optical and SAR images and explains each dataset in detail.The fourth part summarizes the difficulties and some challenging problems of optical and SAR image fusion and highlights the future trends in the field of optical and SAR image fusion.The future development trend of fusion has four main aspects,namely,datasets,time series image fusion,fusion evaluation system,and algorithm lightweight.First,having well-targeted and sufficient datasets is an important part of training excellent fusion models,and the production of data sets is the key to improving the applicability of future models.Second,time series optical images are being used as supplementary information to SAR data with the increasing amount of satellite time series data,thus improving the utilization of image fusion.Third,the evaluation of the image fusion performance is an essential topic.No unified evaluation metric is available for objectively and comprehensively evaluating fusion algorithms,and an evaluation metric system in the field of optical and SAR image fusion must be developed.Finally,the lightweight of deep learning algorithms is an important future research direction.This paper provides a reference for researchers in optical and SAR image fusion.