PAPS: Progressive Attention-Based Pan-sharpening

Pan-sharpening aims to seek high-resolution multispectral(HRMS) images from paired multispectral images of low resolution(LRMS) and panchromatic(PAN) images, the key to which is how to maximally integrate spatial and spectral information from PAN and LRMS images. Following the principle of gradual advance, this paper designs a novel network that contains two main logical functions, i.e., detail enhancement and progressive fusion, to solve the problem. More specifically, the detail enhancement module attempts to produce enhanced MS results with the same spatial sizes as corresponding PAN images, which are of higher quality than directly up-sampling LRMS images.Having a better MS base(enhanced MS) and its PAN, we progressively extract information from the PAN and enhanced MS images, expecting to capture pivotal and complementary information of the two modalities for the purpose of constructing the desired HRMS. Extensive experiments together with ablation studies on widely-used datasets are provided to verify the efficacy of our design, and demonstrate its superiority over other state-of-the-art methods both quantitatively and qualitatively. Our code has been released at https://github.com/JiaYN1/PAPS.

基于深度金字塔网络的Pan-Sharpening算法

为了利用高空间分辨率单波段的全色(PAN)图像和低空间分辨率的多光谱图像(MS)生成高分辨率的多光谱图像,提出一种基于深度金字塔网络的遥感图像融合(即pan-sharpening)算法,通过图像金字塔的方式逐层上采样来重构高分辨率的多光谱图像.在细节保持方面,针对全色图像和多光谱图像在尺度上跨度过大的问题,采用深度金字塔网络多尺度地融合全色图像的细节信息;在光谱保持方面,使用反卷积层代替传统的超分辨算法来上采样低分率的多光谱图像;最后将这2部分相加,得到最终的融合图像. GeoEye-1数据集上的实验结果表明,文中算法综合性能优于BDSD, PRACS, PNN and PanNet算法.

一种新的基于压缩传感理论解决Pan-Sharpening问题的算法

Pan-Sharpening问题是将高分辨率全色图像和低分辨率多光谱图像融合,重构出高分辨率多光谱图像的过程。压缩传感理论解决了相应目标函数的稀疏解问题(求解含有l0范数的极小化问题)。利用压缩传感理论,提出一种新的解决Pan-Sharpening问题的算法:以离散余弦变换矩阵作为字典进行重构。然后针对重构过程中产生的块效应、色彩偏差和洗涤效应等问题进行相应的处理。QuickBird卫星图像的实验结果表明:与经典算法进行对比,该算法在效果上有一定的优势。

A Comprehensive Evaluation of PAN-Sharpening Algorithms Coupled with Resampling Methods for Image Synthesis of Very High Resolution Remotely Sensed Satellite Data

The merging of a panchromatic (PAN) image with a multispectral satellite image (MSI) to increase the spatial resolution of the MSI, while simultaneously preserving its spectral information is classically referred as PAN-sharpening. We employed a recent dataset derived from very high resolution of WorldView-2 satellite (PAN and MSI) for two test sites (one over an urban area and the other over Antarctica), to comprehensively evaluate the performance of six existing PAN-sharpening algorithms. The algorithms under consideration were the Gram-Schmidt (GS), Ehlers fusion (EF), modified hue-intensity-saturation (Mod-HIS), high pass filtering (HPF), the Brovey transform (BT), and wavelet-based principal component analysis (W-PC). Quality assessment of the sharpened images was carried out by using 20 quality indices. We also analyzed the performance of nearest neighbour (NN), bilinear interpolation (BI), and cubic convolution (CC) resampling methods to test their practicability in the PAN-sharpening process. Our results indicate that the comprehensive performance of PAN-sharpening methods decreased in the following order: GS > W-PC > EF > HPF > Mod-HIS > BT, while resampling methods followed the order: NN > BI > CC.

A Novel Hybrid Pan-Sharpen Method Using IHS Transform and Optimization

Intensity-hue-saturation (IHS) transform is the most commonly used method for image fusion purpose. Usually, the intensity image is replaced by Panchromatic (PAN) image, or the difference between PAN and intensity image is added to each bands of RGB images. Spatial structure information in the PAN image can be effectively injected into the fused multi-spectral (MS) images using IHS method. However, spectral distortion has become the typical factor deteriorating the quality of fused results. A hybrid image fusion method which integrates IHS and minimum mean-square-error (MMSE) was proposed to mitigate the spectral distortion phenomenon in this study. Firstly, IHS transform was used to derive the intensity image;secondly, the MMSE algorithm was used to fuse the histogram matched PAN image and intensity image;thirdly, optimization calculation was employed to derive the combination coefficients, and the new intensity image could be expressed as the combination of intensity image and PAN image. Fused MS images with high spatial resolution can be generated by inverse IHS transform. In numerical experiments, QuickBird images were used to evaluate the performance of the proposed algorithm. It was found that the spatial resolution was increased significantly;meanwhile, spectral distortion phenomenon was abated in the fusion results.

Joint AIHS and Particle Swarm Optimization for Pan-sharpening

Pan-sharpening is a process of obtaining a high spatial and spectral multispectral image(HMS)by combining a low-resolution multispectral image(LMS)with a high-resolution panchromatic image(PAN).In this paper,a pan-sharpening method called PAIHS is proposed,which is based on adaptive intensity-hue-saturation(AIHS)transformation,variational pan-sharpening framework and the two fidelity hypotheses.The suitable objective function is established and optimized by adopting particle swarm optimization(PSO)to obtain the optimal control parameters and minimum value.This value corresponds to the best pan-sharpening quality.The experimental results show that the proposed method has high efficiency and reliability,and the obtained performance index is superior to the four mainstream pan-sharpening methods.

基于边缘增强与光谱特性保持的Pan-sharpening融合模型

为生成兼具高光谱质量与高空间质量的融合图像,本文提出了一种新的Pan-sharpening变分融合模型.通过拟合退化后的全色(Panchromatic, Pan)波段图像与低分辨率多光谱(Multispectral, MS)波段图像间的线性关系得到各波段MS图像的权重系数,计算从Pan图像抽取的空间细节;基于全色波段图像的梯度定义加权函数,增强了图像的强梯度边缘并对因噪声而引入的虚假边缘进行了抑制,有效地保持了全色波段图像中目标的几何结构;基于MS波段传感器的调制传输函数定义低通滤波器,自适应地限制注入空间细节的数量,显著降低了融合MS图像的光谱失真;针对Pan-sharpening模型的不适定性问题,引入L1正则化能量项,保证了数值解的稳定性.采用Split Bregman数值方法求解能量泛函的最优解,提高了算法的计算效率.QuickBird、IKONOS和GeoEye-1数据集上的实验结果表明,模型的综合融合性能优于MTF-CON、 AWLP、 SparseFI、TVR和MTF-Variational等算法.

全色图像引导高光谱图像泛锐化

泛锐化方法通过融合低分辨率高光谱图像和全色图像,从而获取具有高空间、高光谱分辨率的图像。为充分挖掘全色图像空间纹理特征,在VGG19中引入八度卷积,同时关注全色图像不同空间尺度的高低频特征;为增强光谱图像的局部空间特征,添加特征调制模块,将全色图像高频特征注入光谱图像空间细节;为加强光谱图像特征同全色图像低频特征的全局依赖关系、降低网络复杂度,网络深层对多头注意力进行优化,使光谱图像只关注全色图像最相关的图像区域;重建阶段采用Sobel滤波器、GCN图神经网络以及ECA注意力聚合光谱图像边缘细节和显著波段特征,同时,引入VGG感知损失和边缘损失,提高融合后图像的边缘特征比重。在Pavia Center、Botswana和Chikusei这3个数据集与其他泛锐化方法进行验证,其中峰值信噪比(Peak Signal-to-Noise Ratio,PSNR)值分别达到40.80%、45.90%和43.26%,SAM值分别降低到4.15%、1.38%和2.70%。提出泛锐化方法可有效恢复光谱图像空间纹理细节,同时避免光谱失真。

A pan-sharpening method based on the ADMM algorithm

Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel variational model for pan-sharpening is proposed in this paper. The model is mainly based on three hypotheses: 1) the pan-shaipened image can be linearly represented by the corresponding panchromatic image;2) the low-resolution multispectral image is down-sampled from the high-resolution multispectral image through the down-sampling operator;and 3) the satellite image has the low-rank property. Three energy components corresponding to these assumptions are integrated into a variational framework to obtain a total energy function. We adopt the alternating direction method of multipliers (ADMM) to optimize the total energy function. The experimental results show that the proposed method performs better than other mainstream methods in spectral and spatial information preserving aspect.

小波系数指导的全色锐化网络

针对现有全色锐化网络无法同时兼顾空间信息与光谱信息保留的问题,提出一种基于小波系数指导的由融合网络和指导网络组成的全色锐化网络。融合网络分别提取PAN和MS图像的多级特征,并在同一级别进行特征的选择和融合,融合后的特征分别用于指导后一级别特征的提取;指导网络用于学习HRMS与已知的输入图像的小波系数之间的映射关系,并利用学习到的映射对融合网络的输出提供额外的监督。实验结果表明,该方法能够在保留MS图像光谱信息的同时恢复尽可能多的空间信息。在模拟数据集和真实数据集上的对比实验也表明,该方法融合效果优于其他传统方法和深度学习方法,具有一定的实用价值。

基于生成对抗网络的遥感图像锐化方法

现有遥感图像锐化方法普遍采用Wald协议,导致重建图像存在空间纹理细节和颜色模糊、边缘过于平滑的问题。针对该问题,提出基于生成对抗网络(Generative Adversarial Networks,GAN)的遥感图像锐化方法PAN-GAN。该方法将多光谱图像作为参考图像,使用灰度化的参考图像模拟全色图像,并与模糊化的参考图像共同作为生成器输入,由生成器分别提取前者的纹理细节特征和后者的光谱特征并进行融合重构;引入感知损失,联合对抗损失和像素损失共同优化重构图像,使重构图像具有更加逼近参考图像的光谱和纹理细节特征。在QuickBird,GaoFen-2和WorldView-2这3个遥感卫星的图像数据集上进行实验,结果表明:与常用方法相比,使用PAN-GAN得到的重构图像具有更加逼真的光谱和空间纹理细节;使用灰度化的参考图像能够显著提升原有方法的性能并且平均灰度化提升最为明显;感知损失的引入进一步优化了重构结果,验证了所提方法的有效性。

基于MTF和变分的全色与多光谱图像融合模型

Pan-sharpening将高分辨率图像全色(Panchromatic,Pan)波段的空间细节注入多光谱(Multispectral,MS)波段,以生成同时具有高光谱和高空间分辨率的多光谱图像.为改善融合效果,需要考虑多光谱和全色波段的调制传输函数(Modulation transfer function,MTF).本文提出了一个新的基于MTF和变分的Pan-sharpening模型.该模型的能量泛函包括两项,第1项为细节注入项,基于高通滤波器从Pan波段中提取细节信息并注入融合图像;第2项为光谱保真项,基于MTF设计多孔小波的低通滤波器以保持MS波段的多光谱信息.在Quick Bird、IKONOS和Geo Eye数据集上的融合结果表明,该模型可以生成同时具有高空间和高光谱质量的融合图像,融合效果优于AWLP、IHS BT、HPM-CC-PSF、NAWL、快速变分等算法.

基于高频引导网络的遥感图像融合方法

为了有效地恢复多光谱图像每个波段的高频空间纹理,同时避免引入多余的高频信息导致光谱失真,提出了高频引导网络用于遥感图像融合。高频引导网络包含空间信息提取分支、光谱信息提取分支和引导恢复融合模块。空间信息提取分支使用编解码结构从全色图像中提取丰富的空间信息;光谱信息提取分支使用编解码结构并除去了所有的批标准化操作来保留更多的光谱信息;引导恢复融合模块利用光谱信息生成多个具有光谱保真的初步结果,与此同时利用空间信息生成具有引导作用的热图,最后通过热图选择合适的初步结果生成高空间分辨率多光谱图像。在IKONOS和QuickBird数据上与多种类型的算法进行对比实验,高频引导网络在多种客观评价指标上获得了最优结果。从视觉角度分析,高频引导网络生成的融合图像每个波段具有清晰的空间纹理且没有引入多余的高频空间纹理。

遥感影像融合AIHS转换与粒子群优化算法

Pan-sharpening是通过将低分辨率多光谱图像(LMS)与高分辨率全色图像(PAN)进行合成而获得高光谱高空间分辨率的多光谱图像(HMS)的过程。本文提出一种Pan-sharpening方法,称为PAIHS。该方法基于自适应亮度-色度-饱和度(AIHS)转换和变分Pan-sharpening框架以及两个假设(①Pan-sharpening图像和原始多光谱图像(MS)具有相同的光谱信息;②Pan-sharpening图像与全色图像(PAN)包含的几何信息保持一致),同时确定目标函数,然后用粒子群算法(PSO)进行优化,目的是得到最佳控制参数并求得目标函数最小值,此时对应着最好的Pan-sharpening质量。试验结果表明,本文提出的方法具有高效性和可靠性,获得的性能指标也优于目前一些主流的融合方法。

IKONOS图像的线性回归波段拟合融合方法

讨论基于线性回归波段拟合的空间细节信息提取方法的可行性。首先通过全色与多光谱图像构造线性回归方程,根据全色图像的高频成分设置最小二乘求解的权系数,然后利用回归系数构造低分辨率的全色图像,提取空间细节信息,最后将空间细节注入多光谱图像中进行融合。通过IKONOS全色和多光谱图像的融合实验,比较了本文方法与基于光谱响应函数的方法,结果表明:采用本文方法提取的空间细节信息进行融合,能达到甚至超过基于光谱响应函数方法的融合质量;相对于与FastIHS融合方法,本文方法的融合质量也有较大的提高。

Pleiades遥感数据融合在地理国情调查中的应用

地理国情调查工作中,数据源的选取和预处理在工程过程中占重要地位,由于航空影像数据获取时相受到一定的限制,因此,航天遥感数据作为影像有效补充。普利亚(Pleiades)航天数据以其优越的数据质量在地理国情调查工作中得到有效应用。本文从普利亚(Pleiades)地理国情调查应用出发,以探索全波段与多波段数据融合方法,提高数据可用性为目的,研究了遥感数据的几种融合方法,并对融合后的数据进行定量评价,从中选取最优方案,从而提高普利亚(Pleiades)数据的空间分辨率,达到数据分割,解译准确、精确的目标。

基于卡通-纹理分解的遥感图像融合方法

针对当前遥感图像融合算法在全色图像与多光谱图像融合过程中存在光谱失真和模糊等问题,提出一种基于卡通-纹理分解的融合方法。通过特征分解将遥感图像分解为具有光谱信息的卡通空间和包含空间信息的纹理空间;采用全色纹理分量替换多光谱纹理分量,对遥感图像进行全色锐化;利用全色卡通元素的梯度,在保持光谱质量的同时,强化图像的边缘信息和一些重要细节。实验结果表明,该方法可以在保留光谱特征的同时获得高分辨率的多光谱图像,而且该方法在空间和光谱质量方面的性能表现明显优于其他方法。

调制传递函数对全色多光谱图像融合影响

针对高分辨率影像全色(Panchromatic,Pan)波段和多光谱(Multispectral,MS)波段的pan-sharpening融合后图像光谱失真的问题,基于调制传递函数(Modulation Transfer Function,MTF)的全色多光谱图像融合模型考虑到了多光谱图像的MTF值对融合图像质量的影响,采用了与多光谱图像相同的MTF值所构建的低通滤波器,得到较好的融合结果,但如何选择一个合适的MTF值还没有很好地解决。该文针对不同MTF值对模型融合结果的影响做了详细的分析与实验,并通过线性搜索的方式找出最优的MTF值。实验结果证明了该最优MTF能够同时提高模型融合结果的光谱细节和空间细节。

Smart Photogrammetric and Remote Sensing Image Processing for Very High Resolution Optical Images——Examples from the CRC-AGIP Lab at UNB

This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB), Canada. The techniques were developed by innovatively/“smartly” utilizing the characteristics of the available very high resolution optical remote sensing images to solve important problems or create new applications in photogrammetry and remote sensing. The techniques to be introduced are: automated image fusion (UNB-PanSharp), satellite image online mapping, street view technology, moving vehicle detection using single set satellite imagery, supervised image segmentation, image matching in smooth areas, and change detection using images from different viewing angles. Because of their broad application potential, some of the techniques have made a global impact, and some have demonstrated the potential for a global impact.

用于全色锐化的相对平均生成对抗网络

为解决全色锐化过程中对原图像特征提取不足导致融合结果细节信息易丢失,以及图像融合过程中因忽略不同区域的空间特征差异而导致信息冗余等问题,采用深度学习算法,提出一种用于全色锐化的相对平均生成对抗网络(Pan-RaGAN)。在生成器中利用改进的密集块结构对原图像进行特征提取,充分利用原图像各级特征,获取包含了更多细节信息的融合结果;提出基于空间注意力机制的特征细化模块,用于特征选择,可在保留有效高频信息的同时剔除冗余信息;利用图像重建模块将细化后的特征与上采样的低分辨率多光谱图像进行融合,以保持光谱信息;利用相对平均鉴别器改进网络的损失函数,进一步优化融合效果。在高分2号卫星和快鸟卫星图像数据集上的实验结果表明,与已有用于遥感图像全色锐化的生成对抗网络相比,Pan-RaGAN网络的光谱角映射指标平均降低了0.075,验证了Pan-RaGAN网络的有效性。