Smart Photogrammetric and Remote Sensing Image Processing for Very High Resolution Optical Images——Examples from the CRC-AGIP Lab at UNB

This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB), Canada. The techniques were developed by innovatively/“smartly” utilizing the characteristics of the available very high resolution optical remote sensing images to solve important problems or create new applications in photogrammetry and remote sensing. The techniques to be introduced are: automated image fusion (UNB-PanSharp), satellite image online mapping, street view technology, moving vehicle detection using single set satellite imagery, supervised image segmentation, image matching in smooth areas, and change detection using images from different viewing angles. Because of their broad application potential, some of the techniques have made a global impact, and some have demonstrated the potential for a global impact.

High-resolution Solar Image Reconstruction Based on Non-rigid Alignment

Suppressing the interference of atmospheric turbulence and obtaining observation data with a high spatial resolution are an issue to be solved urgently for ground observations. One way to solve this problem is to perform a statistical reconstruction of short-exposure speckle images. Combining the rapidity of Shift-Add and the accuracy of speckle masking, this paper proposes a novel reconstruction algorithm-NASIR(Non-rigid Alignment based Solar Image Reconstruction). NASIR reconstructs the phase of the object image at each frequency by building a computational model between geometric distortion and intensity distribution and reconstructs the modulus of the object image on the aligned speckle images by speckle interferometry. We analyzed the performance of NASIR by using the correlation coefficient, power spectrum, and coefficient of variation of intensity profile in processing data obtained by the NVST(1 m New Vacuum Solar Telescope). The reconstruction experiments and analysis results show that the quality of images reconstructed by NASIR is close to speckle masking when the seeing is good, while NASIR has excellent robustness when the seeing condition becomes worse. Furthermore, NASIR reconstructs the entire field of view in parallel in one go, without phase recursion and block-by-block reconstruction, so its computation time is less than half that of speckle masking. Therefore, we consider NASIR is a robust and highquality fast reconstruction method that can serve as an effective tool for data filtering and quick look.

A novel high resolution image denoising algorithm based on Calman filter and texture feature extraction

Calman filtering method based on wavelet transform has been successfully applied to signal denoising. According to the different application methods and the realization forms of Calman filter, combined with the structural analysis of wavelet decomposition, we present kinds of multi-scale filtering methods into the category of the three. The simulation results show that the multi-scale Calman filtering method based on system layer has better performance. Synthetic aperture radar （SAR） images have rich texture information, which can reflect the spatial structure of objects. The texture feature is widely used in SAR image classification and SAR image segmentation. Affected by imaging factors, the direct use of texture features extracted from SAR images is not good enough. In order to avoid the traditional method of filtering followed the texture feature extraction caused by the loss of texture and edge information, this paper presents a texture feature extraction of SAR image, then using Robust PCA method, finally using texture feature clustering method K-means test after treatment with RPCA expression.

An accelerated direct demodulation method for image reconstruction using spherical data from the hard X-ray modulation telescope

The hard X-ray modulation telescope （HXMT） mission is mainly devoted to performing an all-sky survey at 1- 250 keV with both high sensitivity and high spatial resolution. The observed data reduction as well as the image reconstruction for HXMT can be achieved by using the direct demodulation method （DDM）. However the original DDM is too computationally expensive for multi-dimensional data with high resolution to be employed for HXMT data. We propose an accelerated direct demodulation method especially adapted for data from HXMT. Simulations are also presented to demonstrate this method.

Aerial-BiSeNet:A real-time semantic segmentation network for high resolution aerial imagery

The aircraft system has recently gained its reputation as a reliable and efficient tool for sensing and parsing aerial scenes.However,accurate and fast semantic segmentation of highresolution aerial images for remote sensing applications is still facing three challenges:the requirements for limited processing resources and low-latency operations based on aerial platforms,the balance between high accuracy and real-time efficiency for model performance,and the confusing objects with large intra-class variations and small inter-class differences in high-resolution aerial images.To address these issues,a lightweight and dual-path deep convolutional architecture,namely Aerial Bilateral Segmentation Network(Aerial-Bi Se Net),is proposed to perform realtime segmentation on high-resolution aerial images with favorable accuracy.Specifically,inspired by the receptive field concept in human visual systems,Receptive Field Module(RFM)is proposed to encode rich multi-scale contextual information.Based on channel attention mechanism,two novel modules,called Feature Attention Module(FAM)and Channel Attention based Feature Fusion Module(CAFFM)respectively,are proposed to refine and combine features effectively to boost the model performance.Aerial-Bi Se Net is evaluated on the Potsdam and Vaihingen datasets,where leading performance is reported compared with other state-of-the-art models,in terms of both accuracy and efficiency.

Optimization of post-classification processing of high-resolution satellite image:A case study

The application of remote sensing monitoring techniques plays a crucial role in evaluating and governing the vast amount of ecological construction projects in China. However, extracting information of ecological engineering target through high-resolution satellite image is arduous due to the unique topography and complicated spatial pattern on the Loess Plateau of China. As a result, enhancing classification accuracy is a huge challenge to high-resolution image processing techniques. Image processing techniques have a definitive effect on image properties and the selection of different parameters may change the final classification accuracy during post-classification processing. The common method of eliminating noise and smoothing image is majority filtering. However, the filter function may modify the original classified image and the final accuracy. The aim of this study is to develop an efficient and accurate post-processing technique for acquiring information of soil and water conservation engineering, on the Loess Plateau of China, using SPOT image with 2.5 rn resolution. We argue that it is vital to optimize satellite image filtering parameters for special areas and purposes, which focus on monitoring ecological construction projects. We want to know how image filtering influences final classified results and which filtering kernel is optimum. The study design used a series of window sizes to filter the original classified image, and then assess the accuracy of each output map and image quality. We measured the relationship between filtering window size and classification accuracy, and optimized the post-processing techniques of SPOT5satellite images. We conclude that (1) smoothing with the majority filter is sensitive to the information accuracy of soil and water conservation engineering, and (2) for SPOT5 2.5 m image, the 5×5 pixel majority filter is most suitable kernel for extracting information of ecological construction sites in the Loess Plateau of China.

A super-high angular resolution principle for coded-mask X-ray imaging beyond the diffraction limit of a single pinhole

High angular resolution X-ray imaging is always useful in astrophysics and solar physics. In principle, it can be performed by using coded-mask imaging with a very long mask-detector distance. Previously, the diffraction-interference effect was thought to degrade coded-mask imaging performance dramatically at the low energy end with its very long mask-detector distance. The diffraction-interference effect is described with numerical calculations, and the diffraction-interference cross correlation reconstruction method （DICC） is developed in order to overcome the imaging performance degradation. Based on the DICC, a super-high angular resolution principle （SHARP） for coded-mask X-ray imaging is proposed. The feasibility of coded mask imaging beyond the diffraction limit of a single pinhole is demonstrated with simulations. With the specification that the mask element size is 50 × 50 μm^2 and the mask-detector distance is 50 m, the achieved angular resolution is 0.32arcsec above about 10keV and 0.36arcsec at 1.24keV （λ = 1 nm）, where diffraction cannot be neglected. The on-axis source location accuracy is better than 0.02 arcsec. Potential applications for solar observations and wide-field X-ray monitors are also briefly discussed.

A method for analyzing on-line video images of crystallization at high-solid concentrations

Recent research has demonstrated that on-line video imaging is a very promising technique for monitoring crystallization processes. The bottleneck in applying the technique for real-time closed-loop control is considered as image analysis that needs to be robust, fast and able to handle varied image qualities due to temporal variations of operating conditions such as mixing and solid concentrations. Image analysis at highsolid concentrations turns out to be extremely challenging because crystals tend to overlap or attach to each other and the boundaries between the crystals are usually ambiguous. This paper presents an image segmentation algorithm that can effectively deal with images taken at high-solid concentrations. The method segments crystals attached to each other along the mostly related concave points on the contours of crystal blocks. The detailed procedure is introduced with application to crystallization of L-glutamic acid in a hot-stage reactor.

Object Extraction Based on Evolutionary Morphological Processing

This paper introduces a novel technique for object detection using genetic algorithms and morphological processing. The method employs a kind of object oriented structure element, which is derived by genetic algorithms. The population of morphological filters is iteratively evaluated according to a statistical performance index corresponding to object extraction ability, and evolves into an optimal structuring element using the evolution principles of genetic search. Experimental results of road extraction from high resolution satellite images are presented to illustrate the merit and feasibility of the proposed method.

基于轻量化NDFEDet-SOLOv2的遥感图像建筑物提取方法

目的在地籍测绘和灾害管理等领域中,建筑物轮廓和位置的自动提取是至关重要的一环。为了解决高分辨率遥感图像建筑物因环境因素导致分割精度不准确等问题,提出了一种改进的轻量化SOLOv2实例分割模型——NDFEDet-SOLOv2。方法该模型选用双向特征金字塔网络(BiFPN)特征融合方式的轻量级EfficientDet网络,其中将骨干网络部分的EfficientNet升级为EfficientNetv2,EfficientNetv2中的三层MBConv模块SE注意力更换为含有DropBlock正则化的轻量级标准化注意力机制(NAM),构成NAD-MBConv模块。BiFPN特征融合部分,向其尾端各特征层并入双水平路由注意视觉变压器(BiFormer),形成双向水平路由注意特征金字塔网络结构(Bi-FPN-Former),从而聚焦微小建筑物轮廓信息,以实现更高层次的特征融合。结果NDFEDet-SOLOv2模型相较于传统轻量级SOLOv2实例分割算法,平均精度mAP、mAP 50和mAP 75分别提高了3.9%、3.7%和2.5%,检测帧率(FPS)提高了2.7帧/s。结论轻量化NDFEDet-SOLOv2实例分割算模型消除了建筑物边角的图像畸变,在地理环境空间不均等复杂情况下也能准确提取出遥感图像建筑物的基本轮廓,从而为城市布局更新和建筑变化检测提供理论参考。

结合通道交互空间组注意力与金字塔池化的高分影像语义分割网络

高空间分辨率(高分)遥感影像中存在海量信息,因此对高分影像的语义分割研究十分重要。传统机器学习方法的语义分割精度和效率均不高,近年来,深度学习方法迅速发展,逐渐成为影像语义分割领域的常用方法,已有研究将SegNet、Deeplabv3+、U-Net等神经网络引入遥感影像语义分割,但效果有限。考虑高分影像的特性,对用于遥感影像语义分割的U-Net网络进行了改进。首先,在U-Net网络特征提取过程中使用通道交互空间组注意力模块(channel interaction and spatial group attention module,CISGAM),使得网络能够获取更多有效特征。其次,在编码过程中将普通卷积层变换为残差模块,并在U-Net的编码器和解码器之间用加入了CISGAM的注意力金字塔池化模块(attention pyramid pooling module,APPM)连接,以加强网络对多尺度特征的提取。最后,在0.3 m分辨率的UC Merced数据集和1 m分辨率的GID数据集上进行实验,与U-Net、Deeplabv3+等原始网络相比,在UC Merced数据集上的平均交并比(mean intersection over union,MIoU)分别提升了14.56%和8.72%,平均像素准确率(mean pixel accuracy,MPA)分别提升了12.71%和8.24%。在GID数据集的分割结果中,水体、建筑物等地物的综合分割精度大幅提升,在平均分割精度上,CISGAM和APPM较常用的CBAM和PPM有一定提升。实验结果表明,加入CISGAM和APPM的网络可行性与鲁棒性均较传统网络强,其较强的特征提取能力有利于提升高分辨率遥感影像语义分割的精度,为高分辨率遥感影像智能解译提供新方案。

一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法

如何精细遥感监测复杂的不同类型农田作物种植情况,是智慧农业农村领域实现农耕面积调查与农作物估产的关键。目前的高分辨率影像的作物种植像素级语义分割中,深度卷积神经网络难以兼顾空间多尺度全局特征和局部细节特征,从而导致各类农田地块之间边界轮廓模糊和同类农田区域内部完整性不高等问题。针对这些不足,本文提出了一种耦合DeepLabv3+和Transformer编码器的双分支并行特征融合网络FDTNet,以实现农作物种植类型的精细遥感监测。首先,在FDTNet中并行嵌入DeepLabv3+和Transformer分别捕获农田影像的局部特征和全局特征;其次,应用耦合注意力融合模块CAFM有效融合两者的特征;然后,在解码器阶段应用卷积注意力模块CBAM增强卷积层有效特征的权重;最后,采用渐进式多层特征融合策略将编码器和解码器中的有效特征全面融合并输出特征图,以实现晚稻、中稻、藕田、菜地和大棚的高精度分类识别。为了验证FDTNet网络模型在高分辨率作物分类应用的有效性,本文选择不同高分辨率的Yuhu数据集和Zhejiang数据集验证,mIoU分别达到74.7%和81.4%。相比于已有的UNet、DeepLabv3、DeepLabv3+、ResT和Res-Swin等深度学习方法,FDTNet的mIoU可分别高2.2%和3.6%。结果表明,FDTNet在纹理单一、大样本量,以及纹理多样、小样本量的两类农田场景中同时表现出优于对比方法的性能,具有较全面的多类别农作物有效特征提取能力。

基于解耦区域校准的高分辨率超像素生成算法

超像素分割是计算机视觉领域的一项重要任务,该任务将具有相似属性的像素分组到称为超像素的簇中.图像超像素不仅可以增益图像注释,而且还是各种下游应用的基础,如分割、光流估计和深度估计.尽管超像素分割技术取得了显著进展,特别是随着深度学习方法的出现,但现有解决方案由于GPU内存和计算能力的限制,一直无法有效处理高分辨率图像.针对这个问题,作者提出了一种名为区域解耦校准的高分辨率超像素网络(Patch Calibration Network,PCNet)的新型深度学习框架,通过采用解耦的一致性学习策略,解决了现有方法的局限性.这种方法允许通过从低分辨率输入预测高分辨率输出来高效生成高分辨率超像素结果,从而绕过了GPU内存限制.PCNet的一个关键贡献是解耦的区域块校准(DPC)分支,它将高分辨率图像块作为额外输入,以保留细节并增强边界像素分配.为了改善边界像素的识别,作者利用二进制掩模设计了一种动态引导训练机制.这种机制鼓励网络专注于区域内的主要边界,将任务从多类分类简化为二分类问题.这一创新策略不仅减少了网络优化的复杂性,而且显著提高了边界检测的精度.本文通过在包括Mapillary Vistas、BIG和新创建的Face-Human数据集在内的多样化数据集上进行广泛的实验,证明了PCNet的有效性.结果表明,PCNet能够成功处理5K分辨率图像,并与现有的最先进的SCN方法相比,实现了更优越的性能,后者在处理高分辨率输入时存在困难.作者的贡献包括开发了PCNet,一种针对高分辨率超像素分割的深度学习解决方案,引入了解耦的区域校准架构,并构建了一个超高分辨率基准测试集,用于评估高分辨率场景中超像素分割算法的性能.本文首先回顾了超像素分割领域的相关工作,然后详细介绍了PCNet框架,接着展示了实验结果并与最先进的方法进行了比较.结论部分总结了研究结果并概述了未来研究的潜在方向.代码、预训练模型和新的基准数据集的可用性无疑将促进高分辨率超像素分割领域的进一步发展.总之,本文在超像素分割领域提供了一个重要的进步,提供了一种能够高效、准确处理高分辨率图像的解决方案.所提出的PCNet框架,凭借其创新的DPC分支和动态引导训练机制,为未来在计算机视觉领域的研究和应用提供了一个有前景的方向.本文的代码、预训练模型以及新构建的评估基准数据集可在https://github.com/wangyxxjtu/PCNet上获取.

基于多尺度及DESTIN约束的高分遥感影像田块语义分割方法研究

[目的]本研究旨在改善基于深度学习的遥感影像田块语义分割中出现的区域不封闭、边缘不贴合、噪点问题,并进一步修正语义分割的识别错误。[方法]以安徽省阜南县、江苏省淮安市为研究地点,自建了农田田块数据集,引入考虑影像多尺度特征的尺度分割思想与基于物候学的DESTIN(delineation by fusing spatial and temporal information)分割算法,提出了基于多尺度及DESTIN约束的高分遥感影像农田田块语义分割方法。[结果]多尺度与DESTIN约束下基于深度模型的田块语义分割有效改善模型出现的区域不封闭、边缘不贴合、噪点和块状模糊等问题,一定程度修正了深度模型语义分割的错误识别,IoU指标在2个测试集上分别达到94.08%和90.79%,相较深度模型的遥感影像田块语义分割分别提高1.65%和2.32%,对研究区域的田块提取区域更完整、精度更高。[结论]多尺度及DESTIN约束进一步改善了田块语义分割问题,有助于提高高分遥感影像的田块识别精度。

基于轻量语义分割网络的遥感土地覆盖分类

高分辨率遥感图像有丰富的空间特征,针对遥感土地覆盖方法中模型复杂,边界模糊和多尺度分割等问题,提出了一种基于边界与多尺度信息的轻量化语义分割网络.首先,使用轻量化的MobileNetV3分类器,采用深度可分离卷积来减少计算量.其次,使用自顶向下和自底向上的特征金字塔结构来进行多尺度分割.接着,设计了一个边界增强模块,为分割任务提供丰富的边界细节信息.然后,设计了一个特征融合模块,融合边界与多尺度语义特征.最后,使用交叉熵损失函数和Dice损失函数来处理样本不平衡的问题.在WHDLD数据集的平均交并比达到了59.64%,总体精度达到了87.68%.在DeepGlobe数据集的平均交并比达到了70.42%,总体精度达到了88.81%.实验结果表明,该模型能快速有效地实现遥感图像土地覆盖分类.

注意力机制和全局卷积在光伏板分割中的应用

准确识别光伏对光伏产业有效健康发展至关重要。高分辨率遥感图像复杂的背景和光伏板形状颜色多变给光伏识别带来巨大的挑战。针对高分辨率遥感图像中光伏用地提取问题,提出网络以精确地提取光伏用地。该网络采用编码器和解码器的形式融合多层特征以结合丰富的语义信息,利用全局卷积和双注意力机制捕获重要的空间特征和通道特征,并使用通道融合模块恢复丢失的部分通道信息。提出的方法可以有效解决光伏板边缘模糊和光伏板粘连的问题。在公开光伏数据集上的实验表明,与U-Net、SegNet、DeepLabv3和DeepLabv3+相比,所提方法在PV01、PV03、PV08三个数据集上的IoU分别达到87.02%、92.98%和88.43%。实验证明所提方法能对高分辨率遥感图像光伏板进行高准确率分割。

基于深度学习的多尺度无人机遥感图像道路提取

针对高分辨率遥感影像和目标场景下道路影像数据集获取难度大、成本高等问题,本文探究网络模型在不同尺度下执行提取任务的最佳影像分辨率,并评价各模型在道路提取上的适用性及可靠性,为道路识别工程提供方法借鉴和案例参考。引入图像分割领域3个经典网络模型,使用公开数据集进行模型训练,以无人机航拍的安徽省滁州市影像为试验数据,进行不同尺度下的道路提取,找出各模型在新场景下的最佳分辨率和模型适用性,并进行可靠性评价。试验结果表明,D-LinkNet网络模型在不同尺度的道路提取任务中适用性较强;DeepLabV3+网络模型的可靠性较差;U-Net、D-LinkNet网络模型的道路提取输入影像最佳分辨率分别为1.0、0.5 m。

一种DeepLabv3+结构改进的高分遥感影像红树林边界识别方法

针对红树林自动监测与保护的迫切需求,文章提出一种DeepLabv3+改进模型的高分辨率遥感影像红树林的识别方案。改进方案主要包括在DeepLabv3+的ASPP(Atrous Spatial Pyramid Pooling)结构中,引入深度可分离卷积和SE(Squeeze and Excitation)注意力机制,以及在解码端加入CBAM(Convolutional Block Attention Module)注意力机制和多尺度融合技术,以提高模型对红树林关键特征的捕捉和表征能力,从而减少漏检和误检现象。经过严格的精度评价,改进后的DeepLabv3+模型在总体精度上达到了99.60%,在召回率、红树林类交并比(Mangrove-IoU)和类F1-score上也分别达96.05%、95.31%和97.60%。与原始DeepLabv3+、HRNet和PSPNet模型相比,改进模型在所有主要评价指标上表现更优,红树林的识别准确性和边界提取能力明显提升。应用分析也进一步验证了模型的泛化能力和应用潜力。研究成果可优化红树林的实时监测技术。

基于高分卫星影像的湖南某地土地利用分类研究

随着遥感技术的快速发展,越来越多高分辨率遥感影像的出现,为快速准确地获取地面信息创造了有利条件。面向对象的分类方法在进行地物信息提取时,考虑了更多的分类特征,且能与地学知识以及其他专题特征相结合,使分类过程与人类的认知过程更加接近,已成为土地利用信息提取研究的主流方向之一。研究以高分二号影像为基础,探索多尺度分割最优参数的选取方法,构建了影像分类特征空间并对其进行优化。基于多层次分类体系提取土地利用信息,并进行精度评价。通过空间大数据对建设用地信息进行细分,并对其空间分布特征进行分析。

基于可变形卷积技术的街景图像语义分割算法

目前图像语义分割算法中可能会出现分割图像的不连续与细尺度目标丢失的缺陷,故提出可变形卷积融合增强图像的语义分割算法。算法集HRNet网络框架、Xception Module以及可变形的卷积于一体,用轻量级Xception Module优化HRNet原先存在的Bottleneck模块,同时在网络的第一阶段串联融合可变形卷积,通过建立轻量级融合加强网络从而增强针对细尺度目标特征物的辨识精度,从而使得该轻量级融合增强网络在粗尺度目标物被分割时取得相对多的细尺度目标的语义特征信息,进一步缓解语义分割图像的不连续与细尺度的目标丢失。使用Cityscapes数据集,实验结果可以说明,优化后的算法对于细尺度目标分割精度得到了显著的增强,同时解决了图像语义分割导致的分割不连续的问题。然后进行实验使用的是公开数据集PASCAL VOC 2012,实验进一步的验证了优化算法的鲁棒性以及泛化能力。