Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts ...Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.展开更多
Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images wit...Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.展开更多
Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larg...Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.展开更多
Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontroll...Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.展开更多
敦煌壁画图像含有丰富的纹理和结构信息,在破损壁画修复过程中,容易忽略受损特征信息与完整特征信息之间的区别,从而误导修复过程生成不合理的壁画内容。针对该问题,提出一种融合动态特征选择和像素级通道注意力的壁画修复模型。设计基...敦煌壁画图像含有丰富的纹理和结构信息,在破损壁画修复过程中,容易忽略受损特征信息与完整特征信息之间的区别,从而误导修复过程生成不合理的壁画内容。针对该问题,提出一种融合动态特征选择和像素级通道注意力的壁画修复模型。设计基于U-Net的网络生成器,实现对破损图像的编码与解码操作;采用有效可迁移卷积模块,通过动态选择采样空间位置,实现对有效特征信息的灵活提取,采用区域综合归一化模块减少修复区域与完整区域的期望和方差的偏移,从而加强对有效特征信息的选择和利用;在解码层设计像素级通道注意力模块,在增强有效特征权重的同时使模型可从相隔较远的空间位置学习有效特征。在敦煌壁画数据集上的实验结果表明,该算法能够利用有效信息修复掩膜区域比例不一的不规则破损壁画图像,相比PConv、PRVS、DSNet算法,在峰值信噪比(PSNR)指标上平均提升0.502 d B,在结构相似性(SSIM)指标上平均提升1.384%。展开更多
基金supported by the National Natural Science Foundation of China(62375144 and 61875092)Tianjin Foundation of Natural Science(21JCYBJC00260)Beijing-Tianjin-Hebei Basic Research Cooperation Special Program(19JCZDJC65300).
文摘Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.
基金supported in part by the National Natural Science Foundation of China under Grant 62062061/in part by the Major Project Cultivation Fund of Xizang Minzu University under Grant 324112300447.
文摘Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.
基金supported by the National Natural Science Foundation of China under Grants 62172059 and 62072055Scientific Research Fund of Hunan Provincial Education Department of China under Grant 22A0200.
文摘Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.
基金supported by the National Natural Science Foundation of China(Grant No.61871380)the Shandong Provincial Natural Science Foundation(Grant No.ZR2020MF019)Beijing Natural Science Foundation(Grant No.4172034).
文摘Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.
文摘敦煌壁画图像含有丰富的纹理和结构信息,在破损壁画修复过程中,容易忽略受损特征信息与完整特征信息之间的区别,从而误导修复过程生成不合理的壁画内容。针对该问题,提出一种融合动态特征选择和像素级通道注意力的壁画修复模型。设计基于U-Net的网络生成器,实现对破损图像的编码与解码操作;采用有效可迁移卷积模块,通过动态选择采样空间位置,实现对有效特征信息的灵活提取,采用区域综合归一化模块减少修复区域与完整区域的期望和方差的偏移,从而加强对有效特征信息的选择和利用;在解码层设计像素级通道注意力模块,在增强有效特征权重的同时使模型可从相隔较远的空间位置学习有效特征。在敦煌壁画数据集上的实验结果表明,该算法能够利用有效信息修复掩膜区域比例不一的不规则破损壁画图像,相比PConv、PRVS、DSNet算法,在峰值信噪比(PSNR)指标上平均提升0.502 d B,在结构相似性(SSIM)指标上平均提升1.384%。
