Deep learning-based inpainting of saturation artifacts in optical coherence tomography images

Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.

Mangrove monitoring and extraction based on multi-source remote sensing data:a deep learning method based on SAR and optical image fusion

Mangroves are indispensable to coastlines,maintaining biodiversity,and mitigating climate change.Therefore,improving the accuracy of mangrove information identification is crucial for their ecological protection.Aiming at the limited morphological information of synthetic aperture radar(SAR)images,which is greatly interfered by noise,and the susceptibility of optical images to weather and lighting conditions,this paper proposes a pixel-level weighted fusion method for SAR and optical images.Image fusion enhanced the target features and made mangrove monitoring more comprehensive and accurate.To address the problem of high similarity between mangrove forests and other forests,this paper is based on the U-Net convolutional neural network,and an attention mechanism is added in the feature extraction stage to make the model pay more attention to the mangrove vegetation area in the image.In order to accelerate the convergence and normalize the input,batch normalization(BN)layer and Dropout layer are added after each convolutional layer.Since mangroves are a minority class in the image,an improved cross-entropy loss function is introduced in this paper to improve the model’s ability to recognize mangroves.The AttU-Net model for mangrove recognition in high similarity environments is thus constructed based on the fused images.Through comparison experiments,the overall accuracy of the improved U-Net model trained from the fused images to recognize the predicted regions is significantly improved.Based on the fused images,the recognition results of the AttU-Net model proposed in this paper are compared with its benchmark model,U-Net,and the Dense-Net,Res-Net,and Seg-Net methods.The AttU-Net model captured mangroves’complex structures and textural features in images more effectively.The average OA,F1-score,and Kappa coefficient in the four tested regions were 94.406%,90.006%,and 84.045%,which were significantly higher than several other methods.This method can provide some technical support for the monitoring and protection of mangrove ecosystems.

Enhancing transjugular intrahepatic portosystemic shunt procedure efficiency with digital subtraction angiography image overlay technology in esophagogastric variceal bleeding

BACKGROUND Transjugular intrahepatic portosystemic shunt(TIPS)is a pivotal intervention for managing esophagogastric variceal bleeding in patients with chronic hepatic schistosomiasis.AIM To evaluate the efficacy of digital subtraction angiography image overlay tech-nology(DIT)in guiding the TIPS procedure.METHODS We conducted a retrospective analysis of patients who underwent TIPS at our hospital,comparing outcomes between an ultrasound-guided group and a DIT-guided group.Our analysis focused on the duration of the portosystemic shunt puncture,the number of punctures needed,the total surgical time,and various clinical indicators related to the surgery.RESULTS The study included 52 patients with esophagogastric varices due to chronic hepatic schistosomiasis.Results demonstrated that the DIT-guided group expe-rienced significantly shorter puncture times(P<0.001)and surgical durations(P=0.022)compared to the ultrasound-guided group.Additionally,postoperative assessments showed significant reductions in aspartate aminotransferase,B-type natriuretic peptide,and portal vein pressure in both groups.Notably,the DIT-guided group also showed significant reductions in total bilirubin(P=0.001)and alanine aminotransferase(P=0.023).CONCLUSION The use of DIT for guiding TIPS procedures highlights its potential to enhance procedural efficiency and reduce surgical times in the treatment of esophagogastric variceal bleeding in patients with chronic hepatic schistoso-miasis.

The Application of Digital Optical 3D Image Analyzer Evaskin in The Evaluation of Wrinkles

To verify the effectiveness of digital optical 3D image analyzer EvaSKIN in the objective and quantitative evaluation of wrinkles.A total of 115 subjects were recruited,the facial images of the subjects were collected by digital optical 3D image analyzer and manual camera,the changes of crow’s feet with age were analyzed.Pictures obtained by manual photography can be directly used for observation and preliminary grading of wrinkles.However,the requirements for evaluators are high,and the results are prone to errors,which will affect the accuracy of the evaluation.Therefore,skilled raters are needed.Compared with the manual photography method,the digital optical 3D image analyzer EvaSKIN can realize three-dimensional extraction of wrinkles,and obtain the change trend of crow’s feet with age.20~30 years old,wrinkles begin to appear slowly;wrinkles will increase rapidly at the age of 30~50;The length of 50~60 year old wrinkles is basically fixed,the wrinkles develop longitudewise,gradually widen and deepen,and the area,depth and volume increase is obvious,and the skin aging condition is intensified.the digital optical 3D image analyzer EvaSKIN realizes the 3D extraction of wrinkles,quantifies the circumference,area,average depth,maximum depth and volume of wrinkles,realizes the objective and quantitative evaluation of wrinkle state,is more accurate in the measurement of wrinkles,and provides a new instrument and method for the evaluation of wrinkles.it is a perfect and supplement to the traditional evaluation methods,and to a certain extent,it helps the research and development and evaluation institutions of cosmetics to obtain more abundant and three-dimensional data support.

A promising approach for quantifying focal stroke modeling and assessing stroke progression:optical resolution photoacoustic microscopy photothrombosis

To investigate the mechanisms underlying the onset and progression of ischemic stroke,some methods have been proposed that can simultaneously monitor and create embolisms in the animal cerebral cortex.However,these methods often require complex systems and the effect of age on cerebral embolism has not been adequately studied,although ischemic stroke is strongly age-related.In this study,we propose an optical-resolution photoacoustic microscopy-based visualized photothrombosis methodology to create and monitor ischemic stroke in mice simultaneously using a 532 nm pulsed laser.We observed the molding process in mice of different ages and presented age-dependent vascular embolism differentiation.Moreover,we integrated optical coherence tomography angiography to investigate age-associated trends in cerebrovascular variability following a stroke.Our imaging data and quantitative analyses underscore the differential cerebrovascular responses to stroke in mice of different ages,thereby highlighting the technique's potential for evaluating cerebrovascular health and unraveling age-related mechanisms involved in ischemic strokes.

PCA-based sea-ice image fusion of optical data by HIS transform and SAR data by wavelet transform

Sea ice as a disaster has recently attracted a great deal of attention in China. Its monitoring has become a routine task for the maritime sector. Remote sensing, which depends mainly on SAR and optical sensors, has become the primary means for sea-ice research. Optical images contain abundant sea-ice multi-spectral in-formation, whereas SAR images contain rich sea-ice texture information. If the characteristic advantages of SAR and optical images could be combined for sea-ice study, the ability of sea-ice monitoring would be im-proved. In this study, in accordance with the characteristics of sea-ice SAR and optical images, the transfor-mation and fusion methods for these images were chosen. Also, a fusion method of optical and SAR images was proposed in order to improve sea-ice identification. Texture information can play an important role in sea-ice classification. Haar wavelet transformation was found to be suitable for the sea-ice SAR images, and the texture information of the sea-ice SAR image from Advanced Synthetic Aperture Radar （ASAR） loaded on ENVISAT was documented. The results of our studies showed that, the optical images in the hue-intensi-ty-saturation （HIS） space could reflect the spectral characteristics of the sea-ice types more efficiently than in the red-green-blue （RGB） space, and the optical image from the China-Brazil Earth Resources Satellite （CBERS-02B） was transferred from the RGB space to the HIS space. The principal component analysis （PCA） method could potentially contain the maximum information of the sea-ice images by fusing the HIS and texture images. The fusion image was obtained by a PCA method, which included the advantages of both the sea-ice SAR image and the optical image. To validate the fusion method, three methods were used to evaluate the fused image, i.e., objective, subjective, and comprehensive evaluations. It was concluded that the fusion method proposed could improve the ability of image interpretation and sea-ice identification.

Mapping the bathymetry of shallow coastal water using singleframe fine-resolution optical remote sensing imagery

This paper presents a bathymetry inversion method using single-frame fine-resolution optical remote sensing imagery based on ocean-wave refraction and shallow-water wave theory. First, the relationship among water depth, wavelength and wave radian frequency in shallow water was deduced based on shallow-water wave theory. Considering the complex wave distribution in the optical remote sensing imagery, Fast Fourier Transform （FFT） and spatial profile measurements were applied for measuring the wavelengths. Then, the wave radian frequency was calculated by analyzing the long-distance fluctuation in the wavelength, which solved a key problem in obtaining the wave radian frequency in a single-frame image. A case study was conducted for Sanya Bay of Hainan Island, China. Single-flame fine-resolution optical remote sensing imagery from QuickBird satellite was used to invert the bathymetry without external input parameters. The result of the digital elevation model （DEM） was evaluated against a sea chart with a scale of 1：25 000. The root-mean-square error of the inverted bathymetry was 1.07 m, and the relative error was 16.2%. Therefore, the proposed method has the advantages including no requirement for true depths and environmental parameters, and is feasible for mapping the bathymetry of shallow coastal water.

Segmentation of retinal fluid based on deep learning:application of three-dimensional fully convolutional neural networks in optical coherence tomography images

AIM: To explore a segmentation algorithm based on deep learning to achieve accurate diagnosis and treatment of patients with retinal fluid.METHODS: A two-dimensional(2D) fully convolutional network for retinal segmentation was employed. In order to solve the category imbalance in retinal optical coherence tomography(OCT) images, the network parameters and loss function based on the 2D fully convolutional network were modified. For this network, the correlations of corresponding positions among adjacent images in space are ignored. Thus, we proposed a three-dimensional(3D) fully convolutional network for segmentation in the retinal OCT images.RESULTS: The algorithm was evaluated according to segmentation accuracy, Kappa coefficient, and F1 score. For the 3D fully convolutional network proposed in this paper, the overall segmentation accuracy rate is 99.56%, Kappa coefficient is 98.47%, and F1 score of retinal fluid is 95.50%. CONCLUSION: The OCT image segmentation algorithm based on deep learning is primarily founded on the 2D convolutional network. The 3D network architecture proposed in this paper reduces the influence of category imbalance, realizes end-to-end segmentation of volume images, and achieves optimal segmentation results. The segmentation maps are practically the same as the manual annotations of doctors, and can provide doctors with more accurate diagnostic data.

Experimental investigation on the optical remote sensing images of internal solitary waves with a smooth surface

The parameter inversion of internal solitary waves (ISWs) based on optical remote sensing images is a key work. A new approach is proposed and demonstrated for simulating the optical remote sensing images of ISWs with a smooth surface in the laboratory. An optical remote sensing simulation system used to detect ISWs is constructed by a two-dimensional ISW flume, a LED (light emitting diode) light source and two CCD (charge coupled device) cameras. The optical remote sensing images of the horizontal surface and ISWs propagation images of a vertical side are detected simultaneously, which aims to explore the response of optical remote sensing corresponding to ISWs with the smooth surface. The results show that during the propagation of ISWs, dark pattern images are obtained by CCD 1 camera. The characteristics of the dark patterns vary along with the incident angle of the light source. The characteristic parameters of the optical remote sensing images correspond to the wave factors of vertical profiles. The experiment also shows a positive correlation between the dark pattern width and the half wave width under different amplitudes of ISWs. The system has the advantages of clear phenomenon and high repeatability, which provides the scientific basis for quantitative investigation on imaging mechanism of ISW by optical remote sensing.

Optical Image Encryption Based on Mixed Chaotic Maps and Single-Shot Digital Holography

Random phase masks play a key role in optical image encryption schemes based on double random phase technique. In this paper, a mixed chaotic method is proposed, which can efficiently solve some weaknesses that one-dimensional (1-D) single chaotic maps encounter to generate random phase masks. Based on the chaotic random phase masks, optical image encryption and decryption are realized with a single-shot digital holographic technique. In the proposed encryption scheme, the initial value and parameters of mixed chaotic maps serve as secret keys, which is convenient for the key management and transmission. Moreover, it also possesses high resistance against statistical attack, brute-force attack, noise attack and shear attack. Simulation results and security analysis verify the validity and security of the proposed encryption scheme. © 2017, Tianjin University and Springer-Verlag Berlin Heidelberg.

Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

Influences of Atmospheric Turbulence on Image Resolution of Airborne and Space-Borne Optical Remote Sensing System

A new way is proposed to evaluate the influence of atmospheric turbulence on image resolution of airborne and space-borne optical remote sensing system, which is called as arrival angle-method. Applying this method, some engineering examples are selected to analyze the turbulence influences on image resolution based on three different atmospheric turbulence models quantificationally, for the airborne remote sensing system, the resolution errors caused by the atmospheric turbulence are less than 1 cm, and for the space-borne remote sensing system, the errors are around 1 cm. The results are similar to that obtained by the previous Friedmethod. Compared with the Fried-method, the arrival angle-method is rather simple and can be easily used in engineering fields.

An Underwater Image Bubble Noise Removal Method Based on Optical Flow

It is very important for underwater robots to accurately detect and locate target objects. However,under many circumstances,it is difficult to clearly observe the target object due to the existence of bubble noise. In this paper,we proposed a method to solve this problem. First,we used the LK optical flow algorithm to calculate the motion vector of the image background and compensate for the background motion.Then,the optical flow field of the bubbles was calculated by the HS optical flow algorithm,and the area where the bubble existed was obtained by binarizing the image. Finally,we used the adjacent frame image to repair the bubble area. We carried out a bubble noise removal experiment. The results show that this method can effectively remove the bubble noise in the image.

A Steganography-Based Optical Image Encryption System Using RGB Channel Integration

An optical image encryption system with adaptive steganography using red, green, and blue （RGB） channel integration is proposed. The optical image encryption system employs a double random phase encoding algorithm to encrypt and decrypt color images. The RGB channel in a color image is first integrated into a large grayscale image. Then the integrated image is encrypted by two random phase masks. The secret data is then embedded into the encrypted image with a specific hiding sequence generated by the zero-LSB （least significant bits） sorting technique which is a content-dependent and low distortion data embedding method. Experimental results show that the proposed, method has a good performance in both hiding capacity and decrypted image quality.

Fusion of SAR and Optical Image for Sea Ice Extraction

It is difficult to balance local details and global distribution using a single source image in marine target detection of a large scene.To solve this problem,a technique based on the fusion of optical image and synthetic aperture radar(SAR)image for the extraction of sea ice is proposed in this paper.The Band 2(B2 image of Sentinel-2(S2 in the research area is selected as optical image data.Preprocessing on the optical image,such as resampling,projection transformation and format conversion,are conducted to the S2 dataset before fusion.Imaging characteristics of the sea ice have been analyzed,and a new deep learning(DL)model,OceanTDL5,is built to detect sea ices.The fusion of the Sentinel-1(S1 and S2 images is realized by solving the optimal pixel values based on deriving Poisson Equation.The experimental results indicate that the use of a fused image improves the accuracy of sea ice detection compared with the use of a single data source.The fused image has richer spatial details and a clearer texture compared with the original optical image,and its material sense and color are more abundant.

Improvement Detecting Method of Optical Axes Parallelism of Shipboard Photoelectrical Theodolite Based on Image Processing

An improvement detecting method was proposed according to the disadvantages of testing method of optical axes parallelism of shipboard photoelectrical theodolite (short for theodolite) based on image processing. Pointolite replaced 0.2'' collimator to reduce the errors of crosshair images processing and improve the quality of image. What’s more, the high quality images could help to optimize the image processing method and the testing accuracy. The errors between the trial results interpreted by software and the results tested in dock were less than 10'', which indicated the improve method had some actual application values.

Cerebrovascular segmentation from mesoscopic optical images using Swin Transformer

Vascular segmentation is a crucial task in biomedical image processing,which is significant for analyzing and modeling vascular networks under physiological and pathological states.With advances in fluorescent labeling and mesoscopic optical techniques,it has become possible to map the whole-mouse-brain vascular networks at capillary resolution.However,segmenting vessels from mesoscopic optical images is a challenging task.The problems,such as vascular signal discontinuities,vessel lumens,and background fluorescence signals in mesoscopic optical images,belong to global semantic information during vascular segmentation.Traditional vascular segmentation methods based on convolutional neural networks(CNNs)have been limited by their insufficient receptive fields,making it challenging to capture global semantic information of vessels and resulting in inaccurate segmentation results.Here,we propose SegVesseler,a vascular segmentation method based on Swin Transformer.SegVesseler adopts 3D Swin Transformer blocks to extract global contextual information in 3D images.This approach is able to maintain the connectivity and topology of blood vessels during segmentation.We evaluated the performance of our method on mouse cerebrovascular datasets generated from three different labeling and imaging modalities.The experimental results demonstrate that the segmentation effect of our method is significantly better than traditional CNNs and achieves state-of-the-art performance.

Smart Photogrammetric and Remote Sensing Image Processing for Very High Resolution Optical Images——Examples from the CRC-AGIP Lab at UNB

This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB), Canada. The techniques were developed by innovatively/“smartly” utilizing the characteristics of the available very high resolution optical remote sensing images to solve important problems or create new applications in photogrammetry and remote sensing. The techniques to be introduced are: automated image fusion (UNB-PanSharp), satellite image online mapping, street view technology, moving vehicle detection using single set satellite imagery, supervised image segmentation, image matching in smooth areas, and change detection using images from different viewing angles. Because of their broad application potential, some of the techniques have made a global impact, and some have demonstrated the potential for a global impact.

The Soft X-ray Imager(SXI)on the SMILE Mission

The Soft X-ray Imager(SXI)is part of the scientific payload of the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.SMILE is a joint science mission between the European Space Agency(ESA)and the Chinese Academy of Sciences(CAS)and is due for launch in 2025.SXI is a compact X-ray telescope with a wide field-of-view(FOV)capable of encompassing large portions of Earth’s magnetosphere from the vantage point of the SMILE orbit.SXI is sensitive to the soft X-rays produced by the Solar Wind Charge eXchange(SWCX)process produced when heavy ions of solar wind origin interact with neutral particles in Earth’s exosphere.SWCX provides a mechanism for boundary detection within the magnetosphere,such as the position of Earth’s magnetopause,because the solar wind heavy ions have a very low density in regions of closed magnetic field lines.The sensitivity of the SXI is such that it can potentially track movements of the magnetopause on timescales of a few minutes and the orbit of SMILE will enable such movements to be tracked for segments lasting many hours.SXI is led by the University of Leicester in the United Kingdom(UK)with collaborating organisations on hardware,software and science support within the UK,Europe,China and the United States.

An Improved Optical Flow Method for Image Registration with Large-scale Movements

在这份报纸，为图象登记的一个改进光流动方法被建议。它以它与一个起始的运动评估者改进光流动方法的方法是新奇的：扩大了阶段关联技术(EPCT ) ，用补偿前者的缺乏的后者的优点。以一种更多的详细方式，光流动方法能到达亚象素精确性并且计算象啁啾并且倾斜一样的复杂失真模式，这能被说但是与大规模运动弱。因为 EPCT 用象素水平精确性盖住大翻译和旋转的大小并且在精明的负担是有效的，它能为光流动方法被当作一个好起始的运动评估者。测试证明了特别，这个改进方法将显著地与大规模运动为图象提高登记表演并且对随机的噪音柔韧。