Mangrove monitoring and extraction based on multi-source remote sensing data:a deep learning method based on SAR and optical image fusion

Mangroves are indispensable to coastlines,maintaining biodiversity,and mitigating climate change.Therefore,improving the accuracy of mangrove information identification is crucial for their ecological protection.Aiming at the limited morphological information of synthetic aperture radar(SAR)images,which is greatly interfered by noise,and the susceptibility of optical images to weather and lighting conditions,this paper proposes a pixel-level weighted fusion method for SAR and optical images.Image fusion enhanced the target features and made mangrove monitoring more comprehensive and accurate.To address the problem of high similarity between mangrove forests and other forests,this paper is based on the U-Net convolutional neural network,and an attention mechanism is added in the feature extraction stage to make the model pay more attention to the mangrove vegetation area in the image.In order to accelerate the convergence and normalize the input,batch normalization(BN)layer and Dropout layer are added after each convolutional layer.Since mangroves are a minority class in the image,an improved cross-entropy loss function is introduced in this paper to improve the model’s ability to recognize mangroves.The AttU-Net model for mangrove recognition in high similarity environments is thus constructed based on the fused images.Through comparison experiments,the overall accuracy of the improved U-Net model trained from the fused images to recognize the predicted regions is significantly improved.Based on the fused images,the recognition results of the AttU-Net model proposed in this paper are compared with its benchmark model,U-Net,and the Dense-Net,Res-Net,and Seg-Net methods.The AttU-Net model captured mangroves’complex structures and textural features in images more effectively.The average OA,F1-score,and Kappa coefficient in the four tested regions were 94.406%,90.006%,and 84.045%,which were significantly higher than several other methods.This method can provide some technical support for the monitoring and protection of mangrove ecosystems.

Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

Fusion of SAR and Optical Image for Sea Ice Extraction

It is difficult to balance local details and global distribution using a single source image in marine target detection of a large scene.To solve this problem,a technique based on the fusion of optical image and synthetic aperture radar(SAR)image for the extraction of sea ice is proposed in this paper.The Band 2(B2 image of Sentinel-2(S2 in the research area is selected as optical image data.Preprocessing on the optical image,such as resampling,projection transformation and format conversion,are conducted to the S2 dataset before fusion.Imaging characteristics of the sea ice have been analyzed,and a new deep learning(DL)model,OceanTDL5,is built to detect sea ices.The fusion of the Sentinel-1(S1 and S2 images is realized by solving the optimal pixel values based on deriving Poisson Equation.The experimental results indicate that the use of a fused image improves the accuracy of sea ice detection compared with the use of a single data source.The fused image has richer spatial details and a clearer texture compared with the original optical image,and its material sense and color are more abundant.

Optical image encryption algorithm based on a new four-dimensional memristive hyperchaotic system and compressed sensing

Some existing image encryption schemes use simple low-dimensional chaotic systems, which makes the algorithms insecure and vulnerable to brute force attacks and cracking. Some algorithms have issues such as weak correlation with plaintext images, poor image reconstruction quality, and low efficiency in transmission and storage. To solve these issues,this paper proposes an optical image encryption algorithm based on a new four-dimensional memristive hyperchaotic system(4D MHS) and compressed sensing(CS). Firstly, this paper proposes a new 4D MHS, which has larger key space, richer dynamic behavior, and more complex hyperchaotic characteristics. The introduction of CS can reduce the image size and the transmission burden of hardware devices. The introduction of double random phase encoding(DRPE) enables this algorithm has the ability of parallel data processing and multi-dimensional coding space, and the hyperchaotic characteristics of 4D MHS make up for the nonlinear deficiency of DRPE. Secondly, a construction method of the deterministic chaotic measurement matrix(DCMM) is proposed. Using DCMM can not only save a lot of transmission bandwidth and storage space, but also ensure good quality of reconstructed images. Thirdly, the confusion method and diffusion method proposed are related to plaintext images, which require both four hyperchaotic sequences of 4D MHS and row and column keys based on plaintext images. The generation process of hyperchaotic sequences is closely related to the hash value of plaintext images. Therefore, this algorithm has high sensitivity to plaintext images. The experimental testing and comparative analysis results show that proposed algorithm has good security and effectiveness.

Mountain Glacier Flow Velocities Analyzed from Satellite Optical Images

Mountain glaciers are sensitive to environment. It is important to acquire ice flow velocities over time for glacier research and hazard forecast. For this paper, cross-correlating of optical images is used to monitor ice flow velocities, and an improvement, which is called ＂moving grid,＂ is made to this method. For this research, two remote-sensing images in a certain glacier area, dur-ing different times are selected. The first image is divided into grids, and we calculated the correlation coefficient of each window in the grid with the window on the second image. The window with the highest correlation coefficient is considered the counter-part one on the first image. The displacement of the two corresponding windows is the movement of the glacier, and it is used to calculate glacier surface velocity. Compared to the traditional way of dividing an image with ascertain grid, this method uses small steps to move the grid from one location to another adjacent location until the whole glacier area is covered in the image, thus in-creasing corresponding point density. We selected a glacier in the Tianshan Mountains for this experiment and used two re-mote-sensing images with a 10-year interval to determine this method.

Cerebrovascular segmentation from mesoscopic optical images using Swin Transformer

Vascular segmentation is a crucial task in biomedical image processing,which is significant for analyzing and modeling vascular networks under physiological and pathological states.With advances in fluorescent labeling and mesoscopic optical techniques,it has become possible to map the whole-mouse-brain vascular networks at capillary resolution.However,segmenting vessels from mesoscopic optical images is a challenging task.The problems,such as vascular signal discontinuities,vessel lumens,and background fluorescence signals in mesoscopic optical images,belong to global semantic information during vascular segmentation.Traditional vascular segmentation methods based on convolutional neural networks(CNNs)have been limited by their insufficient receptive fields,making it challenging to capture global semantic information of vessels and resulting in inaccurate segmentation results.Here,we propose SegVesseler,a vascular segmentation method based on Swin Transformer.SegVesseler adopts 3D Swin Transformer blocks to extract global contextual information in 3D images.This approach is able to maintain the connectivity and topology of blood vessels during segmentation.We evaluated the performance of our method on mouse cerebrovascular datasets generated from three different labeling and imaging modalities.The experimental results demonstrate that the segmentation effect of our method is significantly better than traditional CNNs and achieves state-of-the-art performance.

The Application of Digital Optical 3D Image Analyzer Evaskin in The Evaluation of Wrinkles

To verify the effectiveness of digital optical 3D image analyzer EvaSKIN in the objective and quantitative evaluation of wrinkles.A total of 115 subjects were recruited,the facial images of the subjects were collected by digital optical 3D image analyzer and manual camera,the changes of crow’s feet with age were analyzed.Pictures obtained by manual photography can be directly used for observation and preliminary grading of wrinkles.However,the requirements for evaluators are high,and the results are prone to errors,which will affect the accuracy of the evaluation.Therefore,skilled raters are needed.Compared with the manual photography method,the digital optical 3D image analyzer EvaSKIN can realize three-dimensional extraction of wrinkles,and obtain the change trend of crow’s feet with age.20~30 years old,wrinkles begin to appear slowly;wrinkles will increase rapidly at the age of 30~50;The length of 50~60 year old wrinkles is basically fixed,the wrinkles develop longitudewise,gradually widen and deepen,and the area,depth and volume increase is obvious,and the skin aging condition is intensified.the digital optical 3D image analyzer EvaSKIN realizes the 3D extraction of wrinkles,quantifies the circumference,area,average depth,maximum depth and volume of wrinkles,realizes the objective and quantitative evaluation of wrinkle state,is more accurate in the measurement of wrinkles,and provides a new instrument and method for the evaluation of wrinkles.it is a perfect and supplement to the traditional evaluation methods,and to a certain extent,it helps the research and development and evaluation institutions of cosmetics to obtain more abundant and three-dimensional data support.

Optical image processing using acousto-optic modulators as programmable volume holograms: a review [Invited]

Bragg processing using a volume hologram offers an alternative in optical image processing in contrast to Fourier-plane processing. By placing a volume hologram near the object in an optical imaging setup, we achieve Bragg processing. In this review, we discuss various image processing methods achievable with acousto-optic modulators as dynamic and programmable volume holograms. In particular, we concentrate on the discussion of various differentiation operations leading to edge extraction capabilities.

PCA-based sea-ice image fusion of optical data by HIS transform and SAR data by wavelet transform

Sea ice as a disaster has recently attracted a great deal of attention in China. Its monitoring has become a routine task for the maritime sector. Remote sensing, which depends mainly on SAR and optical sensors, has become the primary means for sea-ice research. Optical images contain abundant sea-ice multi-spectral in-formation, whereas SAR images contain rich sea-ice texture information. If the characteristic advantages of SAR and optical images could be combined for sea-ice study, the ability of sea-ice monitoring would be im-proved. In this study, in accordance with the characteristics of sea-ice SAR and optical images, the transfor-mation and fusion methods for these images were chosen. Also, a fusion method of optical and SAR images was proposed in order to improve sea-ice identification. Texture information can play an important role in sea-ice classification. Haar wavelet transformation was found to be suitable for the sea-ice SAR images, and the texture information of the sea-ice SAR image from Advanced Synthetic Aperture Radar （ASAR） loaded on ENVISAT was documented. The results of our studies showed that, the optical images in the hue-intensi-ty-saturation （HIS） space could reflect the spectral characteristics of the sea-ice types more efficiently than in the red-green-blue （RGB） space, and the optical image from the China-Brazil Earth Resources Satellite （CBERS-02B） was transferred from the RGB space to the HIS space. The principal component analysis （PCA） method could potentially contain the maximum information of the sea-ice images by fusing the HIS and texture images. The fusion image was obtained by a PCA method, which included the advantages of both the sea-ice SAR image and the optical image. To validate the fusion method, three methods were used to evaluate the fused image, i.e., objective, subjective, and comprehensive evaluations. It was concluded that the fusion method proposed could improve the ability of image interpretation and sea-ice identification.

Mapping the bathymetry of shallow coastal water using singleframe fine-resolution optical remote sensing imagery

This paper presents a bathymetry inversion method using single-frame fine-resolution optical remote sensing imagery based on ocean-wave refraction and shallow-water wave theory. First, the relationship among water depth, wavelength and wave radian frequency in shallow water was deduced based on shallow-water wave theory. Considering the complex wave distribution in the optical remote sensing imagery, Fast Fourier Transform （FFT） and spatial profile measurements were applied for measuring the wavelengths. Then, the wave radian frequency was calculated by analyzing the long-distance fluctuation in the wavelength, which solved a key problem in obtaining the wave radian frequency in a single-frame image. A case study was conducted for Sanya Bay of Hainan Island, China. Single-flame fine-resolution optical remote sensing imagery from QuickBird satellite was used to invert the bathymetry without external input parameters. The result of the digital elevation model （DEM） was evaluated against a sea chart with a scale of 1：25 000. The root-mean-square error of the inverted bathymetry was 1.07 m, and the relative error was 16.2%. Therefore, the proposed method has the advantages including no requirement for true depths and environmental parameters, and is feasible for mapping the bathymetry of shallow coastal water.

SEGMENTATION AND CORRELATION OF OPTICAL COHERENCE TOMOGRAPHY AND X-RAY IMAGES FOR BREAST CANCER DIAGNOSTICS

Pre-operative X ray mammography and int raoperative X-ray specimen radiography are routinely used to identify breast cancer pathology.Recent advances in optical coherence tomography(OCT)have enabled its 1use for the intraoperative assessment of surgical margins during breast cancer surgery.While each modality offers distinct contrast of normal and pathological features,there is an essential need to correlate image based features between the two modalities to take adv antage of the diagnostic capabilities of each technique.We compare OCT to X-ray images of resected human breast tissue and correlate different tissue features between modalities for future use in real-tine intraoperative OCT imaging.X ray imaging(specimen radiography)is currently used during surgical breast cancer procedures to verify tumor margins,but cannot image tissue in situ.OCT has the potential to solve this problem by providing intrao-perative imaging of the resected specimen as well as the in situ tumor cavity.OCT and micro-CT(X-ray)images are automatically segmented using different computational approaches,and quantitatively compared to determine the ability of these algorithms to automat ically differentiate regions of adipose tissue from tumor.Furthermore,two-dimensional(2D)and three-dimensional(3D)results are compared.These correlations,combined with real-time intraoperative OCT,have the potential to identify possible regions of tumor within breast tissue which correlate to tumor regions identified previously on X-ray imaging(mammography or specimen radiography).

An Intelligent Detection Method for Optical Remote Sensing Images Based on Improved YOLOv7

To address the issue of imbalanced detection performance and detection speed in current mainstream object detection algorithms for optical remote sensing images,this paper proposes a multi-scale object detection model for remote sensing images on complex backgrounds,called DI-YOLO,based on You Only Look Once v7-tiny(YOLOv7-tiny).Firstly,to enhance the model’s ability to capture irregular-shaped objects and deformation features,as well as to extract high-level semantic information,deformable convolutions are used to replace standard convolutions in the original model.Secondly,a Content Coordination Attention Feature Pyramid Network(CCA-FPN)structure is designed to replace the Neck part of the original model,which can further perceive relationships between different pixels,reduce feature loss in remote sensing images,and improve the overall model’s ability to detect multi-scale objects.Thirdly,an Implicitly Efficient Decoupled Head(IEDH)is proposed to increase the model’s flexibility,making it more adaptable to complex detection tasks in various scenarios.Finally,the Smoothed Intersection over Union(SIoU)loss function replaces the Complete Intersection over Union(CIoU)loss function in the original model,resulting in more accurate prediction of bounding boxes and continuous model optimization.Experimental results on the High-Resolution Remote Sensing Detection(HRRSD)dataset demonstrate that the proposed DI-YOLO model outperforms mainstream target detection algorithms in terms of mean Average Precision(mAP)for optical remote sensing image detection.Furthermore,it achieves Frames Per Second(FPS)of 138.9,meeting fast and accurate detection requirements.

Experimental Study on the Variation of Optical Imaging Characteristics with Zenith Angle due to Internal Solitary Waves in Sunglint

Internal solitary waves(ISWs)change the roughness of the sea surface,thus producing dark and bright bands in optical images.However,reasons for changes in imaging characteristics with the solar zenith angle remain unclear.In this paper,the optical imaging pattern of ISWs in sunglint under different zenith angles of the light source is investigated by collecting optical images of ISWs through physical simulation.The experiment involves setting 10 zenith angles of the light source,which are divided into area a the optical images of ISWs in the three areas show dark-bright mode,single bright band,and bright-dark mode,which are consistent with those observed by optical remote sensing.In addition,this study analyzed the percentage of the dark and bright areas of the bands and the change in the relative gray difference and found changes in both areas under different zenith angles of the light source.The MODIS and ASAR images display a similar brightness-darkness distance of the same ISWs.Therefore,the relationship between the brightness-darkness distance and the characteristic half-width of ISWs is determined in accordance with the eKdV theory and the imaging mechanism of ISWs of the SAR image.Overall,the relationship between them in the experiment is almost consistent with the theoretical result.

Nonlinear Optical Spatial Filtering for Medical Image Processing

The nonlinear properties of Tris(acetylacetonato) Manganese(III) are used to manipulate the spatial frequencies at the Fourier plane using 4f-z scan. The technique is a simple self-adaptive all-optical system, which performs image processing and nonlinear optical measurements at the same time. Preferred spatial frequencies can be selected by shifting the nonlinear sample through the focus. Edge enhancement was demonstrated by filtering of low frequency with the nonlinear material at the Fourier plane.

Reconstruction algorithm for cross-waveband optical computing imaging

In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds of setups which are able to transform non-visible into visible light imaging,wherein their computing process is replaced by a camera integration mode.The image captured by the camera has a low contrast,so here we present an algorithm that can realize a high quality image in near-infrared to visible cross-waveband imaging.The scheme is verified both by simulation and in actual experiments.The setups demonstrate the great potential for single-pixel imaging and high-speed cross-waveband imaging for future practical applications.

Detecting early changes in choroidal vascularity and thickness using optical coherence tomography in patients with corneal crosslinking for keratoconus

AIM:To investigate changes in choroidal thickness and vascularity in keratoconus patients treated with corneal crosslinking.METHODS:This study evaluated 28 eyes of 22 patients with keratoconus who underwent corneal crosslinking.The choroidal thicknesses were evaluated on enhanced depth imaging optical coherence tomography at the preoperative and postoperative 3d,1,and 3mo.Choroidal thickness in the four cardinal quadrants and the fovea were evaluated.The choroidal vascularity index was also calculated.RESULTS:There was no significant difference in central choroidal thickness between the preoperative and postoperative 3d,1mo(P>0.05).There was a significant increase in the 3mo(P=0.034)and a significant decrease in the horizontal choroidal vascularity index on the postoperative 3d(P=0.014),there was no statistically significant change in vertical axes and other visits in horizontal sections(P>0.05).CONCLUSION:This study sheds light on choroidal changes in postoperative corneal crosslinking for keratoconus.While it suggests the procedure’s relative safety for submacular choroid,more extensive research is necessary to confirm these findings and their clinical significance.

Image edge detection based on beamlet transform

Combining beamlet transform with steerable filters, a new edge detection method based on line gradient is proposed. Compared with operators based on point local properties, the edge-detection results with this method achieve higher SNR and position accuracy, and are quite helpful for image registration, object identification, etc. Some edge-detection experiments on optical and SAR images that demonstrate the significant improvement over classical edge operators axe also presented. Moreover, the template matching result based on edge information of optical reference image and SAR image also proves the validity of this method.

Association of colorectal cancer with pathogenic Escherichia coli: Focus on mechanisms using optical imaging

AIM: To investigate the molecular or cellular mechanisms related to the infection of epithelial colonic mucosa by pks-positive Escherichia coli(E. coli) using optical imaging.METHODS: We choose to evaluate the tumor metabolic activity using a fluorodeoxyglucose analogue as 2-deoxyglucosone fluorescent probes and to correlate it with tumoral volume(mm^3). Inflammation measuring myeloperoxidase(MPO) activity and reactive oxygen species production was monitored by a bioluminescent(BLI) inflammation probe and related to histological examination and MPO levels by enzyme-linked immunosorbent assay(ELISA) on tumor specimens. The detection and quantitation of these two signals were validated on a xenograft model of human colon adenocarcinoma epithelial cells(HCT116) in nude mice infected with a pks-positive E. coli. The inflammatory BLI signal was validated intra-digestively in the colitisCEABAC10 DSS models, which mimicked Crohn's disease. RESULTS: Using a 2-deoxyglucosone fluorescent probe, we observed a high and specific HCT116 tumor uptake in correlation with tumoral volume(P = 0.0036). Using the inflammation probe targeting MPO, we detected a rapid systemic elimination and a significant increase of the BLI signal in the pks-positive E. coli-infected HCT116 xenograft group(P < 0.005). ELISA confirmed that MPO levels were significantly higher(1556 ± 313.6 vs 234.6 ± 121.6 ng/m L P = 0.001) in xenografts infected with the pathogenic E. coli strain. Moreover, histological examination of tumor samples confirmed massive infiltration of pks-positive E. coli-infected HCT116 tumors by inflammatory cells compared to the uninfected group. These data showed that infection with the pathogenic E. coli strain enhanced inflammation and ROS production in tumors before tumor growth. Moreover, we demonstrated that the intra-digestive monitoring of inflammation is feasible in a reference colitis murine model(CEABAC10/DSS).CONCLUSION: Using BLI and fluorescence optical imaging, we provided tools to better understand hostpathogen interactions at the early stage of disease, such as inflammatory bowel disease and colorectal cancer.

Pencil-beam scanning catheter for intracoronary optical coherence tomography

Current gradient-index(GRIN)lens based proximal-driven intracoronary optical coherence tomography(ICOCT)probes consist of a spacer and a GRIN lens with large gradient constant.This design provides great flexibility to control beam profiles,but the spacer length should be well controlled to obtain desired beam profiles and thus it sets an obstacle in mass catheter fabrication.Besides,although GRIN lens with large gradient constant can provide tight focus spot,it has short depth of focus and fast-expanded beam which leads to poor lateral resolution for deep tissue.In this paper,a type of spacer-removed probe is demonstrated with a small gradient constant GRIN lens.This design simplifies the fabrica-tion process and is suitable for mass production.The output beam of the catheter is a narrow nearly collimated light beam,referred to as pencil beam here.The full width at half maximum beam size varies from 35.1μm to 75.3μm in air over 3-mm range.Probe design principles are elaborated with probe/catheter fabrication and performance test.The in vivo imaging of the catheter was verified by a clinical ICOCT system.Those results prove that this novel pencil-beam scanning catheter is potentially a good choice for ICOCT systems.

In Vivo Tumor-Targeted Dual-Modality PET/Optical Imaging with a Yolk/Shell-Structured Silica Nanosystem

Silica nanoparticles have been one of the most promising nanosystems for biomedical applications due to their facile surface chemistry and non-toxic nature. However, it is still challenging to effectively deliver them into tumor sites and noninvasively visualize their in vivo biodistribution with excellent sensitivity and accuracy for effective cancer diagnosis. In this study, we design a yolk/shell-structured silica nanosystem ^(64) Cu-NOTAQD@HMSN-PEG-TRC105, which can be employed for tumor vasculature targeting and dual-modality PET/optical imaging, leading to superior targeting specificity, excellentimaging capability and more reliable diagnostic outcomes.By combining vasculature targeting, pH-sensitive drug delivery, and dual-modality imaging into a single platform,as-designed yolk/shell-structured silica nanosystems may be employed for the future image-guided tumor-targeted drug delivery, to further enable cancer theranostics.