An image mosaic technique with non-overlapping regions based on microscopic vision in precision assembly

Microscopic vision has been widely applied in precision assembly.To achieve sufficiently high resolution in measurements for precision assembly when the sizes of the parts involved exceed the field of view of the vision system,an image mosaic technique must be used.In this paper,a method for constructing an image mosaic with non-overlapping areas with enhanced efficiency is proposed.First,an image mosaic model for the part is created using a geometric model of the measurement system installed on a X-Y-Z precision stages with high repeatability,and a path for image acquisition is established.Second,images are captured along the same path for a specified calibration plate,and an entire image is formed based on the given model.The measurement results obtained from the specified calibration plate are utilized to identify mosaic errors and apply compensation for the part requiring measurement.Experimental results show that the maximum error is less than 4μm for a camera with pixel equivalent 2.46μm,thereby demonstrating the accuracy of the proposed method.This image mosaic technique with non-overlapping regions can simplify image acquisition and reduce the workload involved in constructing an image mosaic.

Spatial-Spectral Joint Network for Cholangiocarcinoma Microscopic Hyperspectral Image Classification

Accurate histopathology classification is a crucial factor in the diagnosis and treatment of Cholangiocarcinoma(CCA).Hyperspectral images(HSI)provide rich spectral information than ordinary RGB images,making them more useful for medical diagnosis.The Convolutional Neural Network(CNN)is commonly employed in hyperspectral image classification due to its remarkable capacity for feature extraction and image classification.However,many existing CNN-based HSI classification methods tend to ignore the importance of image spatial context information and the interdependence between spectral channels,leading to unsatisfied classification performance.Thus,to address these issues,this paper proposes a Spatial-Spectral Joint Network(SSJN)model for hyperspectral image classification that utilizes spatial self-attention and spectral feature extraction.The SSJN model is derived from the ResNet18 network and implemented with the non-local and Coordinate Attention(CA)modules,which extract long-range dependencies on image space and enhance spatial features through the Branch Attention(BA)module to emphasize the region of interest.Furthermore,the SSJN model employs Conv-LSTM modules to extract long-range depen-dencies in the image spectral domain.This addresses the gradient disappearance/explosion phenom-ena and enhances the model classification accuracy.The experimental results show that the pro-posed SSJN model is more efficient in leveraging the spatial and spectral information of hyperspec-tral images on multidimensional microspectral datasets of CCA,leading to higher classification accuracy,and may have useful references for medical diagnosis of CCA.

The application of label-free imaging technologies in transdermal research for deeper mechanism revealing

The penetration behavior of topical substances in the skin not only relates to the transdermal delivery efficiency but also involves the safety and therapeutic effect of topical products,such as sunscreen and hair growth products.Researchers have tried to illustrate the transdermal process with diversified theories and technologies.Directly observing the distribution of topical substances on skin by characteristic imaging is the most convincing approach.Unfortunately,fluorescence labeling imaging,which is commonly used in biochemical research,is limited for transdermal research for most topical substances with a molecular mass less than 500 Da.Label-free imaging technologies possess the advantages of not requiring any macromolecular dyes,no tissue destruction and an extensive substance detection capability,which has enabled rapid development of such technologies in recent years and their introduction to biological tissue analysis,such as skin samples.Through the specific identification of topical substances and endogenous tissue components,label-free imaging technologies can provide abundant tissue distribution information,enrich theoretical and practical guidance for transdermal drug delivery systems.In this review,we expound the mechanisms and applications of the most popular label-free imaging technologies in transdermal research at present,compare their advantages and disadvantages,and forecast development prospects.

Spatial-spectral identication of abnormal leukocytes based on microscopic hyperspectral imaging technology

Screening and diagnosing of abnormal Leukocytes are crucial for the diagnosis of immune diseases and Acute Lymphoblastic Leukemia(ALL).As the deterioration of abnormal leukocytes is mainly due to the changes in the chromatin distribution,which signicantly affects the absorption and reflection of light,the spectral feature is proved to be important for leukocytes classication and identication.This paper proposes an accurate identication method for healthy and abnormal leukocytes based on microscopic hyperspectral imaging(HSI)technology which combines the spectral information.The segmentation of nucleus and cytoplasm is obtained by the morphological watershed algorithm.Then,the spectral features are extracted and combined with the spatial features.Based on this,the support vector machine(SVM)is applied for classication ofve types of leukocytes and abnormal leukocytes.Compared with different classication methods,the proposed method utilizes spectral features which highlight the differences between healthy leukocytes and abnormal leukocytes,improving the accuracy in the classication and identication of leukocytes.This paper only selects one subtype of ALL for test,and the proposed method can be applied for detection of other leukemia in the future.

LABEL-FREE DETECTION OF PROTEIN MICROARRAY WITH HIGH THROUGHPUT SURFACE PLASMON RESONANCE IMAGING(SPRI)

A surface plasmon resonance imaging(SPRI)system was developed for the discrimination of proteins on a gold surface.As a label-free and high-throughput technique,SPRI enables simultaneously monitoring of the biomolecular interactions at low concentrations.We used SPRI as a label-free and parallel method to detect different proteins based on protein microarray.Bovine Serum Albumin(BSA),Casein and Immunoglobulin G(IgG)were immobilized onto the Au surface of a gold-coated glass chip as spots forming a 6×6 matrix.These proteins can be discriminated directly by changing the incident angle of light.Excellent reproducibility for label-free detection of protein molecules was achieved.This SPRI platform represents a simple and robust method for performing high-sensitivity detection of protein microarray.

3D imaging lipidometry in single cell by in-flow holographic tomography

The most recent discoveries in the biochemical field are highlighting the increasingly important role of lipid droplets(LDs)in several regulatory mechanisms in living cells.LDs are dynamic organelles and therefore their complete characterization in terms of number,size,spatial positioning and relative distribution in the cell volume can shed light on the roles played by LDs.Until now,fluorescence microscopy and transmission electron microscopy are assessed as the gold standard methods for identifying LDs due to their high sensitivity and specificity.However,such methods generally only provide 2D assays and partial measurements.Furthermore,both can be destructive and with low productivity,thus limiting analysis of large cell numbers in a sample.Here we demonstrate for the first time the capability of 3D visualization and the full LD characterization in high-throughput with a tomographic phase-contrast flow-cytometer,by using ovarian cancer cells and monocyte cell lines as models.A strategy for retrieving significant parameters on spatial correlations and LD 3D positioning inside each cell volume is reported.The information gathered by this new method could allow more in depth understanding and lead to new discoveries on how LDs are correlated to cellular functions.

Microscopic cracking behaviors of rocks under uniaxial compression with microscopic multiphase heterogeneity by deep learning

Cracking behaviors of rocks significantly affect the safety and stability of the explorations of underground space and deep resources.To understand deeply the microscopic cracking process and mechanical property of rocks,X-ray micro-computed tomography(X-μCT)is applied to capture the rock microstructures.The digital color difference UNet(DCD-UNet)-based deep learning algorithm with 3D reconstruction is proposed to reconstruct the multiphase heterogeneity microstructure models of rocks.The microscopic cracking and mechanical properties are studied based on the proposed microstructure-based peridynamic model.Results show that the DCD-UNet algorithm is more effective to recognize and to represent the microscopic multiphase heterogeneity of rocks.As damage characteristic index of multiphase rocks increases,transgranular cracks in the same grain phase,transgranular and intergranular cracks of pore-grain phase,intergranular and secondary transgranular cracks and transgranular crack between different grains propagate.The ultimate microscopic failure modes of rocks are mainly controlled by the transgranular cracks-based T1-shear,T3-shear,T1-tension,T2-tension and T3-tension failures,and the intergranular cracks-based T1-tension,T1-shear and T3-shear failures under uniaxial compression.

Deep learning of rock microscopic images for intelligent lithology identification: Neural network comparison and selection

An intelligent lithology identification method is proposed based on deep learning of the rock microscopic images.Based on the characteristics of rock images in the dataset,we used Xception,MobileNet_v2,Inception_ResNet_v2,Inception_v3,Densenet121,ResNet101_v2,and ResNet-101 to develop microscopic image classification models,and then the network structures of seven different convolutional neural networks(CNNs)were compared.It shows that the multi-layer representation of rock features can be represented through convolution structures,thus better feature robustness can be achieved.For the loss function,cross-entropy is used to back propagate the weight parameters layer by layer,and the accuracy of the network is improved by frequent iterative training.We expanded a self-built dataset by using transfer learning and data augmentation.Next,accuracy(acc)and frames per second(fps)were used as the evaluation indexes to assess the accuracy and speed of model identification.The results show that the Xception-based model has the optimum performance,with an accuracy of 97.66%in the training dataset and 98.65%in the testing dataset.Furthermore,the fps of the model is 50.76,and the model is feasible to deploy under different hardware conditions and meets the requirements of rapid lithology identification.This proposed method is proved to be robust and versatile in generalization performance,and it is suitable for both geologists and engineers to identify lithology quickly.

Assessing pathological features of breast cancer via the multimodal information of multiphoton and Raman imaging

For unveiling the pathological evolution of breast cancer, nonlinear multiphoton microscopic(MPM) and confocal Raman microspectral imaging(CRMI) techniques were both utilized to address the structural and constitutional characteristics of healthy(H), ductal carcinoma in situ(DCIS), and invasive ductal carcinoma(IDC) tissues. MPM-based techniques,including two-photon excited fluorescence(TPEF) and second harmonic generation(SHG), visualized label-free and the fine structure of breast tissue. Meanwhile, CRMI not only presented the chemical images of investigated samples with the K-mean cluster analysis method(KCA), but also pictured the distribution of components in the scanned area through univariate imaging. MPM images illustrated that the cancer cells first arranged around the basement membrane of the duct,then proliferated to fill the lumens of the duct, and finally broke through the basement membrane to infiltrate into the stroma.Although the Raman imaging failed to visualize the cell structure with high resolution, it explained spectroscopically the gradual increase of nucleic acid and protein components inside the ducts as cancer cells proliferated, and displayed the distribution pattern of each biological component during the evolution of breast cancer. Thus, the combination of MPM and CRMI provided new insights into the on-site pathological diagnosis of malignant breast cancer, also ensured technical support for the development of multimodal optical imaging techniques for precise histopathological analysis.

Label-free visualization of cholesteatoma in the mastoid and tympanic membrane using CARS microscopy

Objective:The present study aimed to evaluate the possibility of using coherent anti-Stokes Raman spectroscopy（CARS） microscopy to determine the specific molecular morphology of cholesteatoma by detecting the natural vibrational contrast of the chemical bonds without any staining.Materials and methods:Specimens from the mastoid and tympanic membrane with and without cholesteatoma were analyzed using CARS microscopy,two-photon excited fluorescence（TPEF） microscopy,and the second harmonic generation（SHG） microscopy.Results:In cholesteatoma tissues from the mastoid,a strong resonant signal at 2845 cm;was observed by CARS,which indicated the detection of the CH;hydro-carbon lipid bonds that do not generate visible signals at 2940 cm;suggestive of CH;bonds in amino acids.A strong resonant signal at 2940 cm;appeared in an area of the same specimen,which also generated abundant signals by TPEF and SHG microscopy at 817 nm,which was suggestive of collagen.In the tympanic membrane specimen with cholesteatoma,a strong resonant signal with corrugated morphology was detected,which indicated the presence of lipids.A strong signal was detected in the tympanic membrane with chronic otitis media using TPEF/SHG at 817 nm,which indicated collagen enrichment.The CARS and TPEF/SHG images were in accordance with the histology results.Conclusion:These results suggest the need to develop a novel CARS microendoscope that can be used in combination with TPEF/SHG to distinguish cholesteatoma from inflammatory tissues.

Automatic Leukaemia Segmentation Approach for Blood Cancer Classification Using Microscopic Images

Leukaemia is a type of blood cancer that is caused by undeveloped White Blood Cells(WBC),and it is also called a blast blood cell.In the marrow of human bones,leukaemia is developed and is responsible for blood cell generation with leukocytes and WBC,and if any cell gets blasted,then it may become a cause of death.Therefore,the diagnosis of leukaemia in its early stages helps greatly in the treatment along with saving human lives.Subsequently,in terms of detection,image segmentation techniques play a vital role,and they turn out to be the important image processing steps for the extraction of feature patterns from the Acute Lymphoblastic Leukaemia(ALL)type of blood cancer.Moreover,the image segmentation technique focuses on the division of cells by segmenting a microscopic image into background and cancer blood cell nucleus,which is well-known as the Region Of Interest(ROI).As a result,in this article,we attempt to build a segmentation technique capable of solving blood cell nucleus segmentation issues using four distinct scenarios,including K-means,FCM(Fuzzy Cmeans),K-means with FFA(Firefly Algorithm),and FCM with FFA.Also,we determine the most effective method of blood cell nucleus segmentation,which we subsequently use for the Leukaemia classification model.Finally,using the Convolution Neural Network(CNN)as a classifier,we developed a leukaemia cancer classification model from the microscopic images.The proposed system’s classification accuracy is tested using the CNN to test the model on the ALL-IDB dataset and equate it to the current state of the art.In terms of experimental analysis,we observed that the accuracy of the model is near to 99%,and it is far better than other existing models that are designed to segment and classify the types of leukaemia cancer in terms of ALL.

Diffusivity-limited q-space trajectory imaging

Q-space trajectory imaging(QTI)allows non-invasive estimation of microstructural features of heterogeneous porous media via diffusion magnetic resonance imaging performed with generalised gradient waveforms.A recently proposed constrained estimation framework,called QTI+,improved QTI's resilience to noise and data sparsity,thus increasing the reliability of the method by enforcing relevant positivity constraints.In this work we consider expanding the set of constraints to be applied during the fitting of the QTI model.We show that the additional conditions,which introduce an upper bound on the diffusivity values,further improve the retrieved parameters on a publicly available human brain dataset as well as on data acquired from healthy volunteers using a scanner-ready protocol.

The Primary Study of Microscopical Focus Problem in the Color Image of Pigmented Spots

Pigmented spot is an important branch in the science of skin. But when processing those images, the microscopical focusing problem arises. It affects the image recognition later. In order to find the best method to solve it, comparison and analysis are given to various existing methods of image fusion in this paper . The conclusion is wavelet transform based on pixel-level.

Micro-Scanning Error Correction Technique for an Optical Micro-Scanning Thermal Microscope Imaging System

An error correction technique for the micro-scanning instrument of the optical micro-scanning thermal microscope imaging system is proposed. The technique is based on micro-scanning technology combined with the proposed second-order oversampling reconstruction algorithm and local gradient image reconstruction algorithm. In this paper, we describe the local gradient image reconstruction model, the error correction technique, down-sampling model and the error correction principle. In this paper, we use a Lena original image and four low-resolution images obtained from the standard half-pixel displacement to simulate and verify the effectiveness of the proposed technique. In order to verify the effectiveness of the proposed technique, two groups of low-resolution thermal microscope images are collected by the actual thermal microscope imaging system for experimental study. Simulations and experiments show that the proposed technique can reduce the optical micro-scanning errors, improve the imaging effect of the system and improve the system's spatial resolution. It can be applied to other electro-optical imaging systems to improve their resolution.

Optical micro-scanning location calibration of thermal microscope imaging system

A method of micro-scanning location adaptive calibration was proposed, which was real- ized by the digital image micro-displacement estimation. With geometric calculation, this calibration method used the displacement estimation of two thermal microscope images to get the size and direc- tion of each scanning location calibration angle. And each location calibration process was repeated according to the offset given by the system beforehand. The comparison experiments of sequence oversampling reconstruction before and after the micro-scanning location calibration were done. The results showed that the calibration method effectively improved the thermal microscope imaging qual- ity.

X-ray phase-sensitive microscope imaging with a grating interferometer:Theory and simulation

A general theoretical framework is presented to explain the formation of the phase signal in an x-ray microscope integrated with a grating interferometer,which simultaneously enables the high spatial resolution imaging and the improved image contrast.By using this theory,several key parameters of phase contrast imaging can be predicted,for instance,the fringe visibility and period,and the conversion condition from the differential phase imaging(DPI)to the phase difference imaging(PDI).Additionally,numerical simulations are performed with certain x-ray optical components and imaging geometry.Comparison with the available experimental measurement[Appl.Phys.Lett.113063105(2018)]demonstrates the accuracy of this developed quantitative analysis method of x-ray phase-sensitive microscope imaging.

Adaptive Gradient-Based and Anisotropic Diffusion Equation Filtering Algorithm for Microscopic Image Preprocessing

In image acquisition process, the quality of microscopic images will be degraded by electrical noise, quantizing noise, light illumination etc. Hence, image preprocessing is necessary and important to improve the quality. The background noise and pulse noise are two common types of noise existing in microscopic images. In this paper, a gradient-based anisotropic filtering algorithm was proposed, which can filter out the background noise while preserve object boundary effectively. The filtering performance was evaluated by comparing that with some other filtering algorithms.

Oversample Reconstruction Based on a Strong Inter-Diagonal Matrix for an Optical Microscanning Thermal Microscope Imaging System

Based on a strong inter-diagonal matrix and Taylor series expansions,an oversample reconstruction method was proposed to calibrate the optical micro-scanning error. The technique can obtain regular 2 ×2 microscanning undersampling images from the real irregular undersampling images,and can then obtain a high spatial oversample resolution image. Simulations and experiments show that the proposed technique can reduce optical micro-scanning error and improve the system＇s spatial resolution. The algorithm is simple,fast and has low computational complexity. It can also be applied to other electro-optical imaging systems to improve their spatial resolution and has a widespread application prospect.

Recent advances in nonlinear optics for bio‐imaging applications

Nonlinear optics,which is a subject for studying the interaction between intense light and materials,has great impact on various research fields.Since many structures in biological tissues exhibit strong nonlinear optical effects,nonlinear optics has been widely applied in biomedical studies.Especially in the aspect of bio-imaging,nonlinear optical techniques can provide rapid,label-free and chemically specific imaging of biological samples,which enable the investigation of biological processes and analysis of samples beyond other microscopy techniques.In this review,we focus on the introduction of nonlinear optical processes and their applications in bio-imaging as well as the recent advances in this filed.Our perspective of this field is also presented.

Autofluorescence imaging endoscopy for identifi cation and assessment of inflammatory ulcerative colitis

AIM:To validate the clinical relevance of autofluores-cence imaging(AFI)endoscopy for the assessment of inflammatory ulcerative colitis(UC).METHODS:A total of 572 endoscopic images were se-lected from 42 UC patients:286 taken with white light imaging(WLI)and 286 with AFI from the same sites.WLI images were assessed for overall mucosal inflammation according to Mayo endoscopic subscore(MES),and for seven characteristic endoscopic features.Likewise,AFI photographs were scored according to relative abundance of red,green and blue color com-ponents within each image based on an RGB additive color model.WLI and AFI endoscopic scores from the same sites were compared.Histological evaluation of biopsies was according to the Riley Index.RESULTS:Relative to red(r=0.52,P<0.01)or blue(r=0.56,P<0.01)color component,the green color component of AFI(r=-0.62,P<0.01)corresponded more closely with mucosal inflammation sites.There were signif icant differences in green color components between MES-0(0.396±0.043)and MES-1(0.340± 0.035)(P<0.01),and between MES-1 and ≥ MES-2(0.318±0.037)(P<0.01).The WLI scores for "vascu-lar patterns"(r=-0.65,P<0.01),"edema"(r=-0.62,P<0.01),histology scores for "polymorphonuclear cells in the lamina propria"(r=-0.51,P<0.01)and "crypt architectural irregularities"(r=-0.51,P<0.01)showed correlation with the green color component of AFI.There were significant differences in green color components between limited(0.399± 0.042)and extensive(0.375±0.044)(P=0.014)polymorpho-nuclear cell inf iltration within MES-0.As the severity of the mucosal inflammation increased,the green color component of AFI decreased.The AFI green color com-ponent was well correlated with the characteristic en-doscopic and histological inflammatory features of UC.CONCLUSION:AFI has application in detecting inflammatory lesions,including microscopic activity in the co-lonic mucosa of UC patients,based on the green color component of images.