NIR-Ⅱ Excitation and NIR-I Emission Based Two-photon Fluorescence Lifetime Microscopic Imaging Using Aggregation-induced Emission Dots

Near-infrared(NIR)lights are powerful tools to conduct deep-tissue imaging since NIR-Ⅰ wavelengths hold less photon absorption and NIR-Ⅱ wavelengths serve low photon scattering in the biological tissues compared with visible lights.Two-photon fluorescence lifetime microscopy(2PFLM)can utilize NIR-Ⅱ excitation and NIR-Ⅰ emission at the same time with the assistance of a well-designed fluorescent agent.Aggregation induced emission(AIE)dyes are famous for unique optical properties and could serve a large two-photon absorption(2PA)cross-section as aggregated dots.Herein,we report two-photon fluorescence lifetime microscopic imaging with NIR-Ⅱ excitation and NIR-Ⅰ emission using a novel deep-red AIE dye.The AIE-gens held a 2PA cross-section as large as 1.61×10^(4)GM at 1040 nm.Prepared AIE dots had a two-photon fluorescence peak at 790 nm and a stable lifetime of 2.2 ns under the excitation of 1040 nm femtosecond laser.The brain vessels of a living mouse were vividly reconstructed with the two-photon fluorescence lifetime information obtained by our home-made 2PFLM system.Abundant vessels as small as 3.17µm were still observed with a nice signal-background ratio at the depth of 750µm.Our work will inspire more insight into the improvement of the working wavelength of fluorescent agents and traditional 2PFLM.

Assessing pathological features of breast cancer via the multimodal information of multiphoton and Raman imaging

For unveiling the pathological evolution of breast cancer, nonlinear multiphoton microscopic(MPM) and confocal Raman microspectral imaging(CRMI) techniques were both utilized to address the structural and constitutional characteristics of healthy(H), ductal carcinoma in situ(DCIS), and invasive ductal carcinoma(IDC) tissues. MPM-based techniques,including two-photon excited fluorescence(TPEF) and second harmonic generation(SHG), visualized label-free and the fine structure of breast tissue. Meanwhile, CRMI not only presented the chemical images of investigated samples with the K-mean cluster analysis method(KCA), but also pictured the distribution of components in the scanned area through univariate imaging. MPM images illustrated that the cancer cells first arranged around the basement membrane of the duct,then proliferated to fill the lumens of the duct, and finally broke through the basement membrane to infiltrate into the stroma.Although the Raman imaging failed to visualize the cell structure with high resolution, it explained spectroscopically the gradual increase of nucleic acid and protein components inside the ducts as cancer cells proliferated, and displayed the distribution pattern of each biological component during the evolution of breast cancer. Thus, the combination of MPM and CRMI provided new insights into the on-site pathological diagnosis of malignant breast cancer, also ensured technical support for the development of multimodal optical imaging techniques for precise histopathological analysis.

Spatial-spectral identication of abnormal leukocytes based on microscopic hyperspectral imaging technology

Screening and diagnosing of abnormal Leukocytes are crucial for the diagnosis of immune diseases and Acute Lymphoblastic Leukemia(ALL).As the deterioration of abnormal leukocytes is mainly due to the changes in the chromatin distribution,which signicantly affects the absorption and reflection of light,the spectral feature is proved to be important for leukocytes classication and identication.This paper proposes an accurate identication method for healthy and abnormal leukocytes based on microscopic hyperspectral imaging(HSI)technology which combines the spectral information.The segmentation of nucleus and cytoplasm is obtained by the morphological watershed algorithm.Then,the spectral features are extracted and combined with the spatial features.Based on this,the support vector machine(SVM)is applied for classication ofve types of leukocytes and abnormal leukocytes.Compared with different classication methods,the proposed method utilizes spectral features which highlight the differences between healthy leukocytes and abnormal leukocytes,improving the accuracy in the classication and identication of leukocytes.This paper only selects one subtype of ALL for test,and the proposed method can be applied for detection of other leukemia in the future.

Spatial-Spectral Joint Network for Cholangiocarcinoma Microscopic Hyperspectral Image Classification

Accurate histopathology classification is a crucial factor in the diagnosis and treatment of Cholangiocarcinoma(CCA).Hyperspectral images(HSI)provide rich spectral information than ordinary RGB images,making them more useful for medical diagnosis.The Convolutional Neural Network(CNN)is commonly employed in hyperspectral image classification due to its remarkable capacity for feature extraction and image classification.However,many existing CNN-based HSI classification methods tend to ignore the importance of image spatial context information and the interdependence between spectral channels,leading to unsatisfied classification performance.Thus,to address these issues,this paper proposes a Spatial-Spectral Joint Network(SSJN)model for hyperspectral image classification that utilizes spatial self-attention and spectral feature extraction.The SSJN model is derived from the ResNet18 network and implemented with the non-local and Coordinate Attention(CA)modules,which extract long-range dependencies on image space and enhance spatial features through the Branch Attention(BA)module to emphasize the region of interest.Furthermore,the SSJN model employs Conv-LSTM modules to extract long-range depen-dencies in the image spectral domain.This addresses the gradient disappearance/explosion phenom-ena and enhances the model classification accuracy.The experimental results show that the pro-posed SSJN model is more efficient in leveraging the spatial and spectral information of hyperspec-tral images on multidimensional microspectral datasets of CCA,leading to higher classification accuracy,and may have useful references for medical diagnosis of CCA.

Recent advances in nonlinear optics for bio‐imaging applications

Nonlinear optics,which is a subject for studying the interaction between intense light and materials,has great impact on various research fields.Since many structures in biological tissues exhibit strong nonlinear optical effects,nonlinear optics has been widely applied in biomedical studies.Especially in the aspect of bio-imaging,nonlinear optical techniques can provide rapid,label-free and chemically specific imaging of biological samples,which enable the investigation of biological processes and analysis of samples beyond other microscopy techniques.In this review,we focus on the introduction of nonlinear optical processes and their applications in bio-imaging as well as the recent advances in this filed.Our perspective of this field is also presented.

Deep learning of rock microscopic images for intelligent lithology identification: Neural network comparison and selection

An intelligent lithology identification method is proposed based on deep learning of the rock microscopic images.Based on the characteristics of rock images in the dataset,we used Xception,MobileNet_v2,Inception_ResNet_v2,Inception_v3,Densenet121,ResNet101_v2,and ResNet-101 to develop microscopic image classification models,and then the network structures of seven different convolutional neural networks(CNNs)were compared.It shows that the multi-layer representation of rock features can be represented through convolution structures,thus better feature robustness can be achieved.For the loss function,cross-entropy is used to back propagate the weight parameters layer by layer,and the accuracy of the network is improved by frequent iterative training.We expanded a self-built dataset by using transfer learning and data augmentation.Next,accuracy(acc)and frames per second(fps)were used as the evaluation indexes to assess the accuracy and speed of model identification.The results show that the Xception-based model has the optimum performance,with an accuracy of 97.66%in the training dataset and 98.65%in the testing dataset.Furthermore,the fps of the model is 50.76,and the model is feasible to deploy under different hardware conditions and meets the requirements of rapid lithology identification.This proposed method is proved to be robust and versatile in generalization performance,and it is suitable for both geologists and engineers to identify lithology quickly.

Automatic Leukaemia Segmentation Approach for Blood Cancer Classification Using Microscopic Images

Leukaemia is a type of blood cancer that is caused by undeveloped White Blood Cells(WBC),and it is also called a blast blood cell.In the marrow of human bones,leukaemia is developed and is responsible for blood cell generation with leukocytes and WBC,and if any cell gets blasted,then it may become a cause of death.Therefore,the diagnosis of leukaemia in its early stages helps greatly in the treatment along with saving human lives.Subsequently,in terms of detection,image segmentation techniques play a vital role,and they turn out to be the important image processing steps for the extraction of feature patterns from the Acute Lymphoblastic Leukaemia(ALL)type of blood cancer.Moreover,the image segmentation technique focuses on the division of cells by segmenting a microscopic image into background and cancer blood cell nucleus,which is well-known as the Region Of Interest(ROI).As a result,in this article,we attempt to build a segmentation technique capable of solving blood cell nucleus segmentation issues using four distinct scenarios,including K-means,FCM(Fuzzy Cmeans),K-means with FFA(Firefly Algorithm),and FCM with FFA.Also,we determine the most effective method of blood cell nucleus segmentation,which we subsequently use for the Leukaemia classification model.Finally,using the Convolution Neural Network(CNN)as a classifier,we developed a leukaemia cancer classification model from the microscopic images.The proposed system’s classification accuracy is tested using the CNN to test the model on the ALL-IDB dataset and equate it to the current state of the art.In terms of experimental analysis,we observed that the accuracy of the model is near to 99%,and it is far better than other existing models that are designed to segment and classify the types of leukaemia cancer in terms of ALL.

Optical micro-scanning location calibration of thermal microscope imaging system

A method of micro-scanning location adaptive calibration was proposed, which was real- ized by the digital image micro-displacement estimation. With geometric calculation, this calibration method used the displacement estimation of two thermal microscope images to get the size and direc- tion of each scanning location calibration angle. And each location calibration process was repeated according to the offset given by the system beforehand. The comparison experiments of sequence oversampling reconstruction before and after the micro-scanning location calibration were done. The results showed that the calibration method effectively improved the thermal microscope imaging qual- ity.

Oversample Reconstruction Based on a Strong Inter-Diagonal Matrix for an Optical Microscanning Thermal Microscope Imaging System

Based on a strong inter-diagonal matrix and Taylor series expansions,an oversample reconstruction method was proposed to calibrate the optical micro-scanning error. The technique can obtain regular 2 ×2 microscanning undersampling images from the real irregular undersampling images,and can then obtain a high spatial oversample resolution image. Simulations and experiments show that the proposed technique can reduce optical micro-scanning error and improve the system＇s spatial resolution. The algorithm is simple,fast and has low computational complexity. It can also be applied to other electro-optical imaging systems to improve their spatial resolution and has a widespread application prospect.

Highly specific characterization and discrimination of monosodium urate crystals in gouty arthritis based on aggregation-induced emission luminogens

Existing technologies used to detect monosodium urate(MSU)crystals for gout diagnosis are not ideal due to their low sensitivity and complexity of operation.The purpose of this study was to explore whether aggregation-induced emission luminogens(AIEgens)can be used for highly specific imaging of MSU crystals to assist in the diagnosis of gout.First,we developed a series of luminogens(i.e.,tetraphenyl ethylene(TPE)-NH_(2),TPE-2NH_(2),TPE-4NH_(2),TPE-COOH,TPE-2COOH,TPE-4COOH,and TPE-Ketoalkyne),each of which was then evenly mixed with MSU crystals.Next,optimal fluorescence imaging of each of the luminogens was characterized by a confocal laser scanning microscope(CLSM).This approach was used for imaging standard samples of MSU,hydroxyapatite(HAP)crystals,and mixed samples with 1:1 mass ratio of MSU/HAP.We also imaged samples from mouse models of acute gouty arthritis,HAP deposition disease,and comorbidities of interest.Subsequently,CLSM imaging results were compared with those of compensated polarized light microscopy,and we assessed the biosafety of TPE-Ketoalkyne in the RAW264.7 cell line.Finally,CLSM time series and three-dimensional imaging were performed on MSU crystal samples from human gouty synovial fluid and tophi.As a promising candidate for MSU crystal labeling,TPE-Ketoalkyne was found to detect MSU crystals accurately and rapidly in standard samples,animal samples,and human samples,and could precisely distinguish gout from HAP deposition disease.This work demonstrates that TPE-Ketoalkyne is suitable for highly specific and timely imaging of MSU crystals in gouty arthritis and may facilitate future research on MSU crystal-related diseases.

Horizontal Voting Ensemble Based Predictive Modeling System for Colon Cancer

Colon cancer is the third most commonly diagnosed cancer in the world.Most colon AdenoCArcinoma(ACA)arises from pre-existing benign polyps in the mucosa of the bowel.Thus,detecting benign at the earliest helps reduce the mortality rate.In this work,a Predictive Modeling System(PMS)is developed for the classification of colon cancer using the Horizontal Voting Ensemble(HVE)method.Identifying different patterns inmicroscopic images is essential to an effective classification system.A twelve-layer deep learning architecture has been developed to extract these patterns.The developedHVE algorithm can increase the system’s performance according to the combined models from the last epochs of the proposed architecture.Ten thousand(10000)microscopic images are taken to test the classification performance of the proposed PMS with the HVE method.The microscopic images obtained from the colon tissues are classified intoACAor benign by the proposed PMS.Results prove that the proposed PMS has∼8%performance improvement over the architecture without using the HVE method.The proposed PMS for colon cancer reduces the misclassification rate and attains 99.2%of sensitivity and 99.4%of specificity.The overall accuracy of the proposed PMS is 99.3%,and without using the HVE method,it is only 91.3%.

A Leukocyte image fast scanning based on max–min distance clustering

A leukocyte image fast scanning method based on max min distance clustering is proposed.Because of the lower proportion and uneven distribution of leukocytes in human peripheral blood,there will not be any leukocyte in lager quantity of the captured images if we directly scan the blood smear along an ordinary zigzag scanning routine with high power(100^(x))objective.Due to the larger field of view of low power(10^(x))objective,the captured low power blood smear images can be used to locate leukocytes.All of the located positions make up a specific routine,if we scan the blood smear along this routine with high power objective,there will be definitely leukocytes in almost all of the captured images.Considering the number of captured images is still large and some leukocytes may be redundantly captured twice or more,a leukocyte clustering method based on max-min distance clustering is developed to reduce the total number of captured images as well as the number of redundantly captured leukocytes.This method can improve the scanning eficiency obviously.The experimental results show that the proposed method can shorten scanning time from 8.0-14.0min to 2.54.0 min while extracting 110 nonredundant individual high power leukocyte images.

An online identity authentication method for blood smear

Blood smear test is the basic method of blood cytology and is also a standard medical test that can help diagnose various conditions and diseases.Morphological examination is the gold stan-dard to determine pathological changes in blood cell morphology.In the biology and medicine automation trend,blood smears'automated management and analysis is very necessary.An online blood smear automatic microscopic image detection system has been constructed.It includes an online blood smear automatic producing part and a blood smear automatic micro-scopic image detection part.Online identity authentication is at the core of the system.The identifiers printed online always present dot matrix digit code(DMDC)whose stroke is not continuous.Considering the particularities of DMDC and the complexities of online application environment,an online identity authentication method for blood smear with heterological theory is proposed.By synthesizing the certain regional features according to the heterological theory,high identification accuracy and high speed have been guaranteed with few features required.In the experiment,the suficient correct matches bet ween the tube barcode and the identification result verified its feasibility and validity.

First-principles Study on Geometric and Electronic Structures of Si（111）-√7× √3-In Surface Reconstruction

In order to determine the structures of Si（111）-√7 √3-In surfaces and to understand their electronic properties, we construct six models of both hexagonal and rectangular types and perform first-principles calculations. Their scanning tunneling microscopic images and work functions are simulated and compared with experimental results. In this way, the hex-H3＇ and rect-T1 models are identified as the experimental configurations for the hexagonal and rectangular types, respectively. The structural evolution mechanism of the In/Si（lll） surface with indium coverage around 1.0 monolayer is discussed. The 4×1 and -√7× √3 phases are suggested to have two different types of evolution mechanisms, consistent with experimental results.

Microscopic hyperspectral imaging studies of normal and diabetic retina of rats

A microscopic hyperspectral imager was developed based on the microscopic technology and the spectral imaging technology. Some microscopic hyperspectral images of retina sections of the normal, the diabetic, and the treated rats were collected by the new imager. Single-band images and pseudo-color images of each group were obtained and the typical transmittance spectrums were ex-tracted. The results showed that the transmittance of outer nuclear layer cells of the diabetic group was generally higher than that of the normal. A small absorption peak appeared near the 180th band in the spectrum of the diabetic group and this peak weakened or disappeared in the spectrum of the treated group. Our findings indicate that the microscopic hyperspectral images include wealthy information of retina sections which is helpful for the ophthalmologist to reveal the pathogenesis of diabetic reti-nopathy and explore the therapeutic effect of drugs.

Optimizing signal collection efficiency of the VIPA-based Brillouin spectrometer

Brillouin spectroscopy is an emerging tool for microscopic optical imaging as it allows for non-invasive and direct assessment of the viscoelastic properties of materials.Recent advances of background-free confocal Brillouin spectrometer allows investigators to acquire the Brillouin spectra for turbid samples as well as transparent ones.However,due to strong signal loss induced by the imperfect optical setup,the Brillouin photons are usually immersed in background noise.In this report,we proposed and experimentally demonstrated multiple approaches to enhance the signal collction eficiency.A signal enhancement by>4 times can be observed,enabling ob-servation of ultra-weak signals.

Comments on Quasicrystals

The discoveries of so-called quasicrystals have broken through the theoretic foundation set up by the classical crystallographic group theory since 1891 and proposed new topics for study of solid structures. Electron diffraction patterns (EDP' s) and high-resolution microscopic (HREM) images have proved invaluable tools of studying the structures of crystals. The recognition and determination of EDP's and HREM images of a real-structure play a key role for understanding the structure. This paper will introduce some new developments about crystallographic group theory and new image processing methods on EDP's and HREM images. Contrary to popular beliefs, the research shows that quasicrystals can be understood (perturbed) complex periodic structures.

Drug Response Prediction of Liver Cancer Cell Line Using Deep Learning

Cancer is the second deadliest human disease worldwide with high mortality rate.Rehabilitation and treatment of this disease requires precise and automatic assessment of effective drug response and control system.Prediction of treated and untreated cancerous cell line is one of the most challenging problems for precise and targeted drug delivery and response.A novel approach is proposed for prediction of drug treated and untreated cancer cell line automatically by employing modified Deep neural networks.Human hepatocellular carcinoma(HepG2)cells are exposed to anticancer drug functionalized CFO@BTO nanoparticles developed by our lab.Prediction models are developed by modifying ResNet101 and exploiting the transfer learning concept.Last three layers of ResNet101 are re-trained for the identification of drug treated cancer cells.Transfer learning approach in an appropriate choice especially when there is limited amount of annotated data.The proposed technique is validated on acquired 203 fluorescentmicroscopy images of human HepG2 cells treated with drug functionalized cobalt ferrite@barium titanate(CFO@BTO)magnetoelectric nanoparticles in vitro.The developed approach achieved high prediction with accuracy of 97.5%and sensitivity of 100%and outperformed other approaches.The high performance reveals the effectiveness of the approach.It is scalable and fully automatic prediction approach which can be extended for other similar cell diseases such as lung,brain tumor and breast cancer.

Microscopic elasticity imaging of vessel walls based on intravascular ultrasound

The effect of the transducer eccentricity on grayscales of intravascualr ultrasound images was corrected based on the scattering properties of high frequency ultrasound in vessel walls. The displacement and strain distributions of vessel walls produced by tissue microelement motion were obtained using a novel motion estimation method in steps and sum based on the optical flow and genetic algorithm. Furthermore, authors firstly reconstructed 'real' elasticity distribution images of cross section tissues of vessel walls in the world. In vitro experimental results of porcine artery demonstrated the methods mentioned above are reasonable. Experimental investigation of vascular mechanics can be advanced to 2D sub-millimeter microstructure levels. These studies have potential to provide new technology means in monitoring and evaluation of Percutaneous transluminal coronary angioplasty process.

Automated Dynamic Cellular Analysis in Time-Lapse Microscopy

Analysis of cellular behavior is significant for studying cell cycle and detecting anti-cancer drugs. It is a very difficult task for image processing to isolate individual cells in confocal microscopic images of non-stained live cell cultures. Because these images do not have adequate textural variations. Manual cell segmentation requires massive labor and is a time consuming process. This paper describes an automated cell segmentation method for localizing the cells of Chinese hamster ovary cell culture. Several kinds of high-dimensional feature descriptors, K-means clustering method and Chan-Vese model-based level set are used to extract the cellular regions. The region extracted are used to classify phases in cell cycle. The segmentation results were experimentally assessed. As a result, the proposed method proved to be significant for cell isolation. In the evaluation experiments, we constructed a database of Chinese Hamster Ovary Cell’s microscopic images which includes various photographing environments under the guidance of a biologist.