Deep learning of rock microscopic images for intelligent lithology identification: Neural network comparison and selection

An intelligent lithology identification method is proposed based on deep learning of the rock microscopic images.Based on the characteristics of rock images in the dataset,we used Xception,MobileNet_v2,Inception_ResNet_v2,Inception_v3,Densenet121,ResNet101_v2,and ResNet-101 to develop microscopic image classification models,and then the network structures of seven different convolutional neural networks(CNNs)were compared.It shows that the multi-layer representation of rock features can be represented through convolution structures,thus better feature robustness can be achieved.For the loss function,cross-entropy is used to back propagate the weight parameters layer by layer,and the accuracy of the network is improved by frequent iterative training.We expanded a self-built dataset by using transfer learning and data augmentation.Next,accuracy(acc)and frames per second(fps)were used as the evaluation indexes to assess the accuracy and speed of model identification.The results show that the Xception-based model has the optimum performance,with an accuracy of 97.66%in the training dataset and 98.65%in the testing dataset.Furthermore,the fps of the model is 50.76,and the model is feasible to deploy under different hardware conditions and meets the requirements of rapid lithology identification.This proposed method is proved to be robust and versatile in generalization performance,and it is suitable for both geologists and engineers to identify lithology quickly.

Automatic Leukaemia Segmentation Approach for Blood Cancer Classification Using Microscopic Images

Leukaemia is a type of blood cancer that is caused by undeveloped White Blood Cells(WBC),and it is also called a blast blood cell.In the marrow of human bones,leukaemia is developed and is responsible for blood cell generation with leukocytes and WBC,and if any cell gets blasted,then it may become a cause of death.Therefore,the diagnosis of leukaemia in its early stages helps greatly in the treatment along with saving human lives.Subsequently,in terms of detection,image segmentation techniques play a vital role,and they turn out to be the important image processing steps for the extraction of feature patterns from the Acute Lymphoblastic Leukaemia(ALL)type of blood cancer.Moreover,the image segmentation technique focuses on the division of cells by segmenting a microscopic image into background and cancer blood cell nucleus,which is well-known as the Region Of Interest(ROI).As a result,in this article,we attempt to build a segmentation technique capable of solving blood cell nucleus segmentation issues using four distinct scenarios,including K-means,FCM(Fuzzy Cmeans),K-means with FFA(Firefly Algorithm),and FCM with FFA.Also,we determine the most effective method of blood cell nucleus segmentation,which we subsequently use for the Leukaemia classification model.Finally,using the Convolution Neural Network(CNN)as a classifier,we developed a leukaemia cancer classification model from the microscopic images.The proposed system’s classification accuracy is tested using the CNN to test the model on the ALL-IDB dataset and equate it to the current state of the art.In terms of experimental analysis,we observed that the accuracy of the model is near to 99%,and it is far better than other existing models that are designed to segment and classify the types of leukaemia cancer in terms of ALL.

Efcient autofocus method for sequential automatic capturing of high-magnification microscopic images

This letter presents an autofocus （AF） method to position a high-magnification microscope lens that automatically captures hundreds of images from a single moving slide. These images are taken by a mobile clinic unit in a rural location, and are later automatically processed and revised by a remote specialist. This process requires high focus precision to enable image processing techniques to achieve proper results. Low focusing times are also required for the system to be operative. We propose a novel method that combines two focus measures with an adapted searching scheme to cope with both constraints.

Spatial-Spectral Joint Network for Cholangiocarcinoma Microscopic Hyperspectral Image Classification

Accurate histopathology classification is a crucial factor in the diagnosis and treatment of Cholangiocarcinoma(CCA).Hyperspectral images(HSI)provide rich spectral information than ordinary RGB images,making them more useful for medical diagnosis.The Convolutional Neural Network(CNN)is commonly employed in hyperspectral image classification due to its remarkable capacity for feature extraction and image classification.However,many existing CNN-based HSI classification methods tend to ignore the importance of image spatial context information and the interdependence between spectral channels,leading to unsatisfied classification performance.Thus,to address these issues,this paper proposes a Spatial-Spectral Joint Network(SSJN)model for hyperspectral image classification that utilizes spatial self-attention and spectral feature extraction.The SSJN model is derived from the ResNet18 network and implemented with the non-local and Coordinate Attention(CA)modules,which extract long-range dependencies on image space and enhance spatial features through the Branch Attention(BA)module to emphasize the region of interest.Furthermore,the SSJN model employs Conv-LSTM modules to extract long-range depen-dencies in the image spectral domain.This addresses the gradient disappearance/explosion phenom-ena and enhances the model classification accuracy.The experimental results show that the pro-posed SSJN model is more efficient in leveraging the spatial and spectral information of hyperspec-tral images on multidimensional microspectral datasets of CCA,leading to higher classification accuracy,and may have useful references for medical diagnosis of CCA.

Spatial-spectral identication of abnormal leukocytes based on microscopic hyperspectral imaging technology

Screening and diagnosing of abnormal Leukocytes are crucial for the diagnosis of immune diseases and Acute Lymphoblastic Leukemia(ALL).As the deterioration of abnormal leukocytes is mainly due to the changes in the chromatin distribution,which signicantly affects the absorption and reflection of light,the spectral feature is proved to be important for leukocytes classication and identication.This paper proposes an accurate identication method for healthy and abnormal leukocytes based on microscopic hyperspectral imaging(HSI)technology which combines the spectral information.The segmentation of nucleus and cytoplasm is obtained by the morphological watershed algorithm.Then,the spectral features are extracted and combined with the spatial features.Based on this,the support vector machine(SVM)is applied for classication ofve types of leukocytes and abnormal leukocytes.Compared with different classication methods,the proposed method utilizes spectral features which highlight the differences between healthy leukocytes and abnormal leukocytes,improving the accuracy in the classication and identication of leukocytes.This paper only selects one subtype of ALL for test,and the proposed method can be applied for detection of other leukemia in the future.

Horizontal Voting Ensemble Based Predictive Modeling System for Colon Cancer

Colon cancer is the third most commonly diagnosed cancer in the world.Most colon AdenoCArcinoma(ACA)arises from pre-existing benign polyps in the mucosa of the bowel.Thus,detecting benign at the earliest helps reduce the mortality rate.In this work,a Predictive Modeling System(PMS)is developed for the classification of colon cancer using the Horizontal Voting Ensemble(HVE)method.Identifying different patterns inmicroscopic images is essential to an effective classification system.A twelve-layer deep learning architecture has been developed to extract these patterns.The developedHVE algorithm can increase the system’s performance according to the combined models from the last epochs of the proposed architecture.Ten thousand(10000)microscopic images are taken to test the classification performance of the proposed PMS with the HVE method.The microscopic images obtained from the colon tissues are classified intoACAor benign by the proposed PMS.Results prove that the proposed PMS has∼8%performance improvement over the architecture without using the HVE method.The proposed PMS for colon cancer reduces the misclassification rate and attains 99.2%of sensitivity and 99.4%of specificity.The overall accuracy of the proposed PMS is 99.3%,and without using the HVE method,it is only 91.3%.

Highly specific characterization and discrimination of monosodium urate crystals in gouty arthritis based on aggregation-induced emission luminogens

Existing technologies used to detect monosodium urate(MSU)crystals for gout diagnosis are not ideal due to their low sensitivity and complexity of operation.The purpose of this study was to explore whether aggregation-induced emission luminogens(AIEgens)can be used for highly specific imaging of MSU crystals to assist in the diagnosis of gout.First,we developed a series of luminogens(i.e.,tetraphenyl ethylene(TPE)-NH_(2),TPE-2NH_(2),TPE-4NH_(2),TPE-COOH,TPE-2COOH,TPE-4COOH,and TPE-Ketoalkyne),each of which was then evenly mixed with MSU crystals.Next,optimal fluorescence imaging of each of the luminogens was characterized by a confocal laser scanning microscope(CLSM).This approach was used for imaging standard samples of MSU,hydroxyapatite(HAP)crystals,and mixed samples with 1:1 mass ratio of MSU/HAP.We also imaged samples from mouse models of acute gouty arthritis,HAP deposition disease,and comorbidities of interest.Subsequently,CLSM imaging results were compared with those of compensated polarized light microscopy,and we assessed the biosafety of TPE-Ketoalkyne in the RAW264.7 cell line.Finally,CLSM time series and three-dimensional imaging were performed on MSU crystal samples from human gouty synovial fluid and tophi.As a promising candidate for MSU crystal labeling,TPE-Ketoalkyne was found to detect MSU crystals accurately and rapidly in standard samples,animal samples,and human samples,and could precisely distinguish gout from HAP deposition disease.This work demonstrates that TPE-Ketoalkyne is suitable for highly specific and timely imaging of MSU crystals in gouty arthritis and may facilitate future research on MSU crystal-related diseases.

Assessing pathological features of breast cancer via the multimodal information of multiphoton and Raman imaging

For unveiling the pathological evolution of breast cancer, nonlinear multiphoton microscopic(MPM) and confocal Raman microspectral imaging(CRMI) techniques were both utilized to address the structural and constitutional characteristics of healthy(H), ductal carcinoma in situ(DCIS), and invasive ductal carcinoma(IDC) tissues. MPM-based techniques,including two-photon excited fluorescence(TPEF) and second harmonic generation(SHG), visualized label-free and the fine structure of breast tissue. Meanwhile, CRMI not only presented the chemical images of investigated samples with the K-mean cluster analysis method(KCA), but also pictured the distribution of components in the scanned area through univariate imaging. MPM images illustrated that the cancer cells first arranged around the basement membrane of the duct,then proliferated to fill the lumens of the duct, and finally broke through the basement membrane to infiltrate into the stroma.Although the Raman imaging failed to visualize the cell structure with high resolution, it explained spectroscopically the gradual increase of nucleic acid and protein components inside the ducts as cancer cells proliferated, and displayed the distribution pattern of each biological component during the evolution of breast cancer. Thus, the combination of MPM and CRMI provided new insights into the on-site pathological diagnosis of malignant breast cancer, also ensured technical support for the development of multimodal optical imaging techniques for precise histopathological analysis.

Identification of Chinese Materia Medicas in Microscopic Powder Images

This paper describes an identification system for Chinese Materia Medicas （CMMs） in microscopic powder images. The imaging processing of the microscopic powder image is very complex because of the low contrast, blurry boundaries, overlapping objects, and messy background. Therefore, the object detection must segment the significant microscopic structures from the complex image. The objects are detected in these images using an adaptable interactive method. After identifying the significant microscopic structures, the system identifies 14 features belonging to three main characteristics. These features form a 14-dimensional vector that represents the microscopic structures. The multi-dimensional vector is then analyzed using a feature assignment algorithm that picks the most notable features to construct a decision tree with thresholds. The identification system consists of a coarse classifier based on the decision tree and a fine classifier using similarity measurements to rank the possible results. Tests on 528 images from 24 different kinds of microscopic structures show the system effectiveness and applicability.

Drug Response Prediction of Liver Cancer Cell Line Using Deep Learning

Cancer is the second deadliest human disease worldwide with high mortality rate.Rehabilitation and treatment of this disease requires precise and automatic assessment of effective drug response and control system.Prediction of treated and untreated cancerous cell line is one of the most challenging problems for precise and targeted drug delivery and response.A novel approach is proposed for prediction of drug treated and untreated cancer cell line automatically by employing modified Deep neural networks.Human hepatocellular carcinoma(HepG2)cells are exposed to anticancer drug functionalized CFO@BTO nanoparticles developed by our lab.Prediction models are developed by modifying ResNet101 and exploiting the transfer learning concept.Last three layers of ResNet101 are re-trained for the identification of drug treated cancer cells.Transfer learning approach in an appropriate choice especially when there is limited amount of annotated data.The proposed technique is validated on acquired 203 fluorescentmicroscopy images of human HepG2 cells treated with drug functionalized cobalt ferrite@barium titanate(CFO@BTO)magnetoelectric nanoparticles in vitro.The developed approach achieved high prediction with accuracy of 97.5%and sensitivity of 100%and outperformed other approaches.The high performance reveals the effectiveness of the approach.It is scalable and fully automatic prediction approach which can be extended for other similar cell diseases such as lung,brain tumor and breast cancer.

Recent advances in nonlinear optics for bio‐imaging applications

Nonlinear optics,which is a subject for studying the interaction between intense light and materials,has great impact on various research fields.Since many structures in biological tissues exhibit strong nonlinear optical effects,nonlinear optics has been widely applied in biomedical studies.Especially in the aspect of bio-imaging,nonlinear optical techniques can provide rapid,label-free and chemically specific imaging of biological samples,which enable the investigation of biological processes and analysis of samples beyond other microscopy techniques.In this review,we focus on the introduction of nonlinear optical processes and their applications in bio-imaging as well as the recent advances in this filed.Our perspective of this field is also presented.

Optimizing signal collection efficiency of the VIPA-based Brillouin spectrometer

Brillouin spectroscopy is an emerging tool for microscopic optical imaging as it allows for non-invasive and direct assessment of the viscoelastic properties of materials.Recent advances of background-free confocal Brillouin spectrometer allows investigators to acquire the Brillouin spectra for turbid samples as well as transparent ones.However,due to strong signal loss induced by the imperfect optical setup,the Brillouin photons are usually immersed in background noise.In this report,we proposed and experimentally demonstrated multiple approaches to enhance the signal collction eficiency.A signal enhancement by>4 times can be observed,enabling ob-servation of ultra-weak signals.

A Leukocyte image fast scanning based on max–min distance clustering

A leukocyte image fast scanning method based on max min distance clustering is proposed.Because of the lower proportion and uneven distribution of leukocytes in human peripheral blood,there will not be any leukocyte in lager quantity of the captured images if we directly scan the blood smear along an ordinary zigzag scanning routine with high power(100^(x))objective.Due to the larger field of view of low power(10^(x))objective,the captured low power blood smear images can be used to locate leukocytes.All of the located positions make up a specific routine,if we scan the blood smear along this routine with high power objective,there will be definitely leukocytes in almost all of the captured images.Considering the number of captured images is still large and some leukocytes may be redundantly captured twice or more,a leukocyte clustering method based on max-min distance clustering is developed to reduce the total number of captured images as well as the number of redundantly captured leukocytes.This method can improve the scanning eficiency obviously.The experimental results show that the proposed method can shorten scanning time from 8.0-14.0min to 2.54.0 min while extracting 110 nonredundant individual high power leukocyte images.

Comments on Quasicrystals

The discoveries of so-called quasicrystals have broken through the theoretic foundation set up by the classical crystallographic group theory since 1891 and proposed new topics for study of solid structures. Electron diffraction patterns (EDP' s) and high-resolution microscopic (HREM) images have proved invaluable tools of studying the structures of crystals. The recognition and determination of EDP's and HREM images of a real-structure play a key role for understanding the structure. This paper will introduce some new developments about crystallographic group theory and new image processing methods on EDP's and HREM images. Contrary to popular beliefs, the research shows that quasicrystals can be understood (perturbed) complex periodic structures.

An online identity authentication method for blood smear

Blood smear test is the basic method of blood cytology and is also a standard medical test that can help diagnose various conditions and diseases.Morphological examination is the gold stan-dard to determine pathological changes in blood cell morphology.In the biology and medicine automation trend,blood smears'automated management and analysis is very necessary.An online blood smear automatic microscopic image detection system has been constructed.It includes an online blood smear automatic producing part and a blood smear automatic micro-scopic image detection part.Online identity authentication is at the core of the system.The identifiers printed online always present dot matrix digit code(DMDC)whose stroke is not continuous.Considering the particularities of DMDC and the complexities of online application environment,an online identity authentication method for blood smear with heterological theory is proposed.By synthesizing the certain regional features according to the heterological theory,high identification accuracy and high speed have been guaranteed with few features required.In the experiment,the suficient correct matches bet ween the tube barcode and the identification result verified its feasibility and validity.

Optical micro-scanning location calibration of thermal microscope imaging system

A method of micro-scanning location adaptive calibration was proposed, which was real- ized by the digital image micro-displacement estimation. With geometric calculation, this calibration method used the displacement estimation of two thermal microscope images to get the size and direc- tion of each scanning location calibration angle. And each location calibration process was repeated according to the offset given by the system beforehand. The comparison experiments of sequence oversampling reconstruction before and after the micro-scanning location calibration were done. The results showed that the calibration method effectively improved the thermal microscope imaging qual- ity.

Automated Dynamic Cellular Analysis in Time-Lapse Microscopy

Analysis of cellular behavior is significant for studying cell cycle and detecting anti-cancer drugs. It is a very difficult task for image processing to isolate individual cells in confocal microscopic images of non-stained live cell cultures. Because these images do not have adequate textural variations. Manual cell segmentation requires massive labor and is a time consuming process. This paper describes an automated cell segmentation method for localizing the cells of Chinese hamster ovary cell culture. Several kinds of high-dimensional feature descriptors, K-means clustering method and Chan-Vese model-based level set are used to extract the cellular regions. The region extracted are used to classify phases in cell cycle. The segmentation results were experimentally assessed. As a result, the proposed method proved to be significant for cell isolation. In the evaluation experiments, we constructed a database of Chinese Hamster Ovary Cell’s microscopic images which includes various photographing environments under the guidance of a biologist.

Structural and Piezoelectric Properties of Sr_(0.6)Ba_(0.4)Nb_2O_6 Micro-rods Synthesized by Molten-Salt Method

Sr0.6 Ba0.4 Nb2 O6 micro-rods are prepared by the molten-salt method with K2 SO4,KCl-K2 SO4,and KCl as fluxes.It reveals that the Sr0.6 Ba0.4 Nb2 O6 synthesized with KCl as a flux exhibits a single phase with tetragonal tungsten bronze structure.The measurement of X-ray diffraction indicates that the Sr0.6 Ba0.4 Nb2 O6 micro-rods synthesized at 1 300℃are anisotropic.The morphology of the powers is examined by transmission electron microscope.It reveals that the length-diameter ratio of Sr0.6 Ba0.4 Nb2 O6 micro-rods increases with increasing annealing temperature from 900℃to 1 300℃.At 1 300℃,the rod possesses a large length-diameter ratio of 8∶1.Moreover,the analysis of the piezoelectric properties of single micro-rods using apiezo-response force microscope indicates that the domains of the material are arranged along its radial direction.

Oversample Reconstruction Based on a Strong Inter-Diagonal Matrix for an Optical Microscanning Thermal Microscope Imaging System

Based on a strong inter-diagonal matrix and Taylor series expansions,an oversample reconstruction method was proposed to calibrate the optical micro-scanning error. The technique can obtain regular 2 ×2 microscanning undersampling images from the real irregular undersampling images,and can then obtain a high spatial oversample resolution image. Simulations and experiments show that the proposed technique can reduce optical micro-scanning error and improve the system＇s spatial resolution. The algorithm is simple,fast and has low computational complexity. It can also be applied to other electro-optical imaging systems to improve their spatial resolution and has a widespread application prospect.

A Voronoi diagram approach for detecting defects in 3D printed fiber-reinforced polymers from microscope images

Fiber-reinforced polymer(FRP)composites are increasingly popular due to their superior strength to weight ratio.In contrast to significant recent advances in automating the FRP manufacturing process via 3D printing,quality inspection and defect detection remain largely manual and inefficient.In this paper,we propose a new approach to automatically detect,from microscope images,one of the major defects in 3D printed FRP parts:fiber-deficient areas(or equivalently,resin-rich areas).From cross-sectional microscope images,we detect the locations and sizes of fibers,construct their Voronoi diagram,and employ-shape theory to determine fiber-deficient areas.Our Voronoi diagram and-shape construction algorithms are specialized to exploit typical characteristics of 3D printed FRP parts,giving significant efficiency gains.Our algorithms robustly handle real-world inputs containing hundreds of thousands of fiber cross-sections,whether in general or non-general position.