Deep neural networks for understanding noisy data applied to physical property extraction in scanning probe microscopy

The rapid development of spectral-imaging methods in scanning probe,electron,and optical microscopy in the last decade have given rise for large multidimensional datasets.In many cases,the reduction of hyperspectral data to the lower-dimension materialsspecific parameters is based on functional fitting,where an approximate form of the fitting function is known,but the parameters of the function need to be determined.However,functional fits of noisy data realized via iterative methods,such as least-square gradient descent,often yield spurious results and are very sensitive to initial guesses.Here,we demonstrate an approach for the reduction of the hyperspectral data using a deep neural network approach.A combined deep neural network/least-square approach is shown to improve the effective signal-to-noise ratio of band-excitation piezoresponse force microscopy by more than an order of magnitude,allowing characterization when very small driving signals are used or when a material’s response is weak.

Toward nanoscale molecular mass spectrometry imaging via physically constrained machine learning on co-registered multimodal data

Mass spectrometry imaging(MSI)plays a pivotal role in investigating the chemical nature of complex systems that underly our understanding in biology and medicine.Multiple fields of life science such as proteomics,lipidomics and metabolomics benefit from the ability to simultaneously identify molecules and pinpoint their distribution across a sample.However,achieving the necessary submicron spatial resolution to distinguish chemical differences between individual cells and generating intact molecular spectra is still a challenge with any single imaging approach.Here,we developed an approach that combines two MSI techniques,matrix-assisted laser desorption/ionization(MALDI)and time-of-flight secondary ion mass spectrometry(ToF-SIMS),one with low spatial resolution but intact molecular spectra and the other with nanometer spatial resolution but fragmented molecular signatures,to predict molecular MSI spectra with submicron spatial resolution.

Application of pan-sharpening algorithm for correlative multimodal imaging using AFM-IR

The coupling of atomic force microscopy with infrared spectroscopy(AFM-IR)offers the unique capability to characterize the local chemical and physical makeup of a broad variety of materials with nanoscale resolution.However,in order to fully utilize the measurement capability of AFM-IR,a three-dimensional dataset(2D map with a spectroscopic dimension)needs to be acquired,which is prohibitively time-consuming at the same spatial resolution of a regular AFM scan.In this paper,we provide a new approach to process spectral AFM-IR data based on a multicomponent pan-sharpening algorithm.This approach requires only a low spatial resolution spectral and a limited number of high spatial resolution single wavenumber chemical maps to generate a high spatial resolution hyperspectral image,greatly reducing data acquisition time.As a result,we are able to generate highresolution maps of component distribution,produce chemical maps at any wavenumber available in the spectral range,and perform correlative analysis of the physical and chemical properties of the samples.We highlight our approach via imaging of plant cell walls as a model system and showcase the interplay between mechanical stiffness of the sample and its chemical composition.We believe our pan-sharpening approach can be more generally applied to different material classes to enable deeper understanding of that structure-property relationship at the nanoscale.

Deep data analytics for genetic engineering of diatoms linking genotype to phenotype via machine learning

Genome engineering for materials synthesis is a promising avenue for manufacturing materials with unique properties under ambient conditions.Biomineralization in diatoms,unicellular algae that use silica to construct micron-scale cell walls with nanoscale features,is an attractive candidate for functional synthesis of materials for applications including photonics,sensing,filtration,and drug delivery.Therefore,controllably modifying diatom structure through targeted genetic modifications for these applications is a very promising field.In this work,we used gene knockdown in Thalassiosira pseudonana diatoms to create modified strains with changes to structural morphology and linked genotype to phenotype using supervised machine learning.An artificial neural network(NN)was developed to distinguish wild and modified diatoms based on the SEM images of frustules exhibiting phenotypic changes caused by a specific protein(Thaps3_21880),resulting in 94% detection accuracy.Class activation maps visualized physical changes that allowed the NNs to separate diatom strains,subsequently establishing a specific gene that controls pores.A further NN was created to batch process image data,automatically recognize pores,and extract pore-related parameters.Class interrelationship of the extracted paraments was visualized using a multivariate data visualization tool,called CrossVis,and allowed to directly link changes in morphological diatom phenotype of pore size and distribution with changes in the genotype.