Machine learning for automated experimentation in scanning transmission electron microscopy

Machine learning(ML)has become critical for post-acquisition data analysis in(scanning)transmission electron microscopy,(S)TEM,imaging and spectroscopy.An emerging trend is the transition to real-time analysis and closed-loop microscope operation.The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centric experiment workflow design and optimization.Here,we discuss the associated challenges with the transition to active ML,including sequential data analysis and out-of-distribution drift effects,the requirements for edge operation,local and cloud data storage,and theory in the loop operations.Specifically,we discuss the relative contributions of human scientists and ML agents in the ideation,orchestration,and execution of experimental workflows,as well as the need to develop universal hyper languages that can apply across multiple platforms.These considerations will collectively inform the operationalization of ML in next-generation experimentation.

Rapid and flexible segmentation of electron microscopy data using few-shot machine learning

Automatic segmentation of key microstructural features in atomic-scale electron microscope images is critical to improved understanding of structure–property relationships in many important materials and chemical systems.However,the present paradigm involves time-intensive manual analysis that is inherently biased,error-prone,and unable to accommodate the large volumes of data produced by modern instrumentation.While more automated approaches have been proposed,many are not robust to a high variety of data,and do not generalize well to diverse microstructural features and material systems.Here,we present a flexible,semi-supervised few-shot machine learning approach for segmentation of scanning transmission electron microscopy images of three oxide material systems:(1)epitaxial heterostructures of SrTiO_(3)/Ge,(2)La_(0.8)Sr_(0.2)FeO_(3) thin films,and(3)MoO_(3) nanoparticles.We demonstrate that the few-shot learning method is more robust against noise,more reconfigurable,and requires less data than conventional image analysis methods.This approach can enable rapid image classification and microstructural feature mapping needed for emerging high-throughput characterization and autonomous microscope platforms.

Forecasting of in situ electron energy loss spectroscopy

Forecasting models are a central part of many control systems,where high-consequence decisions must be made on long latency control variables.These models are particularly relevant for emerging artificial intelligence(AI)-guided instrumentation,in which prescriptive knowledge is needed to guide autonomous decision-making.Here we describe the implementation of a long short-term memory model(LSTM)for forecasting in situ electron energy loss spectroscopy(EELS)data,one of the richest analytical probes of materials and chemical systems.We describe key considerations for data collection,preprocessing,training,validation,and benchmarking,showing how this approach can yield powerful predictive insight into order-disorder phase transitions.Finally,we comment on how such a model may integrate with emerging AI-guided instrumentation for powerful high-speed experimentation.