In-flow holographic tomography boosts lipid droplet quantification

In their recently published paper in Opto-Electronic Ad-vances,Pietro Ferraro and his colleagues report on a new high-throughput tomographic phase instrument that precisely quantifies intracellular lipid droplets(LDs)1.LDs are lipid storage organelles found in most cell types and play an active role in critical biological pro-cesses,including energy metabolism,membrane homeo-stasis.

Emerging Advances to Transform Histopathology Using Virtual Staining

In an age where digitization is widespread in clinical and preclinical workflows,pathology is still predominantly practiced by microscopic evaluation of stained tissue specimens affixed on glass slides.Over the last decade,new high throughput digital scanning microscopes have ushered in the era of digital pathology that,along with recent advances in machine vision,have opened up new possibilities for Computer-Aided-Diagnoses.Despite these advances,the high infrastructural costs related to digital pathology and the perception that the digitization process is an additional and nondirectly reimbursable step have challenged its widespread adoption.Here,we discuss how emerging virtual staining technologies and machine learning can help to disrupt the standard histopathology workflow and create new avenues for the diagnostic paradigm that will benefit patients and healthcare systems alike via digital pathology.

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

The immunohistochemical(IHC)staining of the human epidermal growth factor receptor 2(HER2)biomarker is widely practiced in breast tissue analysis,preclinical studies,and diagnostic decisions,guiding cancer treatment and investigation of pathogenesis.HER2 staining demands laborious tissue treatment and chemical processing performed by a histotechnologist,which typically takes one day to prepare in a laboratory,increasing analysis time and associated costs.Here,we describe a deep learning-based virtual HER2 IHC staining method using a conditional generative adversarial network that is trained to rapidly transform autofluorescence microscopic images of unlabeled/label-free breast tissue sections into bright-field equivalent microscopic images,matching the standard HER2 IHC staining that is chemically performed on the same tissue sections.The efficacy of this virtual HER2 staining framework was demonstrated by quantitative analysis,in which three board-certified breast pathologists blindly graded the HER2 scores of virtually stained and immunohistochemically stained HER2 whole slide images(WSIs)to reveal that the HER2 scores determined by inspecting virtual IHC images are as accurate as their immunohistochemically stained counterparts.A second quantitative blinded study performed by the same diagnosticians further revealed that the virtually stained HER2 images exhibit a comparable staining quality in the level of nuclear detail,membrane clearness,and absence of staining artifacts with respect to their immunohistochemically stained counterparts.This virtual HER2 staining framework bypasses the costly,laborious,and time-consuming IHC staining procedures in laboratory and can be extended to other types of biomarkers to accelerate the IHC tissue staining used in life sciences and biomedical workflow.

Multifunctional nanotherapeutics for treatment of ocular disease

The eye is an important organ,it provides vision and it is an important component of our facial identity(1,2).There is an old saying“The eye is the window of the soul”,clear and bright eyes bring esthetic pleasure to people.Human eye is globular and consists of two main parts,the anterior and posterior segments(1,2).Although the posterior part of the eye is comfortably located in the orbit,they are delicate because its anterior segment including the cornea is exposed to the outside world,thus accessible to wear and tear.To protect and maintain the eye functions while reduce the disruptions from outside and inside the body,the eyes are equipped with defense mechanisms for both the anterior and posterior segments(1,2).

OAM-based diffractive all-optical classification

Object classification is an important aspect of machine intelligence.Current practices in object classification entail the digitization of object information followed by the application of digital algorithms such as deep neural networks.The execution of digital neural networks is power-consuming,and the throughput is limited.The existing von Neumann digital computing paradigm is also less suited for the implementation of highly parallel neural network architectures.^(1)

All-optical image denoising using a diffractive visual processor

Image denoising,one of the essential inverse problems,targets to remove noise/artifacts from input images.In general,digital image denoising algorithms,executed on computers,present latency due to several iterations implemented in,e.g.,graphics processing units(GPUs).While deep learning-enabled methods can operate non-iteratively,they also introduce latency and impose a significant computational burden,leading to increased power consumption.Here,we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images–implemented at the speed of light propagation within a thin diffractive visual processor that axially spans<250×λ,whereλis the wavelength of light.This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features,causing them to miss the output image Field-of-View(FoV)while retaining the object features of interest.Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of~30–40%.We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum.Owing to their speed,power-efficiency,and minimal computational overhead,all-optical diffractive denoisers can be transformative for various image display and projection systems,including,e.g.,holographic displays.

Universal linear intensity transformations using spatially incoherent diffractive processors

Under spatially coherent light,a diffractive optical network composed of structured surfaces can be designed to perform any arbitrary complex-valued linear transformation between its input and output fields-of-view(FOVs)if the total number(N)of optimizable phase-only diffractive features is≥~2N_(i)N_(o),where Ni and No refer to the number of useful pixels at the input and the output FOVs,respectively.Here we report the design of a spatially incoherent diffractive optical processor that can approximate any arbitrary linear transformation in time-averaged intensity between its input and output FOVs.Under spatially incoherent monochromatic light,the spatially varying intensity point spread function(H)of a diffractive network,corresponding to a given,arbitrarily-selected linear intensity transformation,can be written as H(m,n;m′,n′)=|h(m,n;m′,n′)|^(2),where h is the spatially coherent point spread function of the same diffractive network,and(m,n)and(m′,n′)define the coordinates of the output and input FOVs,respectively.Using numerical simulations and deep learning,supervised through examples of input-output profiles,we demonstrate that a spatially incoherent diffractive network can be trained to all-optically perform any arbitrary linear intensity transformation between its input and output if N≥~2N_(i)N_(o).We also report the design of spatially incoherent diffractive networks for linear processing of intensity information at multiple illumination wavelengths,operating simultaneously.Finally,we numerically demonstrate a diffractive network design that performs all-optical classification of handwritten digits under spatially incoherent illumination,achieving a test accuracy of>95%.Spatially incoherent diffractive networks will be broadly useful for designing all-optical visual processors that can work under natural light.

Deep learning-enabled virtual histological staining of biological samples

Histological staining is the gold standard for tissue examination in clinical pathology and life-science research,which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue.However,the current histological staining workflow requires tedious sample preparation steps,specialized laboratory infrastructure,and trained histotechnologists,making it expensive,time-consuming,and not accessible in resource-limited settings.Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks,providing rapid,cost-effective,and accurate alternatives to standard chemical staining methods.These techniques,broadly referred to as virtual staining,were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples;similar approaches were also used for transforming images of an already stained tissue sample into another type of stain,performing virtual stain-to-stain transformations.In this Review,we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques.The basic concepts and the typical workflow of virtual staining are introduced,followed by a discussion of representative works and their technical innovations.We also share our perspectives on the future of this emerging field,aiming to inspire readers from diverse scientifc fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.

High-throughput terahertz imaging:progress and challenges

Many exciting terahertz imaging applications,such as non-destructive evaluation,biomedical diagnosis,and security screening,have been historically limited in practical usage due to the raster-scanning requirement of imaging systems,which impose very low imaging speeds.However,recent advancements in terahertz imaging systems have greatly increased the imaging throughput and brought the promising potential of terahertz radiation from research laboratories closer to real-world applications.Here,we review the development of terahertz imaging technologies from both hardware and computational imaging perspectives.We introduce and compare different types of hardware enabling frequency-domain and time-domain imaging using various thermal,photon,and field image sensor arrays.We discuss how different imaging hardware and computational imaging algorithms provide opportunities for capturing time-of-flight,spectroscopic,phase,and intensity image data at high throughputs.Furthermore,the new prospects and challenges for the development of future high-throughput terahertz imaging systems are briefly introduced.

Phase recovery and holographic image reconstruction using deep learning in neural networks

Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography.In this study,we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training.This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts.This neural network-based method is fast to compute and reconstructs phase and amplitude images of the objects using only one hologram,requiring fewer measurements in addition to being computationally faster.We validated this method by reconstructing the phase and amplitude images of various samples,including blood and Pap smears and tissue sections.These results highlight that challenging problems in imaging science can be overcome through machine learning,providing new avenues to design powerful computational imaging systems.

PhaseStain:the digital staining of label-free quantitative phase microscopy images using deep learning

Using a deep neural network,we demonstrate a digital staining technique,which we term PhaseStain,to transform the quantitative phase images(QPI)of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained.Through pairs of image data(QPI and the corresponding brightfield images,acquired after staining),we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin,kidney,and liver tissue,matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin,Jones’stain,and Masson’s trichrome stain,respectively.This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general,by eliminating the need for histological staining,reducing sample preparation related costs and saving time.Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

A deep learning-enabled portable imaging flow cytometer for cost-effective, highthroughput, and label-free analysis of natural water samples

We report a deep learning-enabled field-portable and cost-effective imaging flow cytometer that automatically captures phase-contrast color images of the contents of a continuously flowing water sample at a throughput of 100 mL/h.The device is based on partially coherent lens-free holographic microscopy and acquires the diffraction patterns of flowing micro-objects inside a microfluidic channel.These holographic diffraction patterns are reconstructed in real time using a deep learning-based phase-recovery and image-reconstruction method to produce a color image of each micro-object without the use of external labeling.Motion blur is eliminated by simultaneously illuminating the sample with red,green,and blue light-emitting diodes that are pulsed.Operated by a laptop computer,this portable device measures 15.5 cm×15 cm×12.5 cm,weighs 1 kg,and compared to standard imaging flow cytometers,it provides extreme reductions of cost,size and weight while also providing a high volumetric throughput over a large object size range.We demonstrated the capabilities of this device by measuring ocean samples at the Los Angeles coastline and obtaining images of its micro-and nanoplankton composition.Furthermore,we measured the concentration of a potentially toxic alga(Pseudo-nitzschia)in six public beaches in Los Angeles and achieved good agreement with measurements conducted by the California Department of Public Health.The cost-effectiveness,compactness,and simplicity of this computational platform might lead to the creation of a network of imaging flow cytometers for largescale and continuous monitoring of the ocean microbiome,including its plankton composition.

Glucose-responsive oral insulin delivery for postprandial glycemic regulation

Controlling postprandial glucose levels for diabetic patients is critical to achieve the tight glycemic control that decreases the risk for developing long-term micro- and macrovascular complications.Herein,we report a glucose-responsive oral insulin delivery system based on Fc receptor (FcRn)-targeted liposomes with glucose-sensitive hyaluronic acid (HA) shell for postprandial glycemic regulation.After oral administration,the HA shell can quickly detach in the presence of increasing intestinal glucose concentration due to the competitive binding of glucose with the phenylboronic acid groups conjugated with HA.The exposed Fc groups on the surface of liposomes then facilitate enhanced intestinal absorption in an FcRn-mediated transport pathway.In vivo studies on chemically-induced type 1 diabetic mice show this oral glucose-responsive delivery approach can effectively reduce postprandial blood glucose excursions.This work is the first demonstration of an oral insulin delivery system directly triggered by increasing postprandial glucose concentrations in the intestine to provide an on-demand insulin release with ease of administration.

Ensemble learning of diffractive optical networks

A plethora of research advances have emerged in the fields of optics and photonics that benefit from harnessing the power of machine learning.Specifically,there has been a revival of interest in optical computing hardware due to its potential advantages for machine learning tasks in terms of parallelization,power efficiency and computation speed.Diffractive deep neural networks(D^(2)NNs)form such an optical computing framework that benefits from deep learning-based design of successive diffractive layers to all-optically process information as the input light diffracts through these passive layers.D^(2)NNs have demonstrated success in various tasks,including object classification,the spectral encoding of information,optical pulse shaping and imaging.Here,we substantially improve the inference performance of diffractive optical networks using feature engineering and ensemble learning.After independently training 1252 D^(2)NNs that were diversely engineered with a variety of passive input filters,we applied a pruning algorithm to select an optimized ensemble of D^(2)NNs that collectively improved the image classification accuracy.Through this pruning,we numerically demonstrated that ensembles of N=14 and N=30 D^(2)NNs achieve blind testing accuracies of 61.14±0.23%and 62.13±0.05%,respectively,on the classification of GFAR-10 test images,providing an inference improvennent of>16%compared to the average performance of the individual D^(2)NNs within each ensemble.These results constitute the highest inference accuracies achieved to date by any diffractive optical neural network design on the same dataset and might provide a significant leap to extend the application space of diffractive optical image classification and machine vision systems.

Synthetic aperture-based on-chip microscopy

Wide field-of-view(FOV)and high-resolution imaging requires microscopy modalities to have large space-bandwidth products.Lensfree on-chip microscopy decouples resolution from FOV and can achieve a space-bandwidth product greater than one billion under unit magnification using state-of-the-art opto-electronic sensor chips and pixel super-resolution techniques.However,using vertical illumination,the effective numerical aperture(NA)that can be achieved with an on-chip microscope is limited by a poor signal-to-noise ratio(SNR)at high spatial frequencies and imaging artifacts that arise as a result of the relatively narrow acceptance angles of the sensor’s pixels.Here,we report,for the first time,a synthetic aperture-based on-chip microscope in which the illumination angle is scanned across the surface of a dome to increase the effective NA of the reconstructed lensfree image to 1.4,achieving e.g.,,250-nm resolution at 700-nm wavelength under unit magnification.This synthetic aperture approach not only represents the largest NA achieved to date using an on-chip microscope but also enables color imaging of connected tissue samples,such as pathology slides,by achieving robust phase recovery without the need for multi-height scanning or any prior information about the sample.To validate the effectiveness of this synthetic aperture-based,partially coherent,holographic on-chip microscope,we have successfully imaged color-stained cancer tissue slides as well as unstained Papanicolaou smears across a very large FOV of 20.5 mm^(2).This compact on-chip microscope based on a synthetic aperture approach could be useful for various applications in medicine,physical sciences and engineering that demand high-resolution wide-field imaging.

Polarization multiplexed diffractive computing:all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

Research on optical computing has recently attracted significant attention due to the transformative advances in machine learning.Among different approaches,diffractive optical networks composed of spatially-engineered transmissive surfaces have been demonstrated for all-optical statistical inference and performing arbitrary linear transformations using passive,free-space optical layers.Here,we introduce a polarization-multiplexed diffractive processor to all-optically perform multiple,arbitrarily-selected linear transformations through a single diffractive network trained using deep learning.In this framework,an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic,and different target linear transformations(complex-valued)are uniquely assigned to different combinations of input/output polarization states.The transmission layers of this polarization-multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to diffferent input/output polarization combinations.Our results and analysis reveal that a single diffractive network can successfully approximate and all-optically implement a group of arbitrarily-selected target transformations with a negligible error when the number of trainable diffractive features/neurons(N)approaches N_(p)N_(i)N_(o),where Ni and N_(o) represent the number of pixels at the input and output fields-of-view,respectively,and N_(p) refers to the number of unique linear transformations assigned to different input/output polarization combinations.This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.

Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Histological staining is a vital step in diagnosing various diseases and has been used for more than a century to provide contrast in tissue sections,rendering the tissue constituents visible for microscopic analysis by medical experts.However,this process is time consuming,labour intensive,expensive and destructive to the specimen.Recently,the ability to virtually stain unlabelled tissue sections,entirely avoiding the histochemical staining step,has been demonstrated using tissue-stain-specific deep neural networks.Here,we present a new deep-learning-based framework that generates virtually stained images using label-free tissue images,in which different stains are merged following a micro-structure map defined by the user.This approach uses a single deep neural network that receives two different sources of information as its input:(1)autofluorescence images of the label-free tissue sample and(2)a“digital staining matrix”,which represents the desired microscopic map of the different stains to be virtually generated in the same tissue section.This digital staining matrix is also used to virtually blend existing stains,digitally synthesizing new histological stains.We trained and blindly tested this virtual-staining network using unlabelled kidney tissue sections to generate micro-structured combinations of haematoxylin and eosin(H&E),Jones’silver stain,and Masson’s trichrome stain.Using a single network,this approach multiplexes the virtual staining of label-free tissue images with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created in the same tissue cross section,which is currently not feasible with standard histochemical staining methods.

Air quality monitoring using mobile microscopy and machine learning

Rapid,accurate and high-throughput sizing and quantification of particulate matter(PM)in air is crucial for monitoring and improving air quality.In fact,particles in air with a diameter of≤2.5μm have been classified as carcinogenic by the World Health Organization.Here we present a field-portable cost-effective platform for high-throughput quantification of particulate matter using computational lens-free microscopy and machine-learning.This platform,termed c-Air,is also integrated with a smartphone application for device control and display of results.This mobile device rapidly screens 6.5 L of air in 30 s and generates microscopic images of the aerosols in air.It provides statistics of the particle size and density distribution with a sizing accuracy of~93%.We tested this mobile platform by measuring the air quality at different indoor and outdoor environments and measurement times,and compared our results to those of an Environmental Protection Agency–approved device based on beta-attenuation monitoring,which showed strong correlation to c-Air measurements.Furthermore,we used c-Air to map the air quality around Los Angeles International Airport(LAX)over 24 h to confirm that the impact of LAX on increased PM concentration was present even at 47 km away from the airport,especially along the direction of landing flights.With its machinelearning-based computational microscopy interface,c-Air can be adaptively tailored to detect specific particles in air,for example,various types of pollen and mold and provide a cost-effective mobile solution for highly accurate and distributed sensing of air quality.

Additively manufactured metallic biomaterials

Metal additive manufacturing(AM)has led to an evolution in the design and fabrication of hard tissue substitutes,enabling personalized implants to address each patient’s specific needs.In addition,internal pore architectures integrated within additively manufactured scaffolds,have provided an opportunity to further develop and engineer functional implants for better tissue integration,and long-term durability.In this review,the latest advances in different aspects of the design and manufacturing of additively manufactured metallic biomaterials are highlighted.After introducing metal AM processes,biocompatible metals adapted for integration with AM machines are presented.Then,we elaborate on the tools and approaches undertaken for the design of porous scaffold with engineered internal architecture including,topology optimization techniques,as well as unit cell patterns based on lattice networks,and triply periodic minimal surface.Here,the new possibilities brought by the functionally gradient porous structures to meet the conflicting scaffold design requirements are thoroughly discussed.Subsequently,the design constraints and physical characteristics of the additively manufactured constructs are reviewed in terms of input parameters such as design features and AM processing parameters.We assess the proposed applications of additively manufactured implants for regeneration of different tissue types and the efforts made towards their clinical translation.Finally,we conclude the review with the emerging directions and perspectives for further development of AM in the medical industry.