PhaseStain:the digital staining of label-free quantitative phase microscopy images using deep learning

Using a deep neural network,we demonstrate a digital staining technique,which we term PhaseStain,to transform the quantitative phase images(QPI)of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained.Through pairs of image data(QPI and the corresponding brightfield images,acquired after staining),we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin,kidney,and liver tissue,matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin,Jones’stain,and Masson’s trichrome stain,respectively.This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general,by eliminating the need for histological staining,reducing sample preparation related costs and saving time.Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

Fourier Imager Network(FIN):A deep neural network for hologram reconstruction with superior external generalization

Deep learning-based image reconstruction methods have achieved remarkable success in phase recovery and holographic imaging.However,the generalization of their image reconstruction performance to new types of samples never seen by the network remains a challenge.Here we introduce a deep learning framework,termed Fourier Imager Network(FIN),that can perform end-to-end phase recovery and image reconstruction from raw holograms of new types of samples,exhibiting unprecedented success in external generalization.FIN architecture is based on spatial Fourier transform modules that process the spatial frequencies of its inputs using learnable filters and a global receptive field.Compared with existing convolutional deep neural networks used for hologram reconstruction,FIN exhibits superior generalization to new types of samples,while also being much faster in its image inference speed,completing the hologram reconstruction task in~0.04 s per 1 mm^(2) of the sample area.We experimentally validated the performance of FIN by training it using human lung tissue samples and blindly testing it on human prostate,salivary gland tissue and Pap smear samples,proving its superior external generalization and image reconstruction speed.Beyond holographic microscopy and quantitative phase imaging,FIN and the underlying neural network architecture might open up various new opportunities to design broadly generalizable deep learning models in computational imaging and machine vision fields.

Motility-based label-free detection of parasites in bodily fluids using holographic speckle analysis and deep learning

Parasitic infections constitute a major global public health issue.Existing screening methods that are based on manual microscopic examination often struggle to provide sufficient volumetric throughput and sensitivity to facilitate early diagnosis.Here,we demonstrate a motility-based label-free computational imaging platform to rapidly detect motile parasites in optically dense bodily fluids by utilizing the locomotion of the parasites as a specific biomarker and endogenous contrast mechanism.Based on this principle,a cost-effective and mobile instrument,which rapidly screens~3.2 mL of fluid sample in three dimensions,was built to automatically detect and count motile microorganisms using their holographic time-lapse speckle patterns.We demonstrate the capabilities of our platform by detecting trypanosomes,which are motile protozoan parasites,with various species that cause deadly diseases affecting millions of people worldwide.Using a holographic speckle analysis algorithm combined with deep learningbased classification,we demonstrate sensitive and label-free detection of trypanosomes within spiked whole blood and artificial cerebrospinal fluid(CSF)samples,achieving a limit of detection of ten trypanosomes per mL of whole blood(~five-fold better than the current state-of-the-art parasitological method)and three trypanosomes per mL of CSF.We further demonstrate that this platform can be applied to detect other motile parasites by imaging Trichomonas vaginalis,the causative agent of trichomoniasis,which affects 275 million people worldwide.With its costeffective,portable design and rapid screening time,this unique platform has the potential to be applied for sensitive and timely diagnosis of neglected tropical diseases caused by motile parasites and other parasitic infections in resource-limited regions.

Biopsy-free in vivo virtual histology of skin using deep learning

An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors.The process is cumbersome and time-consuming,often leading to unnecessary biopsies and scars.Emerging noninvasive optical technologies such as reflectance confocal microscopy(RCM)can provide label-free,cellular-level resolution,in vivo images of skin without performing a biopsy.Although RCM is a useful diagnostic tool,it requires specialized training because the acquired images are grayscale,lack nuclear features,and are difficult to correlate with tissue pathology.Here,we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution,enabling visualization of the epidermis,dermal-epidermal junction,and superficial dermis layers.The network was trained under an adversarial learning scheme,which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth.We show that this trained neural network can be used to rapidly perform virtual histology of in vivo,label-free RCM images of normal skin structure,basal cell carcinoma,and melanocytic nevi with pigmented melanocytes,demonstrating similar histological features to traditional histology from the same excised tissue.This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.

Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data

Optical coherence tomography(OCT)is a widely used non-invasive biomedical imaging modality that can rapidly provide volumetric images of samples.Here,we present a deep learning-based image reconstruction framework that can generate swept-source OCT(SS-OCT)images using undersampled spectral data,without any spatial aliasing artifacts.This neural network-based image reconstruction does not require any hardware changes to the optical setup and can be easily integrated with existing swept-source or spectral-domain OCT systems to reduce the amount of raw spectral data to be acquired.To show the efficacy of this framework,we trained and blindly tested a deep neural network using mouse embryo samples imaged by an SS-OCT system.Using 2-fold undersampled spectral data(i.e.,640 spectral points per A-line),the trained neural network can blindly reconstruct 512 A-lines in 0.59 ms using multiple graphics-processing units(GPUs),removing spatial aliasing artifacts due to spectral undersampling,also presenting a very good match to the images of the same samples,reconstructed using the full spectral OCT data(i.e.,1280 spectral points per A-line).We also successfully demonstrate that this framework can be further extended to process 3×undersampled spectral data per A-line,with some performance degradation in the reconstructed image quality compared to 2×spectral undersampling.Furthermore,an A-line-optimized undersampling method is presented by jointly optimizing the spectral sampling locations and the corresponding image reconstruction network,which improved the overall imaging performance using less spectral data points per A-line compared to 2×or 3×spectral undersampling results.This deep learning-enabled image reconstruction approach can be broadly used in various forms of spectral-domain OCT systems,helping to increase their imaging speed without sacrificing image resolution and signal-to-noise ratio.

Computational cytometer based on magnetically modulated coherent imaging and deep learning

Detecting rare cells within blood has numerous applications in disease diagnostics.Existing rare cell detection techniques are typically hindered by their high cost and low throughput.Here,we present a computational cytometer based on magnetically modulated lensless speckle imaging,which introduces oscillatory motion to the magneticbead-conjugated rare cells of interest through a periodic magnetic force and uses lensless time-resolved holographic speckle imaging to rapidly detect the target cells in three dimensions(3D).In addition to using cell-specific antibodies to magnetically label target cells,detection specificity is further enhanced through a deep-learning-based classifier that is based on a densely connected pseudo-3D convolutional neural network(P3D CNN),which automatically detects rare cells of interest based on their spatio-temporal features under a controlled magnetic force.To demonstrate the performance of this technique,we built a high-throughput,compact and cost-effective prototype for detecting MCF7 cancer cells spiked in whole blood samples.Through serial dilution experiments,we quantified the limit of detection(LoD)as 10 cells per millilitre of whole blood,which could be further improved through multiplexing parallel imaging channels within the same instrument.This compact,cost-effective and high-throughput computational cytometer can potentially be used for rare cell detection and quantification in bodily fluids for a variety of biomedical applications.