The development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation,which is gradually changing the landscape of op...The development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation,which is gradually changing the landscape of optical imaging and biomedical research.However,current implementations of deep learning usually operate in a supervised manner,and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability.Here,we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy,even in some cases in which supervised models cannot be applied.Through the introduction of a saliency constraint,the unsupervised model,named Unsupervised content-preserving Transformation for Optical Microscopy(UTOM);can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content.UTOM shows promising performance in a wide range of biomedical image transformation tasks,including in silico histological staining,fluorescence image restoration,and virtual fluorescence labeling.Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities.We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.展开更多
基金We would like to acknowledge Weigert et al.for making their source code and data related to image restoration openly available to the comm unity.We thank the Rubin Lab at Harvard,the Finkbeiner Lab at Gladstone,and Google Accelerated Science for releasing their datasets on virtual cell staining.We thank Jingjing Wang,affiliated with the apparatus sharing platform of Tsinghua University,for assistance with the imaging of histopathology slides.This work was supported by the National Natural Science Foundation of China(62088102,61831014,62071271,and 62071272)Projects of MOST(2020AA0105500 and 2020AAA0130000)+1 种基金Shenzhen Science and Technology Projects(ZDYBH201900000002 and JCYJ20180508152042002)the National Postdoctoral Program for Innovative Talents(BX20190173).
文摘The development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation,which is gradually changing the landscape of optical imaging and biomedical research.However,current implementations of deep learning usually operate in a supervised manner,and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability.Here,we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy,even in some cases in which supervised models cannot be applied.Through the introduction of a saliency constraint,the unsupervised model,named Unsupervised content-preserving Transformation for Optical Microscopy(UTOM);can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content.UTOM shows promising performance in a wide range of biomedical image transformation tasks,including in silico histological staining,fluorescence image restoration,and virtual fluorescence labeling.Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities.We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.
