Multispectral imaging (MSI) technique is often used to capture imagesof the fundus by illuminating it with different wavelengths of light. However,these images are taken at different points in time such that eyeball m...Multispectral imaging (MSI) technique is often used to capture imagesof the fundus by illuminating it with different wavelengths of light. However,these images are taken at different points in time such that eyeball movementscan cause misalignment between consecutive images. The multispectral imagesequence reveals important information in the form of retinal and choroidal bloodvessel maps, which can help ophthalmologists to analyze the morphology of theseblood vessels in detail. This in turn can lead to a high diagnostic accuracy of several diseases. In this paper, we propose a novel semi-supervised end-to-end deeplearning framework called “Adversarial Segmentation and Registration Nets”(ASRNet) for the simultaneous estimation of the blood vessel segmentation andthe registration of multispectral images via an adversarial learning process. ASRNet consists of two subnetworks: (i) A segmentation module S that fulfills theblood vessel segmentation task, and (ii) A registration module R that estimatesthe spatial correspondence of an image pair. Based on the segmention-drivenregistration network, we train the segmentation network using a semi-supervisedadversarial learning strategy. Our experimental results show that the proposedASRNet can achieve state-of-the-art accuracy in segmentation and registrationtasks performed with real MSI datasets.展开更多
Deformable image registration(DIR)has been well explored in recent decades,and it is widely utilized in clinical tasks,especially dose warping.Nowadays,as deep learning(DL)develops rapidly,many DL-based methods were a...Deformable image registration(DIR)has been well explored in recent decades,and it is widely utilized in clinical tasks,especially dose warping.Nowadays,as deep learning(DL)develops rapidly,many DL-based methods were also applied in DIR.This paper reviews DL-based DIR methods in recent years and the application of DIR in dose warping.We collected and categorized the latest DL-based DIR studies.A thorough review of each category was presented,in which studies were discussed based on their supervision,advantage,and challenges.Then,we reviewed DIR-based dose warping and discussed its rationale,feasibility,successes,and difficulties.Lastly,we summarized the review on both parts and discussed their future development trend.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.81871508 and 61773246)the Major Program of Shandong Province Natural Science Foundation(Grant No.ZR2019ZD04 and ZR2018ZB0419)the Taishan Scholar Program of Shandong Province of China(Grant No.TSHW201502038).
文摘Multispectral imaging (MSI) technique is often used to capture imagesof the fundus by illuminating it with different wavelengths of light. However,these images are taken at different points in time such that eyeball movementscan cause misalignment between consecutive images. The multispectral imagesequence reveals important information in the form of retinal and choroidal bloodvessel maps, which can help ophthalmologists to analyze the morphology of theseblood vessels in detail. This in turn can lead to a high diagnostic accuracy of several diseases. In this paper, we propose a novel semi-supervised end-to-end deeplearning framework called “Adversarial Segmentation and Registration Nets”(ASRNet) for the simultaneous estimation of the blood vessel segmentation andthe registration of multispectral images via an adversarial learning process. ASRNet consists of two subnetworks: (i) A segmentation module S that fulfills theblood vessel segmentation task, and (ii) A registration module R that estimatesthe spatial correspondence of an image pair. Based on the segmention-drivenregistration network, we train the segmentation network using a semi-supervisedadversarial learning strategy. Our experimental results show that the proposedASRNet can achieve state-of-the-art accuracy in segmentation and registrationtasks performed with real MSI datasets.
基金This research was partly supported by Hong Kong research grants(General Research Fund(GRF)from University Grants Committee:GRF 151021/18M and GRF 151022/19MHealth and Medical Research Fund(HMRF)from Food and Health Bureau:HMRF 06173276 and HMRF 07183266).
文摘Deformable image registration(DIR)has been well explored in recent decades,and it is widely utilized in clinical tasks,especially dose warping.Nowadays,as deep learning(DL)develops rapidly,many DL-based methods were also applied in DIR.This paper reviews DL-based DIR methods in recent years and the application of DIR in dose warping.We collected and categorized the latest DL-based DIR studies.A thorough review of each category was presented,in which studies were discussed based on their supervision,advantage,and challenges.Then,we reviewed DIR-based dose warping and discussed its rationale,feasibility,successes,and difficulties.Lastly,we summarized the review on both parts and discussed their future development trend.
