Application of machine learning in ophthalmic imaging modalities

In clinical ophthalmology,a variety of image-related diagnostic techniques have begun to offer unprecedented insights into eye diseases based on morphological datasets with millions of data points.Artificial intelligence(AI),inspired by the human multilayered neuronal system,has shown astonishing success within some visual and auditory recognition tasks.In these tasks,AI can analyze digital data in a comprehensive,rapid and non-invasive manner.Bioinformatics has become a focus particularly in the field of medical imaging,where it is driven by enhanced computing power and cloud storage,as well as utilization of novel algorithms and generation of data in massive quantities.Machine learning(ML)is an important branch in the field of AI.The overall potential of ML to automatically pinpoint,identify and grade pathological features in ocular diseases will empower ophthalmologists to provide high-quality diagnosis and facilitate personalized health care in the near future.This review offers perspectives on the origin,development,and applications of ML technology,particularly regarding its applications in ophthalmic imaging modalities.

Different fundus imaging modalities and technical factors in AI screening for diabetic retinopathy:a review

Background:Effective screening is a desirable method for the early detection and successful treatment for diabetic retinopathy,and fundus photography is currently the dominant medium for retinal imaging due to its convenience and accessibility.Manual screening using fundus photographs has however involved considerable costs for patients,clinicians and national health systems,which has limited its application particularly in less-developed countries.The advent of artificial intelligence,and in particular deep learning techniques,has however raised the possibility of widespread automated screening.Main text:In this review,we first briefly survey major published advances in retinal analysis using artificial intelligence.We take care to separately describe standard multiple-field fundus photography,and the newer modalities of ultrawide field photography and smartphone-based photography.Finally,we consider several machine learning concepts that have been particularly relevant to the domain and illustrate their usage with extant works.Conclusions:In the ophthalmology field,it was demonstrated that deep learning tools for diabetic retinopathy show clinically acceptable diagnostic performance when using colour retinal fundus images.Artificial intelligence models are among the most promising solutions to tackle the burden of diabetic retinopathy management in a comprehensive manner.However,future research is crucial to assess the potential clinical deployment,evaluate the cost-effectiveness of different DL systems in clinical practice and improve clinical acceptance.

Application of machine learning in ophthalmic imaging modalities

In clinical ophthalmology,a variety of image-related diagnostic techniques have begun to offer unprecedented insights into eye diseases based on morphological datasets with millions of data points.Artificial intelligence(Al)z inspired by the human multilayered neuronal system,has shown astonishing success within some visual and auditory recognition tasks.In these tasks,Al can an a lyze digital data in a comprehensive,rapid and non-inv asive manner.Bioinformatics has become a focus particularly in the field of medical imaging,where it is driven by enhanced computing power and cloud storage,as well as utilization of novel algorithms and generation of data in massive quantities.Machine learning(ML)is an important branch in the field of Al.The overall potential of ML to automatically pinpoint,identify and grade pathological features in ocular diseases will empower ophthalmologists to provide high-quality diagnosis and facilitate personalized health care in the near future.This review offers perspectives on the origin,development,and applications of ML technology,particularly regarding its applications in ophthalmic imaging modalities.

Correlation of different imaging modalities in pre-surgical evaluation of pericardial metastasis of liposarcoma

A patient presented with a large pericardial tumor of uncertain etiology. Five years earlier, she had been treated for myxoid liposarcoma of the thigh. For pre-surgical evaluation, conventional radiography, positron emission tomography/computed tomography （PET/CT）, magnetic resonance imaging （MRI）, CT of the heart, transthoracic echocardiography （TTE） and transesophageal echocardiography （TEE） were performed. The final histopathologic diagnosis was metastatic liposarcoma. Each of the imaging modalities used had advantages and disadvantages, and their coordination was necessary for optimal evaluation.

The integration of artificial intelligence models to augment imaging modalities in pancreatic cancer

Pancreatic ductal adenocarcinoma(PDAC)is an aggressive malignancy with a limited number of effective treatments.Using emerging technologies such as artificial intelligence(AI)to facilitate the earlier diagnosis and decision-making process represents one of the most promising areas for investigation.The integration of AI models to augment imaging modalities in PDAC has made great progression in the past 5 years,especially in organ segmentation,AI-aided diagnosis,and radiomics based individualized medicine.In this article,we review the developments of AI in the field of PDAC and the present clinical position.We also examine the barriers to future development and more widespread application which will require increased familiarity of the underlying technology among clinicians to promote the necessary enthusiasm and collaboration with computer professionals.

Quest for the best endoscopic imaging modality for computer-assisted colonic polyp staging

BACKGROUND It was shown in previous studies that high definition endoscopy, high magnification endoscopy and image enhancement technologies, such as chromoendoscopy and digital chromoendoscopy [narrow-band imaging(NBI), iScan] facilitate the detection and classification of colonic polyps during endoscopic sessions. However, there are no comprehensive studies so far that analyze which endoscopic imaging modalities facilitate the automated classification of colonic polyps. In this work, we investigate the impact of endoscopic imaging modalities on the results of computer-assisted diagnosis systems for colonic polyp staging.AIM To assess which endoscopic imaging modalities are best suited for the computerassisted staging of colonic polyps.METHODS In our experiments, we apply twelve state-of-the-art feature extraction methods for the classification of colonic polyps to five endoscopic image databases of colonic lesions. For this purpose, we employ a specifically designed experimental setup to avoid biases in the outcomes caused by differing numbers of images per image database. The image databases were obtained using different imaging modalities. Two databases were obtained by high-definition endoscopy in combination with i-Scan technology(one with chromoendoscopy and one without chromoendoscopy). Three databases were obtained by highmagnification endoscopy(two databases using narrow band imaging and one using chromoendoscopy). The lesions are categorized into non-neoplastic and neoplastic according to the histological diagnosis.RESULTS Generally, it is feature-dependent which imaging modalities achieve high results and which do not. For the high-definition image databases, we achieved overall classification rates of up to 79.2% with chromoendoscopy and 88.9% without chromoendoscopy. In the case of the database obtained by high-magnification chromoendoscopy, the classification rates were up to 81.4%. For the combination of high-magnification endoscopy with NBI, results of up to 97.4% for one database and up to 84% for the other were achieved. Non-neoplastic lesions were classified more accurately in general than non-neoplastic lesions. It was shown that the image recording conditions highly affect the performance of automated diagnosis systems and partly contribute to a stronger effect on the staging results than the used imaging modality.CONCLUSION Chromoendoscopy has a negative impact on the results of the methods. NBI is better suited than chromoendoscopy. High-definition and high-magnification endoscopy are equally suited.

A 63-Year-Old Male with D-Transposition of the Great Arteries Who Had an Early Form of the Arterial Switch Operation

We describe a 63-year-old male who appears to have undergone an early form of the arterial switch operation for D-transposition of the great arteries performed in the mid-1960s.We review the clinical and imaging data that support our conclusion.He had a diagnostic cardiac catheterization which demonstrated severe pulmonary hypertension responsive to epoprostenol and oxygen.Our case may represent one example of the experimental surgical work done prior to Dr.Adibe Jatene’s description of thefirst successful arterial switch performed in 1975.

A Survey of Convolutional Neural Network in Breast Cancer

Problems:For people all over the world,cancer is one of the most feared diseases.Cancer is one of the major obstacles to improving life expectancy in countries around the world and one of the biggest causes of death before the age of 70 in 112 countries.Among all kinds of cancers,breast cancer is the most common cancer for women.The data showed that female breast cancer had become one of themost common cancers.Aims:A large number of clinical trials have proved that if breast cancer is diagnosed at an early stage,it could give patients more treatment options and improve the treatment effect and survival ability.Based on this situation,there are many diagnostic methods for breast cancer,such as computer-aided diagnosis(CAD).Methods:We complete a comprehensive review of the diagnosis of breast cancer based on the convolutional neural network(CNN)after reviewing a sea of recent papers.Firstly,we introduce several different imaging modalities.The structure of CNN is given in the second part.After that,we introduce some public breast cancer data sets.Then,we divide the diagnosis of breast cancer into three different tasks:1.classification;2.detection;3.segmentation.Conclusion:Although this diagnosis with CNN has achieved great success,there are still some limitations.(i)There are too few good data sets.A good public breast cancer dataset needs to involve many aspects,such as professional medical knowledge,privacy issues,financial issues,dataset size,and so on.(ii)When the data set is too large,the CNN-based model needs a sea of computation and time to complete the diagnosis.(iii)It is easy to cause overfitting when using small data sets.

Emerging role of computed tomography coronary angiography in evaluation of children with Kawasaki disease

Coronary artery abnormalities are the most important complications in children with Kawasaki disease(KD).Two-dimensional transthoracic echocardiography currently is the standard of care for initial evaluation and follow-up of children with KD.However,it has inherent limitations with regard to evaluation of mid and distal coronary arteries and,left circumflex artery and the poor acoustic window in older children often makes evaluation difficult in this age group.Catheter angiography(CA)is invasive,has high radiation exposure and fails to demonstrate abnormalities beyond lumen.The limitations of echocardiography and CA necessitate the use of an imaging modality that overcomes these problems.In recent years advances in computed tomography technology have enabled explicit evaluation of coronary arteries along their entire course including major branches with optimal and acceptable radiation exposure in children.Computed tomography coronary angiography(CTCA)can be performed during acute as well as convalescent phases of KD.It is likely that CTCA may soon be considered the reference standard imaging modality for evaluation of coronary arteries in children with KD.

Brain Tumor Segmentation in Multimodal MRI Using U-Net Layered Structure

The brain tumour is the mass where some tissues become old or damaged,but they do not die or not leave their space.Mainly brain tumour masses occur due to malignant masses.These tissues must die so that new tissues are allowed to be born and take their place.Tumour segmentation is a complex and time-taking problem due to the tumour’s size,shape,and appearance variation.Manually finding such masses in the brain by analyzing Magnetic Resonance Images(MRI)is a crucial task for experts and radiologists.Radiologists could not work for large volume images simultaneously,and many errors occurred due to overwhelming image analysis.The main objective of this research study is the segmentation of tumors in brain MRI images with the help of digital image processing and deep learning approaches.This research study proposed an automatic model for tumor segmentation in MRI images.The proposed model has a few significant steps,which first apply the pre-processing method for the whole dataset to convert Neuroimaging Informatics Technology Initiative(NIFTI)volumes into the 3D NumPy array.In the second step,the proposed model adopts U-Net deep learning segmentation algorithm with an improved layered structure and sets the updated parameters.In the third step,the proposed model uses state-of-the-art Medical Image Computing and Computer-Assisted Intervention(MICCAI)BRATS 2018 dataset withMRI modalities such as T1,T1Gd,T2,and Fluidattenuated inversion recovery(FLAIR).Tumour types in MRI images are classified according to the tumour masses.Labelling of these masses carried by state-of-the-art approaches such that the first is enhancing tumour(label 4),edema(label 2),necrotic and non-enhancing tumour core(label 1),and the remaining region is label 0 such that edema(whole tumour),necrosis and active.The proposed model is evaluated and gets the Dice Coefficient(DSC)value for High-grade glioma(HGG)volumes for their test set-a,test set-b,and test set-c 0.9795, 0.9855 and 0.9793, respectively. DSC value for the Low-gradeglioma (LGG) volumes for the test set is 0.9950, which shows the proposedmodel has achieved significant results in segmenting the tumour in MRI usingdeep learning approaches. The proposed model is fully automatic that canimplement in clinics where human experts consumemaximumtime to identifythe tumorous region of the brain MRI. The proposed model can help in a wayit can proceed rapidly by treating the tumor segmentation in MRI.

Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

Optimal C-arm angulation during transcatheter aortic valve replacement: Accuracy of a rotational C-arm computed tomography based three dimensional heart model

AIMTo investigate the accuracy of a rotational C-arm CT-based 3D heart model to predict an optimal C-arm configuration during transcatheter aortic valve replacement (TAVR).METHODSRotational C-arm CT (RCT) under rapid ventricular pacing was performed in 57 consecutive patients with severe aortic stenosis as part of the pre-procedural cardiac catheterization. With prototype software each RCT data set was segmented using a 3D heart model. From that the line of perpendicularity curve was obtained that generates a perpendicular view of the aortic annulus according to the right-cusp rule. To evaluate the accuracy of a model-based overlay we compared model- and expert-derived aortic root diameters.RESULTSFor all 57 patients in the RCT cohort diameter measurements were obtained from two independent operators and were compared to the model-based measurements. The inter-observer variability was measured to be in the range of 0°-12.96° of angular C-arm displacement for two independent operators. The model-to-operator agreement was 0°-13.82°. The model-based and expert measurements of aortic root diameters evaluated at the aortic annulus (r = 0.79, P < 0.01), the aortic sinus (r = 0.93, P < 0.01) and the sino-tubular junction (r = 0.92, P < 0.01) correlated on a high level and the Bland-Altman analysis showed good agreement. The interobserver measurements did not show a significant bias.CONCLUSIONAutomatic segmentation of the aortic root using an anatomical model can accurately predict an optimal C-arm configuration, potentially simplifying current clinical workflows before and during TAVR.

Mediastinal Tuberculoma Mimicking Malignant Cardiac Tumor:A Case Report

Background:The clinical manifestations of cardiac masses are diverse and lack specifi city.Here we report a cardiac mass detected by transthoracic echocardiography.Multimodality imaging and pathological fi ndings after the operation confi rmed the mass as mediastinal tuberculoma.Case presentation:A 45-year-old male patient was admitted to our hospital reporting chest tightness,weight loss,and dyspnea for 3 months after exercise.Transthoracic echocardiography showed that there were a large number of pericardial effusions and a soft tissue mass measuring 7.7 cm×4.5 cm in the upper mediastinum,which oppressed the right pulmonary artery and accelerated the blood fl ow of the left pulmonary artery.Contrast-enhanced ultrasonography showed degenerative inhomogeneous high enhancement of and an unclear boundary in the mass.Contrast-enhanced chest CT revealed punctate and patchy calcifi cation in and uneven enhancement of the mass and the lymph nodes around the aortic arch.The mass was diagnosed as a malignant mediastinal tumor.Pathological analysis of the mass revealed chronic granulomatous tuberculosis.The symptoms abated signifi cantly after antituberculosis treatment.The patient remained asymptomatic during follow-up.Conclusion:This report presents a rare case of mediastinal tuberculoma mimicking a malignant cardiac tumor.Multimodality imaging should be incorporated for differentiation of cardiac masses.

Imaging-Guided Cancer Therapy Based on Multifunctional Magnetic Nanoparticles

In recent years,magnetic nanoparticles(MNPs)have received great attention within the field of biomedicine,especially for cancer therapy.This is because MNPs have many excellent physical and chemical properties to provide sufficient imaging information along with satisfactory therapeutic efficacy.Moreover,by virtue of various modification strategies,the obtained multifunctional MNPs can further achieve synergized multimodal cancer theranostic,which is worthy of further study.In this review,we summarize the recent developments in imaging-guided strategies and synergistic cancer therapy based on multifunctional MNPs.Then,we discuss the challenge and perspective of the next generation of MNPs-based imaging-guided cancer therapy,hoping to provide guidance in potential applications.