Tumour segmentation in medical images(especially 3D tumour segmentation)is highly challenging due to the possible similarity between tumours and adjacent tissues,occurrence of multiple tumours and variable tumour shap...Tumour segmentation in medical images(especially 3D tumour segmentation)is highly challenging due to the possible similarity between tumours and adjacent tissues,occurrence of multiple tumours and variable tumour shapes and sizes.The popular deep learning‐based segmentation algorithms generally rely on the convolutional neural network(CNN)and Transformer.The former cannot extract the global image features effectively while the latter lacks the inductive bias and involves the complicated computation for 3D volume data.The existing hybrid CNN‐Transformer network can only provide the limited performance improvement or even poorer segmentation performance than the pure CNN.To address these issues,a short‐term and long‐term memory self‐attention network is proposed.Firstly,a distinctive self‐attention block uses the Transformer to explore the correlation among the region features at different levels extracted by the CNN.Then,the memory structure filters and combines the above information to exclude the similar regions and detect the multiple tumours.Finally,the multi‐layer reconstruction blocks will predict the tumour boundaries.Experimental results demonstrate that our method outperforms other methods in terms of subjective visual and quantitative evaluation.Compared with the most competitive method,the proposed method provides Dice(82.4%vs.76.6%)and Hausdorff distance 95%(HD95)(10.66 vs.11.54 mm)on the KiTS19 as well as Dice(80.2%vs.78.4%)and HD95(9.632 vs.12.17 mm)on the LiTS.展开更多
Currently,deep learning is widely used in medical image segmentation and has achieved good results.However,3D medical image segmentation tasks with diverse lesion characters,blurred edges,and unstable positions requir...Currently,deep learning is widely used in medical image segmentation and has achieved good results.However,3D medical image segmentation tasks with diverse lesion characters,blurred edges,and unstable positions require complex networks with a large number of parameters.It is computationally expensive and results in high requirements on equipment,making it hard to deploy the network in hospitals.In this work,we propose a method for network lightweighting and applied it to a 3D CNN based network.We experimented on a COVID-19 lesion segmentation dataset.Specifically,we use three cascaded one-dimensional convolutions to replace a 3D convolution,and integrate instance normalization with the previous layer of one-dimensional convolutions to accelerate network inference.In addition,we simplify test-time augmentation and deep supervision of the network.Experiments show that the lightweight network can reduce the prediction time of each sample and the memory usage by 50%and reduce the number of parameters by 60%compared with the original network.The training time of one epoch is also reduced by 50%with the segmentation accuracy dropped within the acceptable range.展开更多
Based on patient computerized tomography data,we segmented a region containing an intracranial hematoma using the threshold method and reconstructed the 3D hematoma model.To improve the efficiency and accuracy of iden...Based on patient computerized tomography data,we segmented a region containing an intracranial hematoma using the threshold method and reconstructed the 3D hematoma model.To improve the efficiency and accuracy of identifying puncture points,a point-cloud search arithmetic method for modified adaptive weighted particle swarm optimization is proposed and used for optimal external axis extraction.According to the characteristics of the multitube drainage tube and the clinical needs of puncture for intracranial hematoma removal,the proposed algorithm can provide an optimal route for a drainage tube for the hematoma,the precise position of the puncture point,and preoperative planning information,which have considerable instructional significance for clinicians.展开更多
In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant fo...In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.展开更多
The volumetric rendering of 3 D medical image data is very effective method for communication about radiological studies to clinicians. Algorithms that produce images with artifacts and inaccuracies are not clinically...The volumetric rendering of 3 D medical image data is very effective method for communication about radiological studies to clinicians. Algorithms that produce images with artifacts and inaccuracies are not clinically useful. This paper proposed a direct voxel projection algorithm to implement volumetric data rendering. Using this algorithm, arbitrary volume rotation, transparent and cutaway views are generated satisfactorily. Compared with the existing raytracing methods, it improves the projection image quality greatly. Some experimental results about real medical CT image data demonstrate the advantages and fidelity of the proposed algorithm.展开更多
As a deep learning network with an encoder-decoder architecture,UNet and its series of improved versions have been widely used in medical image segmentation with great applications.However,when used to segment targets...As a deep learning network with an encoder-decoder architecture,UNet and its series of improved versions have been widely used in medical image segmentation with great applications.However,when used to segment targets in 3D medical images such as magnetic resonance imaging(MRI),computed tomography(CT),these models do not model the relevance of images in vertical space,resulting in poor accurate analysis of consecutive slices of the same patient.On the other hand,the large amount of detail lost during the encoding process makes these models incapable of segmenting small-scale tumor targets.Aiming at the scene of small-scale target segmentation in 3D medical images,a fully new neural network model SUNet++is proposed on the basis of UNet and UNet++.SUNet++improves the existing models mainly in three aspects:1)the modeling strategy of slice superposition is used to thoroughly excavate the three dimensional information of the data;2)by adding an attention mechanism during the decoding process,small scale targets in the picture are retained and amplified;3)in the up-sampling process,the transposed convolution operation is used to further enhance the effect of the model.In order to verify the effect of the model,we collected and produced a dataset of hyperintensity MRI liver-stage images containing over 400 cases of liver nodules.Experimental results on both public and proprietary datasets demonstrate the superiority of SUNet++in small-scale target segmentation of three-dimensional medical images.展开更多
基金supported by the National Key Research and Development Program of China under Grant No.2018YFE0206900the National Natural Science Foundation of China under Grant No.61871440 and CAAI‐Huawei Mind-Spore Open Fund.
文摘Tumour segmentation in medical images(especially 3D tumour segmentation)is highly challenging due to the possible similarity between tumours and adjacent tissues,occurrence of multiple tumours and variable tumour shapes and sizes.The popular deep learning‐based segmentation algorithms generally rely on the convolutional neural network(CNN)and Transformer.The former cannot extract the global image features effectively while the latter lacks the inductive bias and involves the complicated computation for 3D volume data.The existing hybrid CNN‐Transformer network can only provide the limited performance improvement or even poorer segmentation performance than the pure CNN.To address these issues,a short‐term and long‐term memory self‐attention network is proposed.Firstly,a distinctive self‐attention block uses the Transformer to explore the correlation among the region features at different levels extracted by the CNN.Then,the memory structure filters and combines the above information to exclude the similar regions and detect the multiple tumours.Finally,the multi‐layer reconstruction blocks will predict the tumour boundaries.Experimental results demonstrate that our method outperforms other methods in terms of subjective visual and quantitative evaluation.Compared with the most competitive method,the proposed method provides Dice(82.4%vs.76.6%)and Hausdorff distance 95%(HD95)(10.66 vs.11.54 mm)on the KiTS19 as well as Dice(80.2%vs.78.4%)and HD95(9.632 vs.12.17 mm)on the LiTS.
文摘Currently,deep learning is widely used in medical image segmentation and has achieved good results.However,3D medical image segmentation tasks with diverse lesion characters,blurred edges,and unstable positions require complex networks with a large number of parameters.It is computationally expensive and results in high requirements on equipment,making it hard to deploy the network in hospitals.In this work,we propose a method for network lightweighting and applied it to a 3D CNN based network.We experimented on a COVID-19 lesion segmentation dataset.Specifically,we use three cascaded one-dimensional convolutions to replace a 3D convolution,and integrate instance normalization with the previous layer of one-dimensional convolutions to accelerate network inference.In addition,we simplify test-time augmentation and deep supervision of the network.Experiments show that the lightweight network can reduce the prediction time of each sample and the memory usage by 50%and reduce the number of parameters by 60%compared with the original network.The training time of one epoch is also reduced by 50%with the segmentation accuracy dropped within the acceptable range.
基金funded by the National Science Foundation of China,Nos.51674121 and 61702184the Returned Overseas Scholar Funding of Hebei Province,No.C2015005014the Hebei Key Laboratory of Science and Application,and Tangshan Innovation Team Project,No.18130209B.
文摘Based on patient computerized tomography data,we segmented a region containing an intracranial hematoma using the threshold method and reconstructed the 3D hematoma model.To improve the efficiency and accuracy of identifying puncture points,a point-cloud search arithmetic method for modified adaptive weighted particle swarm optimization is proposed and used for optimal external axis extraction.According to the characteristics of the multitube drainage tube and the clinical needs of puncture for intracranial hematoma removal,the proposed algorithm can provide an optimal route for a drainage tube for the hematoma,the precise position of the puncture point,and preoperative planning information,which have considerable instructional significance for clinicians.
基金supported by Shandong Provincial Natural Science Foundation(No.ZR2023MF062)the National Natural Science Foundation of China(No.61771230).
文摘In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.
基金Shanghai Science and Technology Devel-opment Fund(9944 190 2 7)
文摘The volumetric rendering of 3 D medical image data is very effective method for communication about radiological studies to clinicians. Algorithms that produce images with artifacts and inaccuracies are not clinically useful. This paper proposed a direct voxel projection algorithm to implement volumetric data rendering. Using this algorithm, arbitrary volume rotation, transparent and cutaway views are generated satisfactorily. Compared with the existing raytracing methods, it improves the projection image quality greatly. Some experimental results about real medical CT image data demonstrate the advantages and fidelity of the proposed algorithm.
基金This work was supported by the National Natural Science Foundation of China(No.62072135)Natural Science Foundation of Ningxia Hui Autonomous Region(No.2022AAC03346)Fundamental Research Funds for the Central Universities(No.3072020CF0602).
文摘As a deep learning network with an encoder-decoder architecture,UNet and its series of improved versions have been widely used in medical image segmentation with great applications.However,when used to segment targets in 3D medical images such as magnetic resonance imaging(MRI),computed tomography(CT),these models do not model the relevance of images in vertical space,resulting in poor accurate analysis of consecutive slices of the same patient.On the other hand,the large amount of detail lost during the encoding process makes these models incapable of segmenting small-scale tumor targets.Aiming at the scene of small-scale target segmentation in 3D medical images,a fully new neural network model SUNet++is proposed on the basis of UNet and UNet++.SUNet++improves the existing models mainly in three aspects:1)the modeling strategy of slice superposition is used to thoroughly excavate the three dimensional information of the data;2)by adding an attention mechanism during the decoding process,small scale targets in the picture are retained and amplified;3)in the up-sampling process,the transposed convolution operation is used to further enhance the effect of the model.In order to verify the effect of the model,we collected and produced a dataset of hyperintensity MRI liver-stage images containing over 400 cases of liver nodules.Experimental results on both public and proprietary datasets demonstrate the superiority of SUNet++in small-scale target segmentation of three-dimensional medical images.