Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many ...Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.展开更多
In telerobotic system for remote welding, human-machine interface is one of the most important factor for enhancing capability and efficiency. This paper presents an architecture design of human-machine interface for ...In telerobotic system for remote welding, human-machine interface is one of the most important factor for enhancing capability and efficiency. This paper presents an architecture design of human-machine interface for welding telerobotic system： welding multi-modal human-machine interface. The human-machine interface integrated several control modes, which are namely shared control, teleteaching, supervisory control and local autonomous control. Space mouse, panoramic vision camera and graphics simulation system are also integrated into the human-machine interface for welding teleoperation. Finally, weld seam tracing and welding experiments of U-shape seam are performed by these control modes respectively. The results show that the system has better performance of human-machine interaction and complexity environment welding.展开更多
Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer i...Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.展开更多
Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the numbe...Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the number of ophthalmic diseases that can be classified is relatively small.Moreover,imbalanced data distribution of different ophthalmic diseases is not taken into consideration,which limits the application of deep learning techniques in realistic clinical scenes.In this paper,we propose a Multimodal Multi-disease Long-tailed Classification Network(M^(2)LC-Net)in response to the challenges mentioned above.M^(2)LC-Net leverages ResNet18-CBAM to extract features from fundus images and Optical Coherence Tomography(OCT)images,respectively,and conduct feature fusion to classify 11 common ophthalmic diseases.Moreover,Class Activation Mapping(CAM)is employed to visualize each mode to improve interpretability of M^(2)LC-Net.We conduct comprehensive experiments on realistic dataset collected from a Grade III Level A ophthalmology hospital in China,including 34,396 images of 11 disease labels.Experimental results demonstrate effectiveness of our proposed model M^(2)LC-Net.Compared with the stateof-the-art,various performance metrics have been improved significantly.Specifically,Cohen’s kappa coefficient κ has been improved by 3.21%,which is a remarkable improvement.展开更多
目的探讨影像归档和通信系统(picture archiving and communication system,PACS)结合基于问题的学习(problem-based learning,PBL)、基于病例的学习(casebased learning,CBL)、基于团队的学习(team-based learning,TBL)及任务驱动法,...目的探讨影像归档和通信系统(picture archiving and communication system,PACS)结合基于问题的学习(problem-based learning,PBL)、基于病例的学习(casebased learning,CBL)、基于团队的学习(team-based learning,TBL)及任务驱动法,利用线上(微信群)及线下资源的多模态教学模式在放射科住院医师规范化培训(简称住培)学员早读片中的应用价值。方法选取2014年6月—2020年12月在放射科参加住院医师规范化培训的住培学员70名作为研究对象,其中46名作为试验组,早读片采用PACS结合PBL、CBL、TBL及任务驱动法,利用线上(微信群)及线下资源的多模态教学;24名作为对照组,采用传统早读片教学模式。均在完成第一阶段住培结束后进行理论考核、实践技能考核及调查问卷评价。结果试验组理论考核成绩、实践技能考核成绩分别为(86.72±3.54)分、(88.59±4.02)分,对照组理论考核成绩、实践技能考核成绩分别为(79.75±6.43)分、(80.25±6.17)分,两组间成绩差异有统计学意义(P<0.05)。教学模式调查问卷评分试验组(86.13±4.33)分高于对照组(70.25±7.11)分,差异有统计学意义(P<0.05)。结论PACS结合PBL、CBL、TBL及任务驱动法,利用线上及线下资源的多模态教学模式在早读片中的应用,能够提高放射科住培学员的学习效果及综合能力。展开更多
In recent decades,several optimization algorithms have been developed for selecting the most energy efficient clusters in order to save power during trans-mission to a shorter distance while restricting the Primary Us...In recent decades,several optimization algorithms have been developed for selecting the most energy efficient clusters in order to save power during trans-mission to a shorter distance while restricting the Primary Users(PUs)interfer-ence.The Cognitive Radio(CR)system is based on the Adaptive Swarm Distributed Intelligent based Clustering algorithm(ASDIC)that shows better spectrum sensing among group of multiusers in terms of sensing error,power sav-ing,and convergence time.In this research paper,the proposed ASDIC algorithm develops better energy efficient distributed cluster based sensing with the optimal number of clusters on their connectivity.In this research,multiple random Sec-ondary Users(SUs),and PUs are considered for implementation.Hence,the pro-posed ASDIC algorithm improved the convergence speed by combining the multi-users clustered communication compared to the existing optimization algo-rithms.Experimental results showed that the proposed ASDIC algorithm reduced the node power of 9.646%compared to the existing algorithms.Similarly,ASDIC algorithm reduced 24.23%of SUs average node power compared to the existing algorithms.Probability of detection is higher by reducing the Signal-to-Noise Ratio(SNR)to 2 dB values.The proposed ASDIC delivers low false alarm rate compared to other existing optimization algorithms in the primary detection.Simulation results showed that the proposed ASDIC algorithm effectively solves the multimodal optimization problems and maximizes the performance of net-work capacity.展开更多
基金supported in part by the National Key Research and Development Program of China under Grant 2018Y FE0206900in part by the National Natural Science Foundation of China under Grant 61871440in part by the CAAIHuawei MindSpore Open Fund.We gratefully acknowledge the support of MindSpore for this research.
文摘Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.
文摘In telerobotic system for remote welding, human-machine interface is one of the most important factor for enhancing capability and efficiency. This paper presents an architecture design of human-machine interface for welding telerobotic system： welding multi-modal human-machine interface. The human-machine interface integrated several control modes, which are namely shared control, teleteaching, supervisory control and local autonomous control. Space mouse, panoramic vision camera and graphics simulation system are also integrated into the human-machine interface for welding teleoperation. Finally, weld seam tracing and welding experiments of U-shape seam are performed by these control modes respectively. The results show that the system has better performance of human-machine interaction and complexity environment welding.
基金National Natural Science Foundation of China(61976209,62020106015,U21A20388)in part by the CAS International Collaboration Key Project(173211KYSB20190024)in part by the Strategic Priority Research Program of CAS(XDB32040000)。
文摘Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.
基金the National Natural Science Foundation of China(No.62076035)。
文摘Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the number of ophthalmic diseases that can be classified is relatively small.Moreover,imbalanced data distribution of different ophthalmic diseases is not taken into consideration,which limits the application of deep learning techniques in realistic clinical scenes.In this paper,we propose a Multimodal Multi-disease Long-tailed Classification Network(M^(2)LC-Net)in response to the challenges mentioned above.M^(2)LC-Net leverages ResNet18-CBAM to extract features from fundus images and Optical Coherence Tomography(OCT)images,respectively,and conduct feature fusion to classify 11 common ophthalmic diseases.Moreover,Class Activation Mapping(CAM)is employed to visualize each mode to improve interpretability of M^(2)LC-Net.We conduct comprehensive experiments on realistic dataset collected from a Grade III Level A ophthalmology hospital in China,including 34,396 images of 11 disease labels.Experimental results demonstrate effectiveness of our proposed model M^(2)LC-Net.Compared with the stateof-the-art,various performance metrics have been improved significantly.Specifically,Cohen’s kappa coefficient κ has been improved by 3.21%,which is a remarkable improvement.
文摘目的探讨影像归档和通信系统(picture archiving and communication system,PACS)结合基于问题的学习(problem-based learning,PBL)、基于病例的学习(casebased learning,CBL)、基于团队的学习(team-based learning,TBL)及任务驱动法,利用线上(微信群)及线下资源的多模态教学模式在放射科住院医师规范化培训(简称住培)学员早读片中的应用价值。方法选取2014年6月—2020年12月在放射科参加住院医师规范化培训的住培学员70名作为研究对象,其中46名作为试验组,早读片采用PACS结合PBL、CBL、TBL及任务驱动法,利用线上(微信群)及线下资源的多模态教学;24名作为对照组,采用传统早读片教学模式。均在完成第一阶段住培结束后进行理论考核、实践技能考核及调查问卷评价。结果试验组理论考核成绩、实践技能考核成绩分别为(86.72±3.54)分、(88.59±4.02)分,对照组理论考核成绩、实践技能考核成绩分别为(79.75±6.43)分、(80.25±6.17)分,两组间成绩差异有统计学意义(P<0.05)。教学模式调查问卷评分试验组(86.13±4.33)分高于对照组(70.25±7.11)分,差异有统计学意义(P<0.05)。结论PACS结合PBL、CBL、TBL及任务驱动法,利用线上及线下资源的多模态教学模式在早读片中的应用,能够提高放射科住培学员的学习效果及综合能力。
文摘构建多模式交通系统的双动态演化模型,模型包括逐日动态演化和日内动态演化,逐日动态演化在1 d的维度上不断更新用户的多模式感知出行成本;日内动态演化依据多模式感知出行成本,采用Logit模型划分模式,并通过宏观基本图(Macroscopic Fundamental Diagram,MFD)理论计算区域内各模式的平均行驶车速和出行时长。探究共享汽车新型交通模式和区域换乘对出行者选择行为和交通系统演化均衡的影响。算例结果表明:相比长途出行,出行者在短途出行中更愿意使用共享汽车;稳态系统中,共享汽车出行将代替28.09%的私家车出行和8.52%的公共汽车出行;共享汽车出行增加了交通系统的总出行成本(1.07%)和总旅行时间(16.53%);区域换乘是重要的出行模式,降低了交通系统的总出行成本、总旅行时间和小汽车的拥有量。
文摘In recent decades,several optimization algorithms have been developed for selecting the most energy efficient clusters in order to save power during trans-mission to a shorter distance while restricting the Primary Users(PUs)interfer-ence.The Cognitive Radio(CR)system is based on the Adaptive Swarm Distributed Intelligent based Clustering algorithm(ASDIC)that shows better spectrum sensing among group of multiusers in terms of sensing error,power sav-ing,and convergence time.In this research paper,the proposed ASDIC algorithm develops better energy efficient distributed cluster based sensing with the optimal number of clusters on their connectivity.In this research,multiple random Sec-ondary Users(SUs),and PUs are considered for implementation.Hence,the pro-posed ASDIC algorithm improved the convergence speed by combining the multi-users clustered communication compared to the existing optimization algo-rithms.Experimental results showed that the proposed ASDIC algorithm reduced the node power of 9.646%compared to the existing algorithms.Similarly,ASDIC algorithm reduced 24.23%of SUs average node power compared to the existing algorithms.Probability of detection is higher by reducing the Signal-to-Noise Ratio(SNR)to 2 dB values.The proposed ASDIC delivers low false alarm rate compared to other existing optimization algorithms in the primary detection.Simulation results showed that the proposed ASDIC algorithm effectively solves the multimodal optimization problems and maximizes the performance of net-work capacity.
