Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices...Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.展开更多
In recent years evidence has emerged suggesting that Mini-basketball training program(MBTP)can be an effec-tive intervention method to improve social communication(SC)impairments and restricted and repetitive beha-vio...In recent years evidence has emerged suggesting that Mini-basketball training program(MBTP)can be an effec-tive intervention method to improve social communication(SC)impairments and restricted and repetitive beha-viors(RRBs)in preschool children suffering from autism spectrum disorder(ASD).However,there is a considerable degree if interindividual variability concerning these social outcomes and thus not all preschool chil-dren with ASD profit from a MBTP intervention to the same extent.In order to make more accurate predictions which preschool children with ASD can benefit from an MBTP intervention or which preschool children with ASD need additional interventions to achieve behavioral improvements,further research is required.This study aimed to investigate which individual factors of preschool children with ASD can predict MBTP intervention out-comes concerning SC impairments and RRBs.Then,test the performance of machine learning models in predict-ing intervention outcomes based on these factors.Participants were 26 preschool children with ASD who enrolled in a quasi-experiment and received MBTP intervention.Baseline demographic variables(e.g.,age,body,mass index[BMI]),indicators of physicalfitness(e.g.,handgrip strength,balance performance),performance in execu-tive function,severity of ASD symptoms,level of SC impairments,and severity of RRBs were obtained to predict treatment outcomes after MBTP intervention.Machine learning models were established based on support vector machine algorithm were implemented.For comparison,we also employed multiple linear regression models in statistics.Ourfindings suggest that in preschool children with ASD symptomatic severity(r=0.712,p<0.001)and baseline SC impairments(r=0.713,p<0.001)are predictors for intervention outcomes of SC impair-ments.Furthermore,BMI(r=-0.430,p=0.028),symptomatic severity(r=0.656,p<0.001),baseline SC impair-ments(r=0.504,p=0.009)and baseline RRBs(r=0.647,p<0.001)can predict intervention outcomes of RRBs.Statistical models predicted 59.6%of variance in post-treatment SC impairments(MSE=0.455,RMSE=0.675,R2=0.596)and 58.9%of variance in post-treatment RRBs(MSE=0.464,RMSE=0.681,R2=0.589).Machine learning models predicted 83%of variance in post-treatment SC impairments(MSE=0.188,RMSE=0.434,R2=0.83)and 85.9%of variance in post-treatment RRBs(MSE=0.051,RMSE=0.226,R2=0.859),which were better than statistical models.Ourfindings suggest that baseline characteristics such as symptomatic severity of 144 IJMHP,2022,vol.24,no.2 ASD symptoms and SC impairments are important predictors determining MBTP intervention-induced improvements concerning SC impairments and RBBs.Furthermore,the current study revealed that machine learning models can successfully be applied to predict the MBTP intervention-related outcomes in preschool chil-dren with ASD,and performed better than statistical models.Ourfindings can help to inform which preschool children with ASD are most likely to benefit from an MBTP intervention,and they might provide a reference for the development of personalized intervention programs for preschool children with ASD.展开更多
In this paper,we summarize recent progresses made in deep learning based acoustic models and the motivation and insights behind the surveyed techniques.We first discuss models such as recurrent neural networks(RNNs) a...In this paper,we summarize recent progresses made in deep learning based acoustic models and the motivation and insights behind the surveyed techniques.We first discuss models such as recurrent neural networks(RNNs) and convolutional neural networks(CNNs) that can effectively exploit variablelength contextual information,and their various combination with other models.We then describe models that are optimized end-to-end and emphasize on feature representations learned jointly with the rest of the system,the connectionist temporal classification(CTC) criterion,and the attention-based sequenceto-sequence translation model.We further illustrate robustness issues in speech recognition systems,and discuss acoustic model adaptation,speech enhancement and separation,and robust training strategies.We also cover modeling techniques that lead to more efficient decoding and discuss possible future directions in acoustic model research.展开更多
TheUAV pursuit-evasion problem focuses on the efficient tracking and capture of evading targets using unmanned aerial vehicles(UAVs),which is pivotal in public safety applications,particularly in scenarios involving i...TheUAV pursuit-evasion problem focuses on the efficient tracking and capture of evading targets using unmanned aerial vehicles(UAVs),which is pivotal in public safety applications,particularly in scenarios involving intrusion monitoring and interception.To address the challenges of data acquisition,real-world deployment,and the limited intelligence of existing algorithms in UAV pursuit-evasion tasks,we propose an innovative swarm intelligencebased UAV pursuit-evasion control framework,namely“Boids Model-based DRL Approach for Pursuit and Escape”(Boids-PE),which synergizes the strengths of swarm intelligence from bio-inspired algorithms and deep reinforcement learning(DRL).The Boids model,which simulates collective behavior through three fundamental rules,separation,alignment,and cohesion,is adopted in our work.By integrating Boids model with the Apollonian Circles algorithm,significant improvements are achieved in capturing UAVs against simple evasion strategies.To further enhance decision-making precision,we incorporate a DRL algorithm to facilitate more accurate strategic planning.We also leverage self-play training to continuously optimize the performance of pursuit UAVs.During experimental evaluation,we meticulously designed both one-on-one and multi-to-one pursuit-evasion scenarios,customizing the state space,action space,and reward function models for each scenario.Extensive simulations,supported by the PyBullet physics engine,validate the effectiveness of our proposed method.The overall results demonstrate that Boids-PE significantly enhance the efficiency and reliability of UAV pursuit-evasion tasks,providing a practical and robust solution for the real-world application of UAV pursuit-evasion missions.展开更多
This study aims to reveal the impacts of three important uncertainty issues in landslide susceptibility prediction(LSP),namely the spatial resolution,proportion of model training and testing datasets and selection of ...This study aims to reveal the impacts of three important uncertainty issues in landslide susceptibility prediction(LSP),namely the spatial resolution,proportion of model training and testing datasets and selection of machine learning models.Taking Yanchang County of China as example,the landslide inventory and 12 important conditioning factors were acquired.The frequency ratios of each conditioning factor were calculated under five spatial resolutions(15,30,60,90 and 120 m).Landslide and non-landslide samples obtained under each spatial resolution were further divided into five proportions of training and testing datasets(9:1,8:2,7:3,6:4 and 5:5),and four typical machine learning models were applied for LSP modelling.The results demonstrated that different spatial resolution and training and testing dataset proportions induce basically similar influences on the modeling uncertainty.With a decrease in the spatial resolution from 15 m to 120 m and a change in the proportions of the training and testing datasets from 9:1 to 5:5,the modelling accuracy gradually decreased,while the mean values of predicted landslide susceptibility indexes increased and their standard deviations decreased.The sensitivities of the three uncertainty issues to LSP modeling were,in order,the spatial resolution,the choice of machine learning model and the proportions of training/testing datasets.展开更多
Latent variable models can effectively determine the condition of essential rotating machinery without needing labeled data.These models analyze vibration data via an unsupervised learning strategy.Temporal preservati...Latent variable models can effectively determine the condition of essential rotating machinery without needing labeled data.These models analyze vibration data via an unsupervised learning strategy.Temporal preservation is necessary to obtain an informative latent manifold for the fault diagnosis task.In a temporalpreserving context,two approaches exist to develop a condition-monitoring methodology:offline and online.For latent variable models,the available training modes are not different.While many traditional methods use offline training,online training can dynamically adjust the latent manifold,possibly leading to better fault signature extraction from the vibration data.This study explores online training using temporal-preserving latent variable models.Within online training,there are two main methods:one focuses on reconstructing data and the other on interpreting the data components.Both are considered to evaluate how they diagnose faults over time.Using two experimental datasets,the study confirms that models from both training modes can detect changes in machinery health and identify faults even under varying conditions.Importantly,the complementarity of offline and online models is emphasized,reassuring their versatility in fault diagnostics.Understanding the implications of the training approach and the available model formulations is crucial for further research in latent variable modelbased fault diagnostics.展开更多
针对国际疾病分类(ICD)自动编码方法的长文本处理、编码的层次结构以及长尾分布等导致的模型泛化能力弱的问题,提出一种充分利用医学预训练语言模型的基于提示学习和超球原型的小样本ICD自动编码方法(hypersphere prototypical with pro...针对国际疾病分类(ICD)自动编码方法的长文本处理、编码的层次结构以及长尾分布等导致的模型泛化能力弱的问题,提出一种充分利用医学预训练语言模型的基于提示学习和超球原型的小样本ICD自动编码方法(hypersphere prototypical with prompt learning,PromptHP)。首先,将编码描述与临床文本融合进提示学习模型中的提示模板,使得模型能够更加深入地理解临床文本;然后,充分利用预训练语言模型的先验知识进行初始预测;接着,在预训练语言模型输出表示的基础上引入超球原型进行类别建模和度量分类,并在医学数据集上微调网络,充分纳入数据知识,提高模型在小样本ICD编码分配任务上的性能;最后,对以上两部分预测结果集成加权获得最终编码预测结果。在公开医学数据集MIMIC-Ⅲ上的实验结果表明,该模型优于最先进的基线方法,PromptHP将小样本编码的macro-AUC、micro-AUC、macro-F_(1)和micro-F_(1)分别提高了1.77%、1.54%、14.22%、15.01%。实验结果验证了该模型在小样本编码分类任务中的有效性。展开更多
基金supported by the National Natural Science Foundation of China(62171088,U19A2052,62020106011)the Medico-Engineering Cooperation Funds from University of Electronic Science and Technology of China(ZYGX2021YGLH215,ZYGX2022YGRH005)。
文摘Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.
基金supported by grants from the National Natural Science Foundation of China(31771243)the Fok Ying Tong Education Foundation(141113)to Aiguo Chen.
文摘In recent years evidence has emerged suggesting that Mini-basketball training program(MBTP)can be an effec-tive intervention method to improve social communication(SC)impairments and restricted and repetitive beha-viors(RRBs)in preschool children suffering from autism spectrum disorder(ASD).However,there is a considerable degree if interindividual variability concerning these social outcomes and thus not all preschool chil-dren with ASD profit from a MBTP intervention to the same extent.In order to make more accurate predictions which preschool children with ASD can benefit from an MBTP intervention or which preschool children with ASD need additional interventions to achieve behavioral improvements,further research is required.This study aimed to investigate which individual factors of preschool children with ASD can predict MBTP intervention out-comes concerning SC impairments and RRBs.Then,test the performance of machine learning models in predict-ing intervention outcomes based on these factors.Participants were 26 preschool children with ASD who enrolled in a quasi-experiment and received MBTP intervention.Baseline demographic variables(e.g.,age,body,mass index[BMI]),indicators of physicalfitness(e.g.,handgrip strength,balance performance),performance in execu-tive function,severity of ASD symptoms,level of SC impairments,and severity of RRBs were obtained to predict treatment outcomes after MBTP intervention.Machine learning models were established based on support vector machine algorithm were implemented.For comparison,we also employed multiple linear regression models in statistics.Ourfindings suggest that in preschool children with ASD symptomatic severity(r=0.712,p<0.001)and baseline SC impairments(r=0.713,p<0.001)are predictors for intervention outcomes of SC impair-ments.Furthermore,BMI(r=-0.430,p=0.028),symptomatic severity(r=0.656,p<0.001),baseline SC impair-ments(r=0.504,p=0.009)and baseline RRBs(r=0.647,p<0.001)can predict intervention outcomes of RRBs.Statistical models predicted 59.6%of variance in post-treatment SC impairments(MSE=0.455,RMSE=0.675,R2=0.596)and 58.9%of variance in post-treatment RRBs(MSE=0.464,RMSE=0.681,R2=0.589).Machine learning models predicted 83%of variance in post-treatment SC impairments(MSE=0.188,RMSE=0.434,R2=0.83)and 85.9%of variance in post-treatment RRBs(MSE=0.051,RMSE=0.226,R2=0.859),which were better than statistical models.Ourfindings suggest that baseline characteristics such as symptomatic severity of 144 IJMHP,2022,vol.24,no.2 ASD symptoms and SC impairments are important predictors determining MBTP intervention-induced improvements concerning SC impairments and RBBs.Furthermore,the current study revealed that machine learning models can successfully be applied to predict the MBTP intervention-related outcomes in preschool chil-dren with ASD,and performed better than statistical models.Ourfindings can help to inform which preschool children with ASD are most likely to benefit from an MBTP intervention,and they might provide a reference for the development of personalized intervention programs for preschool children with ASD.
文摘In this paper,we summarize recent progresses made in deep learning based acoustic models and the motivation and insights behind the surveyed techniques.We first discuss models such as recurrent neural networks(RNNs) and convolutional neural networks(CNNs) that can effectively exploit variablelength contextual information,and their various combination with other models.We then describe models that are optimized end-to-end and emphasize on feature representations learned jointly with the rest of the system,the connectionist temporal classification(CTC) criterion,and the attention-based sequenceto-sequence translation model.We further illustrate robustness issues in speech recognition systems,and discuss acoustic model adaptation,speech enhancement and separation,and robust training strategies.We also cover modeling techniques that lead to more efficient decoding and discuss possible future directions in acoustic model research.
文摘TheUAV pursuit-evasion problem focuses on the efficient tracking and capture of evading targets using unmanned aerial vehicles(UAVs),which is pivotal in public safety applications,particularly in scenarios involving intrusion monitoring and interception.To address the challenges of data acquisition,real-world deployment,and the limited intelligence of existing algorithms in UAV pursuit-evasion tasks,we propose an innovative swarm intelligencebased UAV pursuit-evasion control framework,namely“Boids Model-based DRL Approach for Pursuit and Escape”(Boids-PE),which synergizes the strengths of swarm intelligence from bio-inspired algorithms and deep reinforcement learning(DRL).The Boids model,which simulates collective behavior through three fundamental rules,separation,alignment,and cohesion,is adopted in our work.By integrating Boids model with the Apollonian Circles algorithm,significant improvements are achieved in capturing UAVs against simple evasion strategies.To further enhance decision-making precision,we incorporate a DRL algorithm to facilitate more accurate strategic planning.We also leverage self-play training to continuously optimize the performance of pursuit UAVs.During experimental evaluation,we meticulously designed both one-on-one and multi-to-one pursuit-evasion scenarios,customizing the state space,action space,and reward function models for each scenario.Extensive simulations,supported by the PyBullet physics engine,validate the effectiveness of our proposed method.The overall results demonstrate that Boids-PE significantly enhance the efficiency and reliability of UAV pursuit-evasion tasks,providing a practical and robust solution for the real-world application of UAV pursuit-evasion missions.
基金This research is funded by the National Natural Science Foundation of China(41807285,41762020,51879127 and 51769014E)Natural Science Foundation of Hebei Province(D2022202005).
文摘This study aims to reveal the impacts of three important uncertainty issues in landslide susceptibility prediction(LSP),namely the spatial resolution,proportion of model training and testing datasets and selection of machine learning models.Taking Yanchang County of China as example,the landslide inventory and 12 important conditioning factors were acquired.The frequency ratios of each conditioning factor were calculated under five spatial resolutions(15,30,60,90 and 120 m).Landslide and non-landslide samples obtained under each spatial resolution were further divided into five proportions of training and testing datasets(9:1,8:2,7:3,6:4 and 5:5),and four typical machine learning models were applied for LSP modelling.The results demonstrated that different spatial resolution and training and testing dataset proportions induce basically similar influences on the modeling uncertainty.With a decrease in the spatial resolution from 15 m to 120 m and a change in the proportions of the training and testing datasets from 9:1 to 5:5,the modelling accuracy gradually decreased,while the mean values of predicted landslide susceptibility indexes increased and their standard deviations decreased.The sensitivities of the three uncertainty issues to LSP modeling were,in order,the spatial resolution,the choice of machine learning model and the proportions of training/testing datasets.
文摘Latent variable models can effectively determine the condition of essential rotating machinery without needing labeled data.These models analyze vibration data via an unsupervised learning strategy.Temporal preservation is necessary to obtain an informative latent manifold for the fault diagnosis task.In a temporalpreserving context,two approaches exist to develop a condition-monitoring methodology:offline and online.For latent variable models,the available training modes are not different.While many traditional methods use offline training,online training can dynamically adjust the latent manifold,possibly leading to better fault signature extraction from the vibration data.This study explores online training using temporal-preserving latent variable models.Within online training,there are two main methods:one focuses on reconstructing data and the other on interpreting the data components.Both are considered to evaluate how they diagnose faults over time.Using two experimental datasets,the study confirms that models from both training modes can detect changes in machinery health and identify faults even under varying conditions.Importantly,the complementarity of offline and online models is emphasized,reassuring their versatility in fault diagnostics.Understanding the implications of the training approach and the available model formulations is crucial for further research in latent variable modelbased fault diagnostics.
文摘针对国际疾病分类(ICD)自动编码方法的长文本处理、编码的层次结构以及长尾分布等导致的模型泛化能力弱的问题,提出一种充分利用医学预训练语言模型的基于提示学习和超球原型的小样本ICD自动编码方法(hypersphere prototypical with prompt learning,PromptHP)。首先,将编码描述与临床文本融合进提示学习模型中的提示模板,使得模型能够更加深入地理解临床文本;然后,充分利用预训练语言模型的先验知识进行初始预测;接着,在预训练语言模型输出表示的基础上引入超球原型进行类别建模和度量分类,并在医学数据集上微调网络,充分纳入数据知识,提高模型在小样本ICD编码分配任务上的性能;最后,对以上两部分预测结果集成加权获得最终编码预测结果。在公开医学数据集MIMIC-Ⅲ上的实验结果表明,该模型优于最先进的基线方法,PromptHP将小样本编码的macro-AUC、micro-AUC、macro-F_(1)和micro-F_(1)分别提高了1.77%、1.54%、14.22%、15.01%。实验结果验证了该模型在小样本编码分类任务中的有效性。
