Withthe rapiddevelopment of deep learning,the size of data sets anddeepneuralnetworks(DNNs)models are also booming.As a result,the intolerable long time for models’training or inference with conventional strategies c...Withthe rapiddevelopment of deep learning,the size of data sets anddeepneuralnetworks(DNNs)models are also booming.As a result,the intolerable long time for models’training or inference with conventional strategies can not meet the satisfaction of modern tasks gradually.Moreover,devices stay idle in the scenario of edge computing(EC),which presents a waste of resources since they can share the pressure of the busy devices but they do not.To address the problem,the strategy leveraging distributed processing has been applied to load computation tasks from a single processor to a group of devices,which results in the acceleration of training or inference of DNN models and promotes the high utilization of devices in edge computing.Compared with existing papers,this paper presents an enlightening and novel review of applying distributed processing with data and model parallelism to improve deep learning tasks in edge computing.Considering the practicalities,commonly used lightweight models in a distributed system are introduced as well.As the key technique,the parallel strategy will be described in detail.Then some typical applications of distributed processing will be analyzed.Finally,the challenges of distributed processing with edge computing will be described.展开更多
当前对于建筑景观格局特征研究的方法缺乏对其隐藏性特征进行提取,因此研究引入共享的思想构建了多重对抗信息共享生成网络模型(Generation Network for Multiple Counter Information Sharing,GNMCIS),并对其有效性和实用性进行了验证...当前对于建筑景观格局特征研究的方法缺乏对其隐藏性特征进行提取,因此研究引入共享的思想构建了多重对抗信息共享生成网络模型(Generation Network for Multiple Counter Information Sharing,GNMCIS),并对其有效性和实用性进行了验证。实验结果表明,GNMCIS模型在甄别器训练出现过好状态时会对其权值参数进行及时调整,训练过差时会对其进行拉升操作,从而有效规避了梯度下降和消失的出现。同时其在50×100的像素下的训练时间为124min,在50×200的像素下的训练时间为154min,均优于对比模型。并且其监测出三个区域内最优监测点的平均温度维持在26℃左右,有效分析出相关热环境信息。综合来看,GNMCIS模型在形成具备获取特征的建筑景观格局上具备较高的有效性,对实际优化建筑景观格局上具备借鉴意义。展开更多
近年来,深度神经网络(deep neural network,DNN)在众多领域取得了广泛应用,甚至在安全关键系统中已经可以代替人类作出决策,如自动驾驶和智慧医疗等,这对DNN的可靠性提出了更高的要求.由于DNN具有复杂的多层非线性网络结构,难以理解其...近年来,深度神经网络(deep neural network,DNN)在众多领域取得了广泛应用,甚至在安全关键系统中已经可以代替人类作出决策,如自动驾驶和智慧医疗等,这对DNN的可靠性提出了更高的要求.由于DNN具有复杂的多层非线性网络结构,难以理解其内部预测机制,也很难对其进行调试.现有的DNN调试工作主要通过修改DNN的参数和扩增数据集提升DNN性能,以达到优化的目的.然而直接调整参数难以控制修改的幅度,甚至使模型失去对训练数据的拟合能力;而无指导地扩增训练数据则会大幅增加训练成本.针对此问题,提出了一种基于可解释性分析的DNN优化方法(optimizing DNN based on interpretability analysis,OptDIA).对DNN的训练过程及决策行为进行解释分析,根据解释分析结果,将原始数据中对DNN决策行为产生不同程度影响的部分以不同概率进行数据变换以生成新训练数据,并重训练DNN,以提升模型性能达到优化DNN的目的.在使用3个数据集训练的9个DNN模型上的实验结果表明,OptDIA可以将DNN的准确率提升0.39~2.15个百分点,F1-score提升0.11~2.03个百分点.展开更多
We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently develo...We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently developed deep neural network(DNN)learning methods,our approach is particularly suitable for practical situations when(i)measurement data are available for only a subset of the state variables,and(ii)the system parameters cannot be observed or measured at all.We demonstrate that,with a properly designed DNN structure with memory terms,effective DNN models can be learned from such partially observed data containing hidden parameters.The learned DNN model serves as an accurate predictive tool for system analysis.Through a few representative biological problems,we demonstrate that such DNN models can capture qualitative dynamical behavior changes in the system,such as bifurcations,even when the parameters controlling such behavior changes are completely unknown throughout not only the model learning process but also the system prediction process.The learned DNN model effectively creates a“closed”model involving only the observables when such a closed-form model does not exist mathematically.展开更多
Cross entropy is a measure in machine learning and deep learning that assesses the difference between predicted and actual probability distributions. In this study, we propose cross entropy as a performance evaluation...Cross entropy is a measure in machine learning and deep learning that assesses the difference between predicted and actual probability distributions. In this study, we propose cross entropy as a performance evaluation metric for image classifier models and apply it to the CT image classification of lung cancer. A convolutional neural network is employed as the deep neural network (DNN) image classifier, with the residual network (ResNet) 50 chosen as the DNN archi-tecture. The image data used comprise a lung CT image set. Two classification models are built from datasets with varying amounts of data, and lung cancer is categorized into four classes using 10-fold cross-validation. Furthermore, we employ t-distributed stochastic neighbor embedding to visually explain the data distribution after classification. Experimental results demonstrate that cross en-tropy is a highly useful metric for evaluating the reliability of image classifier models. It is noted that for a more comprehensive evaluation of model perfor-mance, combining with other evaluation metrics is considered essential. .展开更多
基金supported by the Natural Science Foundation of Jiangsu Province of China under Grant No.BK20211284the Financial and Science Technology Plan Project of Xinjiang Production,Construction Corps under Grant No.2020DB005the National Natural Science Foundation of China under Grant Nos.61872219,62002276 and 62177014。
文摘Withthe rapiddevelopment of deep learning,the size of data sets anddeepneuralnetworks(DNNs)models are also booming.As a result,the intolerable long time for models’training or inference with conventional strategies can not meet the satisfaction of modern tasks gradually.Moreover,devices stay idle in the scenario of edge computing(EC),which presents a waste of resources since they can share the pressure of the busy devices but they do not.To address the problem,the strategy leveraging distributed processing has been applied to load computation tasks from a single processor to a group of devices,which results in the acceleration of training or inference of DNN models and promotes the high utilization of devices in edge computing.Compared with existing papers,this paper presents an enlightening and novel review of applying distributed processing with data and model parallelism to improve deep learning tasks in edge computing.Considering the practicalities,commonly used lightweight models in a distributed system are introduced as well.As the key technique,the parallel strategy will be described in detail.Then some typical applications of distributed processing will be analyzed.Finally,the challenges of distributed processing with edge computing will be described.
文摘当前对于建筑景观格局特征研究的方法缺乏对其隐藏性特征进行提取,因此研究引入共享的思想构建了多重对抗信息共享生成网络模型(Generation Network for Multiple Counter Information Sharing,GNMCIS),并对其有效性和实用性进行了验证。实验结果表明,GNMCIS模型在甄别器训练出现过好状态时会对其权值参数进行及时调整,训练过差时会对其进行拉升操作,从而有效规避了梯度下降和消失的出现。同时其在50×100的像素下的训练时间为124min,在50×200的像素下的训练时间为154min,均优于对比模型。并且其监测出三个区域内最优监测点的平均温度维持在26℃左右,有效分析出相关热环境信息。综合来看,GNMCIS模型在形成具备获取特征的建筑景观格局上具备较高的有效性,对实际优化建筑景观格局上具备借鉴意义。
文摘近年来,深度神经网络(deep neural network,DNN)在众多领域取得了广泛应用,甚至在安全关键系统中已经可以代替人类作出决策,如自动驾驶和智慧医疗等,这对DNN的可靠性提出了更高的要求.由于DNN具有复杂的多层非线性网络结构,难以理解其内部预测机制,也很难对其进行调试.现有的DNN调试工作主要通过修改DNN的参数和扩增数据集提升DNN性能,以达到优化的目的.然而直接调整参数难以控制修改的幅度,甚至使模型失去对训练数据的拟合能力;而无指导地扩增训练数据则会大幅增加训练成本.针对此问题,提出了一种基于可解释性分析的DNN优化方法(optimizing DNN based on interpretability analysis,OptDIA).对DNN的训练过程及决策行为进行解释分析,根据解释分析结果,将原始数据中对DNN决策行为产生不同程度影响的部分以不同概率进行数据变换以生成新训练数据,并重训练DNN,以提升模型性能达到优化DNN的目的.在使用3个数据集训练的9个DNN模型上的实验结果表明,OptDIA可以将DNN的准确率提升0.39~2.15个百分点,F1-score提升0.11~2.03个百分点.
基金supported by the NSF(No.DMS-1813071)(Chou)and the AFSOR(No.FA9550-22-1-0011)(Xiu).
文摘We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently developed deep neural network(DNN)learning methods,our approach is particularly suitable for practical situations when(i)measurement data are available for only a subset of the state variables,and(ii)the system parameters cannot be observed or measured at all.We demonstrate that,with a properly designed DNN structure with memory terms,effective DNN models can be learned from such partially observed data containing hidden parameters.The learned DNN model serves as an accurate predictive tool for system analysis.Through a few representative biological problems,we demonstrate that such DNN models can capture qualitative dynamical behavior changes in the system,such as bifurcations,even when the parameters controlling such behavior changes are completely unknown throughout not only the model learning process but also the system prediction process.The learned DNN model effectively creates a“closed”model involving only the observables when such a closed-form model does not exist mathematically.
文摘Cross entropy is a measure in machine learning and deep learning that assesses the difference between predicted and actual probability distributions. In this study, we propose cross entropy as a performance evaluation metric for image classifier models and apply it to the CT image classification of lung cancer. A convolutional neural network is employed as the deep neural network (DNN) image classifier, with the residual network (ResNet) 50 chosen as the DNN archi-tecture. The image data used comprise a lung CT image set. Two classification models are built from datasets with varying amounts of data, and lung cancer is categorized into four classes using 10-fold cross-validation. Furthermore, we employ t-distributed stochastic neighbor embedding to visually explain the data distribution after classification. Experimental results demonstrate that cross en-tropy is a highly useful metric for evaluating the reliability of image classifier models. It is noted that for a more comprehensive evaluation of model perfor-mance, combining with other evaluation metrics is considered essential. .
