Low rank optimization for efficient deep learning:making a balance between compact architecture and fast training

Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.

Exploring the Application Effect of Flipped Classroom Combined with Problem-Based Learning Teaching Method in Clinical Skills Teaching of Standardized Training for Resident Doctors of Traditional Chinese Medicine

Objective: To explore the application effect of flipped classroom combined with problem-based learning teaching method in clinical skills teaching of standardized training for resident doctors of traditional Chinese Medicine. Methods: The study used the experimental control method. The study lasted from September to November 2022. The subjects of this study were 49 students of standardized training for resident doctors of traditional Chinese Medicine from grades 2020, 2021 and 2022 of Dazhou integrated TCM & Western Medicine Hospital. They were randomly divided into experiment group (25) and control group (24). The experiment group adopted flipped classroom combined with problem-based learning teaching method, and the control group adopted traditional teaching method. The teaching content was 4 basic clinical skill projects, including four diagnoses of traditional Chinese Medicine, cardiopulmonary resuscitation, dressing change procedure, acupuncture and massage. The evaluation method was carried out by comparing the students’ performance and a self-designed questionnaire was used to investigate the students’ evaluation of the teaching method. Results: The test scores of total scores in the experimental group (90.12 ± 5.89) were all higher than those in the control group (81.47 ± 7.96) (t = 4.53, P P Conclusions: The teaching process of the flipped classroom combined with problem-based learning teaching method is conducive to improving the efficiency of classroom teaching, cultivating students’ self-learning ability, and enhancing students’ willingness to learn.

M-learning结合CBL在消化科规培教学中的探讨及应用

目的探究移动学习平台(M-learning,ML)结合案例教学(Case-based Learning,CBL)在消化科住院医师规范化培训(简称规培)教学中的应用效果。方法选取2021年1月—2023年1月于上海市徐汇区中心医院消化科参加规培学习的80名医师作为研究对象,将其按照随机数表法分为研究组和对照组,每组40名。对照组给予传统讲授式教学法,研究组给予M-learning结合CBL教学法,对比两组医师的理论考试成绩、实践技能考试成绩和学习满意度。结果研究组的理论成绩和实践技能考试成绩均高于对照组,差异具有统计学意义(P均<0.05);研究组的学习满意度明显高于对照组,差异具有统计学意义(P<0.05)。结论将Mlearning结合CBL教学法应用于消化科规培教学中,不仅能够提升医师的理论考试成绩和实践技能考试成绩,还能够有效提高医师学习满意度。

基于密度峰值聚类的Tri-training算法

Tri-training利用无标签数据进行分类可有效提高分类器的泛化能力,但其易将无标签数据误标,从而形成训练噪声。提出一种基于密度峰值聚类的Tri-training(Tri-training with density peaks clustering,DPC-TT)算法。密度峰值聚类通过类簇中心和局部密度可选出数据空间结构表现较好的样本。DPC-TT算法采用密度峰值聚类算法获取训练数据的类簇中心和样本的局部密度,对类簇中心的截断距离范围内的样本认定为空间结构表现较好,标记为核心数据,使用核心数据更新分类器,可降低迭代过程中的训练噪声,进而提高分类器的性能。实验结果表明:相比于标准Tritraining算法及其改进算法,DPC-TT算法具有更好的分类性能。

Meta-cognitive Strategy Training and the Development of Learners＇ Autonomy in Language Learning

As the ultimate goal of education, autonomy in language learning has aroused a lot of attention from scholars at home and abroad. While in universities of China, students do not have strong autonomy in English language learning. The author tries to adopt specific meta-cognitive strategies to facilitate students＇ autonomy in learning by improving learners＇ capacities in study planning or management, monitoring and evaluating in learning to raise their consciousness and ability in autonomy, and lay a foundation for life-long learning.

基于Tri-training的社交媒体药物不良反应实体抽取

社交媒体因其数据的实时性,对其充分利用可以弥补传统医疗文献药物不良反应中实体抽取的迟滞性问题,但社交媒体文本面临标注数据成本高、数据噪声大等问题,使得模型难以发挥良好的效果。针对社交媒体大量未标注语料存在标注成本高的问题,采用Tri-training半监督的方法进行社交媒体药物不良反应实体抽取,通过三个学习器Transformer+CRF、BiLSTM+CRF和IDCNN+CRF对未标注数据进行标注,再利用一致性评价函数迭代地扩展训练集,最后通过加权投票整合模型输出标签。针对社交媒体的文本不正式性(口语化严重、错别字等)问题,通过融合字与词两个粒度的向量作为整个模型嵌入层的输入,来提取更丰富的语义信息。实验结果表明,提出的模型在“好大夫在线”网站获取的数据集上取得了良好表现。

Cloudless-Training:基于serverless的高效跨地域分布式ML训练框架

跨地域分布式机器学习(ML)训练能够联合多区域的云资源协作训练,可满足许多新兴ML场景(比如大型模型训练、联邦学习)的训练需求。但其训练效率仍受2方面挑战的制约。首先,多区域云资源缺乏有效的弹性调度,这会影响训练的资源利用率和性能;其次,模型跨地域同步需要在广域网(WAN)上高频通信,受WAN的低带宽和高波动的影响,会产生巨大通信开销。本文提出Cloudless-Training,从3个方面实现高效的跨地域分布式ML训练。首先,它基于serverless计算模式实现,使用控制层和训练执行层的2层架构,支持多云区域的弹性调度和通信。其次,它提供一种弹性调度策略,根据可用云资源的异构性和训练数据集的分布自适应地部署训练工作流。最后,它提供了2种高效的跨云同步策略,包括基于梯度累积的异步随机梯度下降(ASGD-GA)和跨云参数服务器(PS)间的模型平均(MA)。Cloudless-Training是基于OpenFaaS实现的,并被部署在腾讯云上评估,实验结果表明Cloudless-Training可显著地提高跨地域分布式ML训练的资源利用率(训练成本降低了9.2%~24.0%)和同步效率(训练速度最多比基线快1.7倍),并能保证模型的收敛精度。

基于Tri-training GPR的半监督软测量建模方法

集成学习因通过构建并结合多个学习器,常获得比单一学习器显著优越的泛化能力。但是在标记数据比例较少时,建立高性能的集成学习软测量模型依然是个挑战。针对这一个问题,提出一种基于半监督集成学习的软测量建模方法——Tri-training GPR模型。该建模策略充分发挥了半监督学习的优势,减轻建模过程对标记样本数据的需求,在低数据标签率下,仍能通过对无标记数据进行筛选从而扩充可用于建模的有标记样本数据集,并进一步结合半监督学习和集成学习的优势,提出一种新的选择高置信度样本的思路。将所提方法应用于青霉素发酵和脱丁烷塔过程,建立青霉素和丁烷浓度预测软测量模型,与传统的建模方法相比获得了更优的预测结果,验证了模型的有效性。

An Application of Machine Learning Methods to Detect Mango Varieties

The mango, a fruit of immense economic and dietary significance in numerous tropical and subtropical regions, plays a pivotal role in our agricultural landscape. Accurate identification is not just a necessity, but a crucial step for effective classification, sorting, and marketing. This study delves into the potential of machine learning for this task, comparing the performance of four models: MobileNetV2, Xception, VGG16, and ResNet50V2. These models were trained on a dataset of annotated mango images, and their performance was evaluated using precision, accuracy, F1 score, and recall, which are standard metrics for image classification. The Xception model, with its exceptional performance, outshone the other models on all performance indicators. It achieved a staggering accuracy of 99.47%, an F1 score of 99.43%, and a recall of 99.43%, showcasing its remarkable ability to accurately identify mango varieties. MobileNetV2 followed closely with performances of 98.95% accuracy, 98.85% F1 score, and 98.86% recall. ResNet50V2 also delivered satisfactory results with 97.39% accuracy, 97.08% F1 score, and 97.17% recall. VGG16, however, was the least effective, with a precision rate of 83.25%, an F1 score of 83.25%, and a recall of 85.47%. These results confirm the superiority of the Xception model in detecting mango varieties. Its advanced architecture allows it to capture more distinguishing features of mango images, leading to greater precision and reliability. Xception’s robustness in identifying true positives is another advantage, minimizing false positives and contributing to more accurate classification. This study highlights the promising potential of machine learning, particularly the Xception model, for accurately identifying mango varieties.

Optimizing Network Security via Ensemble Learning: A Nexus with Intrusion Detection

Network intrusion detection systems need to be updated due to the rise in cyber threats. In order to improve detection accuracy, this research presents a strong strategy that makes use of a stacked ensemble method, which combines the advantages of several machine learning models. The ensemble is made up of various base models, such as Decision Trees, K-Nearest Neighbors (KNN), Multi-Layer Perceptrons (MLP), and Naive Bayes, each of which offers a distinct perspective on the properties of the data. The research adheres to a methodical workflow that begins with thorough data preprocessing to guarantee the accuracy and applicability of the data. In order to extract useful attributes from network traffic data—which are essential for efficient model training—feature engineering is used. The ensemble approach combines these models by training a Logistic Regression model meta-learner on base model predictions. In addition to increasing prediction accuracy, this tiered approach helps get around the drawbacks that come with using individual models. High accuracy, precision, and recall are shown in the model’s evaluation of a network intrusion dataset, indicating the model’s efficacy in identifying malicious activity. Cross-validation is used to make sure the models are reliable and well-generalized to new, untested data. In addition to advancing cybersecurity, the research establishes a foundation for the implementation of flexible and scalable intrusion detection systems. This hybrid, stacked ensemble model has a lot of potential for improving cyberattack prevention, lowering the likelihood of cyberattacks, and offering a scalable solution that can be adjusted to meet new threats and technological advancements.

Temporally Preserving Latent Variable Models:Offline and Online Training for Reconstruction and Interpretation of Fault Data for Gearbox Condition Monitoring

Latent variable models can effectively determine the condition of essential rotating machinery without needing labeled data.These models analyze vibration data via an unsupervised learning strategy.Temporal preservation is necessary to obtain an informative latent manifold for the fault diagnosis task.In a temporalpreserving context,two approaches exist to develop a condition-monitoring methodology:offline and online.For latent variable models,the available training modes are not different.While many traditional methods use offline training,online training can dynamically adjust the latent manifold,possibly leading to better fault signature extraction from the vibration data.This study explores online training using temporal-preserving latent variable models.Within online training,there are two main methods:one focuses on reconstructing data and the other on interpreting the data components.Both are considered to evaluate how they diagnose faults over time.Using two experimental datasets,the study confirms that models from both training modes can detect changes in machinery health and identify faults even under varying conditions.Importantly,the complementarity of offline and online models is emphasized,reassuring their versatility in fault diagnostics.Understanding the implications of the training approach and the available model formulations is crucial for further research in latent variable modelbased fault diagnostics.

Learning Strategy Training in English Teaching

This paper mainly deals with the comprehensive knowledge system of learning strategy.Then it tries to probe the steps of strategy training and it's significance to English teaching.

Learning strategies training in classroom teaching

In contemporary English teaching, the study of English learning strategies has become one of the main concerns in teachers＇ teaching and research processes. The necessity of implementation of learning strategies training in the field of English teaching practice still remains disputable. This essay first introduces the different reactions towards the field; then advances the idea of combining English teaching strategies with English learning strategies attempting the implementation of learning strategies training in the classroom teaching. Finally, some thinking produced in the process of practicing the strategies training is raised to reexamine this teaching mode.

Prediction of Outcomes in Mini-Basketball Training Program for Preschool Children with Autism Using Machine Learning Models

In recent years evidence has emerged suggesting that Mini-basketball training program(MBTP)can be an effec-tive intervention method to improve social communication(SC)impairments and restricted and repetitive beha-viors(RRBs)in preschool children suffering from autism spectrum disorder(ASD).However,there is a considerable degree if interindividual variability concerning these social outcomes and thus not all preschool chil-dren with ASD profit from a MBTP intervention to the same extent.In order to make more accurate predictions which preschool children with ASD can benefit from an MBTP intervention or which preschool children with ASD need additional interventions to achieve behavioral improvements,further research is required.This study aimed to investigate which individual factors of preschool children with ASD can predict MBTP intervention out-comes concerning SC impairments and RRBs.Then,test the performance of machine learning models in predict-ing intervention outcomes based on these factors.Participants were 26 preschool children with ASD who enrolled in a quasi-experiment and received MBTP intervention.Baseline demographic variables(e.g.,age,body,mass index[BMI]),indicators of physicalfitness(e.g.,handgrip strength,balance performance),performance in execu-tive function,severity of ASD symptoms,level of SC impairments,and severity of RRBs were obtained to predict treatment outcomes after MBTP intervention.Machine learning models were established based on support vector machine algorithm were implemented.For comparison,we also employed multiple linear regression models in statistics.Ourfindings suggest that in preschool children with ASD symptomatic severity(r=0.712,p<0.001)and baseline SC impairments(r=0.713,p<0.001)are predictors for intervention outcomes of SC impair-ments.Furthermore,BMI(r=-0.430,p=0.028),symptomatic severity(r=0.656,p<0.001),baseline SC impair-ments(r=0.504,p=0.009)and baseline RRBs(r=0.647,p<0.001)can predict intervention outcomes of RRBs.Statistical models predicted 59.6%of variance in post-treatment SC impairments(MSE=0.455,RMSE=0.675,R2=0.596)and 58.9%of variance in post-treatment RRBs(MSE=0.464,RMSE=0.681,R2=0.589).Machine learning models predicted 83%of variance in post-treatment SC impairments(MSE=0.188,RMSE=0.434,R2=0.83)and 85.9%of variance in post-treatment RRBs(MSE=0.051,RMSE=0.226,R2=0.859),which were better than statistical models.Ourfindings suggest that baseline characteristics such as symptomatic severity of 144 IJMHP,2022,vol.24,no.2 ASD symptoms and SC impairments are important predictors determining MBTP intervention-induced improvements concerning SC impairments and RBBs.Furthermore,the current study revealed that machine learning models can successfully be applied to predict the MBTP intervention-related outcomes in preschool chil-dren with ASD,and performed better than statistical models.Ourfindings can help to inform which preschool children with ASD are most likely to benefit from an MBTP intervention,and they might provide a reference for the development of personalized intervention programs for preschool children with ASD.

Adversarial Training Against Adversarial Attacks for Machine Learning-Based Intrusion Detection Systems

Intrusion detection system plays an important role in defending networks from security breaches.End-to-end machine learning-based intrusion detection systems are being used to achieve high detection accuracy.However,in case of adversarial attacks,that cause misclassification by introducing imperceptible perturbation on input samples,performance of machine learning-based intrusion detection systems is greatly affected.Though such problems have widely been discussed in image processing domain,very few studies have investigated network intrusion detection systems and proposed corresponding defence.In this paper,we attempt to fill this gap by using adversarial attacks on standard intrusion detection datasets and then using adversarial samples to train various machine learning algorithms(adversarial training)to test their defence performance.This is achieved by first creating adversarial sample based on Jacobian-based Saliency Map Attack(JSMA)and Fast Gradient Sign Attack(FGSM)using NSLKDD,UNSW-NB15 and CICIDS17 datasets.The study then trains and tests JSMA and FGSM based adversarial examples in seen(where model has been trained on adversarial samples)and unseen(where model is unaware of adversarial packets)attacks.The experiments includes multiple machine learning classifiers to evaluate their performance against adversarial attacks.The performance parameters include Accuracy,F1-Score and Area under the receiver operating characteristic curve(AUC)Score.

一种有效的培训方式:C-training与E-learning的融合

联合国教科文组织亚太地区总办事处、东南亚教育组织教育技术和改革区域中心举办的东南亚优化学校教学与课程领导培训，采取了C-training与E-learning相融合的工作坊培训。此种培训方式可以达到其中任意一种或是两者简单叠加达不到的培训效果：既解决了单纯C—training的开支及时间局限等问题，又避免了单纯E—Learning在情感沟通、操作指导及教学监督方面的弱势。

Multimodal teaching,learning and training in virtual reality:a review and case study

It is becoming increasingly prevalent in digital learning research to encompass an array of different meanings,spaces,processes,and teaching strategies for discerning a global perspective on constructing the student learning experience.Multimodality is an emergent phenomenon that may influence how digital learning is designed,especially when employed in highly interactive and immersive learning environments such as Virtual Reality(VR).VR environments may aid students'efforts to be active learners through consciously attending to,and reflecting on,critique leveraging reflexivity and novel meaning-making most likely to lead to a conceptual change.This paper employs eleven industrial case-studies to highlight the application of multimodal VR-based teaching and training as a pedagogically rich strategy that may be designed,mapped and visualized through distinct VR-design elements and features.The outcomes of the use cases contribute to discern in-VR multimodal teaching as an emerging discourse that couples system design-based paradigms with embodied,situated and reflective praxis in spatial,emotional and temporal VR learning environments.

Effect of behavior training on learning,memory and the expression of NR2B,GluR1 in hippocampus of rats offspring with fetal growth restriction

Objective： To study effects of behavior training on learning, memory and the expression of NR2B, GluR1 in hippocampus of rat＇ s offspring with fetal growth restriction（FGR）. Methods： The rat model of FGR was established by passive smoking method. The rats offspring were divided into the FGR group and the control group, then randomly divided into the trained and untrained group, respectively. Morris water maze test was proceeded on postnatal month（PM2/4） as a behavior training method, then the learning-memory of rats was detected through dark-avoidance and step-down tests. The expressions of NR2B and GluR1 subunits in hippocampal CA1 and CA3 areas were detected by immunohistochemical method. Results： In the dark-avoidance and step-down tests, the performance record of rats with FGR was worse than that of control rats, and the behavior-trained rats was better than the untrained rats, when the FGR model and training factors were analyzed singly. The model factor and training factor had significant interaction（P 〈 0.05）. The expressions of NR2B and GluR1 subunits in hippocampal CA1 and CA3 areas of rats with FGR reduced. In contrast, the expressions of GluR1 and NR2B subunits in CA1 area of behavior-trained rats increased, when the FGR model and training factors were analyzed singly. Conclusion： These findings suggested that the effect of behavior training on the expressions of NR2B and GluR1 subunits in CA1 area should be the mechanistic basis for the training-induced improvement in learning-memory abilities.

Communication Skills Training for Residents: Which Variables Predict Learning of Skills?

Objective: This study aimed to determine variables associated (predictors and correlates) with the learning of assessment and supportive skills in the context of a communication skills training for medical residents. Methods: Learning was measured by comparing residents’ communication skills in a simulated consultation before and after a communication skills training. Communication skills were transcribed and tagged with a computer-assisted program. Potential variables associated with learning (residents’ characteristics, contextual characteristics and pre-training communication skills) were measured before the training and entered in regression analysis. Results: Fifty-six residents followed the training between 2002 and 2006. Poor pre-training assessment and supportive skills predicted the respective learning of these skills. Better assessment skills’ learning was predicted by copings (i.e. lower level of emotional coping), lower levels of self-efficacy and depersonalization. Better supportive skills’ learning was predicted by a lower work experience and associated with a higher training attendance rate. Conclusions: Predictors and correlates of assessment and supportive skills learning were different. Trainers needed to detect certain residents’ characteristics (i.e. depersonalization) in order to optimize assessment skills learning. Trainers needed to be aware that supportive skills are difficult to learn and to teach and may need more training hours.

Effect of behavior training on learning and memory of young rats with fetal growth restriction

Objective: To investigate the effect of behavior training on the learning and memory of young rats with fetal growth restriction (FGR). Methods: The model of FGR was established by passive smoking method to pregnant rats. The new-born rats were divided into FGR group and normal group, and then randomly subdivided into trained and untrained group respectively. Morris water maze behavior training was performed on postnatal months 2 and 4, then learning and memory abilities of young rats were measured by dark-avoidance testing and step-down testing. Results: In the dark-avoidance and step-down testing, the young rats’ performance of FGR group was worse than that of control group, and the trained group was better than the untrained group significantly. Conclusion: FGR young rats have descended learning and memory abilities. Behavior training could improve the young rats’ learning and memory abilities, especially for the FGR young rats.