UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Effects of Perceived Teacher Support on Student Behavioral Engagement in the Blended Learning Environment:Learning Experience as a Mediator

Blended learning(BL)has been widely adopted to improve students’academic achievements in higher education.However,its success relies mainly on student engagement,which plays an essential role in active learning and provides a rich understanding of students’experiences.The study utilized three self-designed scales-the Teacher Support Scale,Student Engagement Scale,and Student Learning Experience Scale-to gauge and examine the impact and relationship between perceived teacher support,student behavioral engagement,and the intermediary role of learning experiences.A cohort of 899 college students undertaking the obligatory College English course through BL modes across five Chinese universities actively participated by completing a comprehensive questionnaire.The results showed significant correlations between perceived teacher support,learning experience,and behavioral engagement.Perceived teacher support significantly predicted students’behavioral engagement,with socio-affective support exerting the most substantial predictive effects.All predictive effects were partially mediated by learning experience(learning mode,online resources,overall LMS-based learning,interaction with their instructor and peers,and learning outcome).The influence of perceived teacher support on behavioral engagement differed between students who reported the most positive(vs.negative)learning experiences.Suggestions for further research are offered for consideration.

Interface Design and Functional Optimization of Chinese Learning Apps Based on User Experience

This research paper investigates the interface design and functional optimization of Chinese learning apps through the lens of user experience.With the increasing popularity of Chinese language learning apps in the era of rapid mobile internet development,users'demands for enhanced interface design and interaction experience have grown significantly.The study aims to explore the influence of user feedback on the design and functionality of Chinese learning apps,proposing optimization strategies to improve user experience and learning outcomes.By conducting a comprehensive literature review,utilizing methods such as surveys and user interviews for data collection,and analyzing user feedback,this research identifies existing issues in the interface design and interaction experience of Chinese learning apps.The results present user opinions,feedback analysis,identified problems,improvement directions,and specific optimization strategies.The study discusses the potential impact of these optimization strategies on enhancing user experience and learning outcomes,compares findings with previous research,addresses limitations,and suggests future research directions.In conclusion,this research contributes to enriching the design theory of Chinese learning apps,offering practical optimization recommendations for developers,and supporting the continuous advancement of Chinese language learning apps.

Remote Learning in Higher Education During the Pandemic: A Study of the Experiences of Chinese International Students at United Kingdom and United States Universities in 2020/21

During the COVID-19 pandemic crisis,many universities around the world made a drastic change by transferring most of their offline classes to emergency remote learning(ERL).The aim of this study was to explore how Chinese students,who studied in United Kingdom(UK)and United States(US)universities during the 2020/21 academic year,perceive their experiences of remote learning.As the UK and the US have two relatively advanced education systems,the arrangements of their universities for ERL and their support for international students are worth exploring.Moreover,during the ERL,a portion of Chinese students had online classes in their home countries instead of the country in which their universities are located.Therefore,semi-structured interviews were carried out to explore the academic experiences and social interaction of students who studied in UK and US universities,while remaining in China.The data were analyzed using the thematic analysis method.The findings showed that ERL was perceived negatively by students despite its flexibility in areas of academic learning experiences and social interaction.

Unveiling the Re,Cr,and I diffusion in saturated compacted bentonite using machine-learning methods

The safety assessment of high-level radioactive waste repositories requires a high predictive accuracy for radionuclide diffusion and a comprehensive understanding of the diffusion mechanism.In this study,a through-diffusion method and six machine-learning methods were employed to investigate the diffusion of ReO_(4)^(−),HCrO_(4)^(−),and I−in saturated compacted bentonite under different salinities and compacted dry densities.The machine-learning models were trained using two datasets.One dataset contained six input features and 293 instances obtained from the diffusion database system of the Japan Atomic Energy Agency(JAEA-DDB)and 15 publications.The other dataset,comprising 15,000 pseudo-instances,was produced using a multi-porosity model and contained eight input features.The results indicate that the former dataset yielded a higher predictive accuracy than the latter.Light gradient-boosting exhibited a higher prediction accuracy(R2=0.92)and lower error(MSE=0.01)than the other machine-learning algorithms.In addition,Shapley Additive Explanations,Feature Importance,and Partial Dependence Plot analysis results indicate that the rock capacity factor and compacted dry density had the two most significant effects on predicting the effective diffusion coefficient,thereby offering valuable insights.

QoE oriented intelligent online learning evaluation technology in B5G scenario

Students'demand for online learning has exploded during the post-COvID-19 pandemic era.However,due to their poor learning experience,students'dropout rate and learning performance of online learning are not always satisfactory.The technical advantages of Beyond Fifth Generation(B5G)can guarantee a good multimedia Quality of Experience(QoE).As a special case of multimedia services,online learning takes into account both the usability of the service and the cognitive development of the users.Factors that affect the Quality of Online Learning Experience(OL-QoE)become more complicated.To get over this dilemma,we propose a systematic scheme by integrating big data,Machine Learning(ML)technologies,and educational psychology theory.Specifically,we first formulate a general definition of OL-QoE by data analysis and experimental verification.This formula considers both the subjective and objective factors(i.e.,video watching ratio and test scores)that most affect OLQoE.Then,we induce an extended layer to the classic Broad Learning System(BLS)to construct an Extended Broad Learning System(EBLS)for the students'OL-QoE prediction.Since the extended layer can increase the width of the BLS model and reduce the redundant nodes of BLS,the proposed EBLS can achieve a trade-off between the prediction accuracy and computation complexity.Finally,we provide a series of early intervention suggestions for different types of students according to their predicted OL-QoE values.Through timely interventions,their OL-QoE and learning performance can be improved.Experimental results verify the effectiveness oftheproposed scheme.

How Presence Affects College Students’Online Learning Outcomes:Unveiling Its Underlying Mechanism

Over the past few years,China’s higher education institutions have experienced remarkable growth in online teaching.However,it remains uncertain whether and how the sense of presence perceived by students affects their online learning outcomes when teachers use online teaching media for communication.This sense specifically pertains to the extent to which students perceive themselves as“real persons”and establish connections with others.Therefore,this study constructs a conceptual model elucidating the impact of presence on students’online learning outcomes and empirically examines the mechanism through which three types of presence influence students’online learning.The test results of the structural equation modeling(SEM)indicate that:(a)teaching presence,social presence,and cognitive presence all exhibit significantly positive outcomes on students’online learning outcomes;(b)these three types of presence can also indirectly and positively influence students’online learning outcomes through the mediating effect of flow experience and learning satisfaction;and(c)flow experience and learning satisfaction play a sequential mediating role in the process by which presence impacts students’online learning outcomes.We hope that the relevant research findings may contribute to unveiling the“black box”of the impact of presence on students’online learning outcomes and offer valuable insights for college educators to overcome online teaching constraints and enhance online teaching quality.

Sampled-data control through model-free reinforcement learning with effective experience replay

Reinforcement Learning(RL)based control algorithms can learn the control strategies for nonlinear and uncertain environment during interacting with it.Guided by the rewards generated by environment,a RL agent can learn the control strategy directly in a model-free way instead of investigating the dynamic model of the environment.In the paper,we propose the sampled-data RL control strategy to reduce the computational demand.In the sampled-data control strategy,the whole control system is of a hybrid structure,in which the plant is of continuous structure while the controller(RL agent)adopts a discrete structure.Given that the continuous states of the plant will be the input of the agent,the state–action value function is approximated by the fully connected feed-forward neural networks(FCFFNN).Instead of learning the controller at every step during the interaction with the environment,the learning and acting stages are decoupled to learn the control strategy more effectively through experience replay.In the acting stage,the most effective experience obtained during the interaction with the environment will be stored and during the learning stage,the stored experience will be replayed to customized times,which helps enhance the experience replay process.The effectiveness of proposed approach will be verified by simulation examples.

Optimal Control of Nonlinear Systems Using Experience Inference Human-Behavior Learning

Safety critical control is often trained in a simulated environment to mitigate risk.Subsequent migration of the biased controller requires further adjustments.In this paper,an experience inference human-behavior learning is proposed to solve the migration problem of optimal controllers applied to real-world nonlinear systems.The approach is inspired in the complementary properties that exhibits the hippocampus,the neocortex,and the striatum learning systems located in the brain.The hippocampus defines a physics informed reference model of the realworld nonlinear system for experience inference and the neocortex is the adaptive dynamic programming(ADP)or reinforcement learning(RL)algorithm that ensures optimal performance of the reference model.This optimal performance is inferred to the real-world nonlinear system by means of an adaptive neocortex/striatum control policy that forces the nonlinear system to behave as the reference model.Stability and convergence of the proposed approach is analyzed using Lyapunov stability theory.Simulation studies are carried out to verify the approach.

Optimization of Interactive Videos Empowered the Experience of Learning Management System

The Learning management system(LMS)is now being used for uploading educational content in both distance and blended setups.LMS platform has two types of users:the educators who upload the content,and the students who have to access the content.The students,usually rely on text notes or books and video tutorials while their exams are conducted with formal methods.Formal assessments and examination criteria are ineffective with restricted learning space which makes the student tend only to read the educational contents and videos instead of interactive mode.The aim is to design an interactive LMS and examination video-based interface to cater the issues of educators and students.It is designed according to Human-computer interaction(HCI)principles to make the interactive User interface(UI)through User experience(UX).The interactive lectures in the form of annotated videos increase user engagement and improve the self-study context of users involved in LMS.The interface design defines how the design will interact with users and how the interface exchanges information.The findings show that interactive videos for LMS allow the users to have a more personalized learning experience by engaging in the educational content.The result shows a highly personalized learning experience due to the interactive video and quiz within the video.

Research on the Influencing Factors of College Students’Learning Experience in Livestreaming Courses

The learning experience of online courses has always been a hot topic.As livestreaming courses are online courses with real-time interaction between instructors and students,its learning experience directly affects the learning behavior and effect.The evaluation indicators related to online course learning experience were combined with the attitudes of scholars,enterprise practitioners,and learners in livestreaming courses.When the questionnaire was designed,the exploratory and confirmatory factors were analyzed,the Questionnaire on Learning Experience of College Students in Livestreaming Courses was developed to evaluate the learning experience of college students attending livestreaming courses.Last but not the least,based on the survey data,important factors affecting college students’learning experience in livestreaming courses,including course content,learning environment,course interaction,and learning incentives were discussed and analyzed;strategies to optimize the learning experience in livestreaming courses were proposed.

The Practical Path to Cultivating Innovative Talents:Implementing Cross-Disciplinary Learning Communities in Environmental Design Education

In response to the national strategy of“vigorously cultivating interdisciplinary talents and actively promoting interdisciplinary integration,”this article focuses on the nationally recognized Environmental Design program at Hezhou University’s College of Design,leveraging local industry advantages to engage in interdisciplinary integration through educational practices.Using the“Construction of the Panoramic Virtual Nature Museum of the Guizhou Crocodile Lizard at Mount Dagui”as a case study,we aim to establish a professional and interdisciplinary learning community,incorporate student-centered interactive teaching methods,boost student motivation,enhance teaching quality,nurture forward-thinking versatile innovative talents,and provide a guideline for interdisciplinary educational reform.

Undergraduate nursing students’experiences of online learning:gaining access in resource-limited circumstances

Background:In response to the need to mitigate the increase in Coronavirus Disease 2019(COVID-19)cases,nursing students undertake online learning in almost all nursing education institutions in Indonesia.These students face distinctive learning experiences,which have not yet been identified in the Indonesian context.This study aimed to explore students’experiences of online learning during the COVID-19 pandemic.Methods:We used a descriptive exploratory design.Eleven students from three nursing education institutions in Indonesia were interviewed through telephone calls or video conference applications.Results:One main theme,Gaining access in resource-limited circumstances,was developed to describe students’experience of online learning during the COVID-19 pandemic.This theme was supported by five subthemes:struggling for internet connection;becoming familiar with the applications;flexibility;supported by others;and dealing with limitations.Conclusions:This current study provides insights into what support should be provided for nursing students to manage limitations in the online learning process.

Learning from International Experiences and Promoting People-to-People Diplomacy

People-to-people diplomacy refers to nongovernmental external relations by NGOs including foreign exchanges and cooperation in certain areas of expertise that could influence state foreign relations.People-to-people diplomacy has five characteristics:firstly,it is non-governmental,thus it neither represents the state or the

Active Machine Learning for Chemical Engineers:A Bright Future Lies Ahead!

By combining machine learning with the design of experiments,thereby achieving so-called active machine learning,more efficient and cheaper research can be conducted.Machine learning algorithms are more flexible and are better than traditional design of experiment algorithms at investigating processes spanning all length scales of chemical engineering.While active machine learning algorithms are maturing,their applications are falling behind.In this article,three types of challenges presented by active machine learning—namely,convincing the experimental researcher,the flexibility of data creation,and the robustness of active machine learning algorithms—are identified,and ways to overcome them are discussed.A bright future lies ahead for active machine learning in chemical engineering,thanks to increasing automation and more efficient algorithms that can drive novel discoveries.

Squeezing More Past Knowledge for Online Class-Incremental Continual Learning

Continual learning(CL)studies the problem of learning to accumulate knowledge over time from a stream of data.A crucial challenge is that neural networks suffer from performance degradation on previously seen data,known as catastrophic forgetting,due to allowing parameter sharing.In this work,we consider a more practical online class-incremental CL setting,where the model learns new samples in an online manner and may continuously experience new classes.Moreover,prior knowledge is unavailable during training and evaluation.Existing works usually explore sample usages from a single dimension,which ignores a lot of valuable supervisory information.To better tackle the setting,we propose a novel replay-based CL method,which leverages multi-level representations produced by the intermediate process of training samples for replay and strengthens supervision to consolidate previous knowledge.Specifically,besides the previous raw samples,we store the corresponding logits and features in the memory.Furthermore,to imitate the prediction of the past model,we construct extra constraints by leveraging multi-level information stored in the memory.With the same number of samples for replay,our method can use more past knowledge to prevent interference.We conduct extensive evaluations on several popular CL datasets,and experiments show that our method consistently outperforms state-of-the-art methods with various sizes of episodic memory.We further provide a detailed analysis of these results and demonstrate that our method is more viable in practical scenarios.

A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

In this paper,a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems.Motivated by the classical on-policy and off-policy algorithms of reinforcement learning,the online feedback relearning(FR)algorithm is developed where the collected data includes the influence of disturbance signals.The FR algorithm has better adaptability to environmental changes(such as the control channel disturbances)compared with the off-policy algorithm,and has higher computational efficiency and better convergence performance compared with the on-policy algorithm.Data processing based on experience replay technology is used for great data efficiency and convergence stability.Simulation experiments are presented to illustrate convergence stability,optimality and algorithmic performance of FR algorithm by comparison.

Collaborative multi-agent reinforcement learning based on experience propagation

For multi-agent reinforcement learning in Markov games, knowledge extraction and sharing are key research problems. State list extracting means to calculate the optimal shared state path from state trajectories with cycles. A state list extracting algorithm checks cyclic state lists of a current state in the state trajectory, condensing the optimal action set of the current state. By reinforcing the optimal action selected, the action policy of cyclic states is optimized gradually. The state list extracting is repeatedly learned and used as the experience knowledge which is shared by teams. Agents speed up the rate of convergence by experience sharing. Competition games of preys and predators are used for the experiments. The results of experiments prove that the proposed algorithms overcome the lack of experience in the initial stage, speed up learning and improve the performance.

Auxiliary guidance manufacture and revealing potential mechanism of perovskite solar cell using machine learning

To promote the development of global carbon neutrality,perovskite solar cells(PSCs)have become a research hotspot in related fields.How to obtain PSCs with expected performance and explore the potential factors affecting device performance are the research priorities in related fields.Although some classical computational methods can facilitate material development,they typically require complex mathematical approximations and manual feature screening processes,which have certain subjectivity and one-sidedness,limiting the performance of the model.In order to alleviate the above challenges,this paper proposes a machine learning(ML)model based on neural networks.The model can assist both PSCs design and analysis of their potential mechanism,demonstrating enhanced and comprehensive auxiliary capabilities.To make the model have higher feasibility and fit the real experimental process more closely,this paper collects the corresponding real experimental data from numerous research papers to develop the model.Compared with other classical ML methods,the proposed model achieved better overall performance.Regarding analysis of underlying mechanism,the relevant laws explored by the model are consistent with the actual experiment results of existing articles.The model exhibits great potential to discover complex laws that are difficult for humans to discover directly.In addition,we also fabricated PSCs to verify the guidance ability of the model in this paper for real experiments.Eventually,the model achieved acceptable results.This work provides new insights into integrating ML methods and PSC design techniques,as well as bridging photovoltaic power generation technology and other fields.

Affective Variables and Learning Experience in SLA

The study aims at finding out the factors that have impact on a second language learner improving his language proficiency. To achieve this purpose, a research was conducted at an American University. A male subject was chosen from the ESL center in the English department of the University. The author followed up his life and study during the whole semester and assessed his improvement in English learning. The results showed that motivation and prior learning experience played vital roles in the subject's improvement of English proficiency.