Computational intelligence interception guidance law using online off-policy integral reinforcement learning

Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.

On the Integrated Learning of English and Law

This paper centers on the integrated learning of English and law in China.Firstly,it outlines the importance of English in the solution of the ever increasing legal disputes between China and the outside world,which inevitably involves an integrated learning of English and law.Secondly,it points out that the content of legal English reflects a combination of legal knowledge and English skills.Thirdly,it expounds on the difficulties that Chinese English majors are facing in the process of learning English and law simultaneously and furnishes some practical suggestions.

Prediction of single cell mechanical properties in microchannels based on deep learning

Traditional methods for measuring single-cell mechanical characteristics face several challenges,including lengthy measurement times,low throughput,and a requirement for advanced technical skills.To overcome these challenges,a novel machine learning(ML)approach is implemented based on the convolutional neural networks(CNNs),aiming at predicting cells'elastic modulus and constitutive equations from their deformations while passing through micro-constriction channels.In the present study,the computational fluid dynamics technology is used to generate a dataset within the range of the cell elastic modulus,incorporating three widely-used constitutive models that characterize the cellular mechanical behavior,i.e.,the Mooney-Rivlin(M-R),Neo-Hookean(N-H),and Kelvin-Voigt(K-V)models.Utilizing this dataset,a multi-input convolutional neural network(MI-CNN)algorithm is developed by incorporating cellular deformation data as well as the time and positional information.This approach accurately predicts the cell elastic modulus,with a coefficient of determination R^(2)of 0.999,a root mean square error of 0.218,and a mean absolute percentage error of 1.089%.The model consistently achieves high-precision predictions of the cellular elastic modulus with a maximum R^(2)of 0.99,even when the stochastic noise is added to the simulated data.One significant feature of the present model is that it has the ability to effectively classify the three types of constitutive equations we applied.The model accurately and reliably predicts single-cell mechanical properties,showcasing a robust ability to generalize.We demonstrate that incorporating deformation features at multiple time points can enhance the algorithm's accuracy and generalization.This algorithm presents a possibility for high-throughput,highly automated,real-time,and precise characterization of single-cell mechanical properties.

Q-learning强化学习制导律

在未来的战场中,智能导弹将成为精确有效的打击武器,导弹智能化已成为一种主要的发展趋势。本文以传统的比例制导律为基础,提出基于强化学习的变比例系数制导算法。该算法以视线转率作为状态,依据脱靶量设计奖励函数,并设计离散化的行为空间,为导弹选择正确的制导指令。实验仿真验证了所提算法比传统的比例制导律拥有更好的制导精度,并使导弹拥有了自主决策能力。

Iterative Learning Control for homing guidance design of missiles

This paper presents an Iterative Learning Control design applied to homing guidance of missiles against maneuvering targets. According to numerical experiments, although an increase of the control energies is appreciated with respect to a previous published base controller for comparison, this strategy, which is simple to realize, is able to reduce the time to reach the head-on condition to target destruction. This fact is important to minimize the missile lateral force-level to fulfill engaging in hyper-sonic target persecutions.

An Exploration on Adaptive Iterative Learning Control for a Class of Commensurate High-order Uncertain Nonlinear Fractional Order Systems

This paper explores the adaptive iterative learning control method in the control of fractional order systems for the first time. An adaptive iterative learning control(AILC) scheme is presented for a class of commensurate high-order uncertain nonlinear fractional order systems in the presence of disturbance.To facilitate the controller design, a sliding mode surface of tracking errors is designed by using sufficient conditions of linear fractional order systems. To relax the assumption of the identical initial condition in iterative learning control(ILC), a new boundary layer function is proposed by employing MittagLeffler function. The uncertainty in the system is compensated for by utilizing radial basis function neural network. Fractional order differential type updating laws and difference type learning law are designed to estimate unknown constant parameters and time-varying parameter, respectively. The hyperbolic tangent function and a convergent series sequence are used to design robust control term for neural network approximation error and bounded disturbance, simultaneously guaranteeing the learning convergence along iteration. The system output is proved to converge to a small neighborhood of the desired trajectory by constructing Lyapnov-like composite energy function(CEF)containing new integral type Lyapunov function, while keeping all the closed-loop signals bounded. Finally, a simulation example is presented to verify the effectiveness of the proposed approach.

Machine learning-based constitutive models for cement-grouted coal specimens under shearing

Cement-based grouting has been widely used in mining engineering;its constitutive law has not been comprehensively studied.In this study,a novel constitutive law of cement-grouted coal specimens(CGCS)was developed using hybrid machine learning(ML)algorithms.Shear tests were performed on CGCS for the analysis of stress-strain curves and the preparation of the dataset.To maintain the interpretation of the trained ML models,regression tree(RT)was used as the main technique.The effect of maximum RT depth(Maxdepth)on its performance was studied,and the hyperparameters of RT were tuned using the genetic algorithm(GA).The RT performance was also compared with ensemble learning techniques.The optimum correlation coefficient on the training set was determined as 0.835,0.946,0.981,and 0.985 for RT models with Maxdepth=3,5,7,and 9,respectively.The overall correlation coefficient was over 0.9 when the Maxdepth≥5,indicating that the constitutive law of CGCS can be well described.However,the failure type of CGCS could not be captured using the trained RT models.Random forest was found to be the optimum algorithm for the constitutive modeling of CGCS,while RT with the Maxdepth=3 performed the worst.

Q-learning强化学习协同拦截制导律

为实现多枚导弹协同拦截机动目标,提升拦截效能,提出了一种Q-learning强化学习协同拦截制导律。首先,基于逃逸域覆盖理论,建立了非线性多弹协同拦截模型。其次,以视线角速率为状态,依据脱靶量构造奖励函数,通过离线训练生成强化学习智能体,并结合传统比例制导控制方法,构建基于强化学习的变导引系数制导律,实时生成实现协同拦截的制导指令。最终,通过数值仿真验证了所提算法的有效性和优越性。

L_2-L_∞ learning of dynamic neural networks

This paper proposes an y2-y∞ learning law as a new learning method for dynamic neural networks with external disturbance. Based on linear matrix inequality （LMI） formulation, the y2-y∞ learning law is presented to not only guarantee asymptotical stability of dynamic neural networks but also reduce the effect of external disturbance to an y2-y∞ induced norm constraint. It is shown that the design of the y2-y∞ learning law for such neural networks can be achieved by solving LMIs, which can be easily facilitated by using some standard numerical packages. A numerical example is presented to demonstrate the validity of the proposed learning law.

Hierarchical reinforcement learning guidance with threat avoidance

The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchical deep deterministic policy gradient(DDPG)algorithm.The reward functions are constructed to minimize the line-of-sight(LOS)angle rate and avoid the threat caused by the opposed obstacles.To attenuate the chattering of the acceleration,a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward.The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.

Learning control of nonhonolomic robot based on support vector machine

A learning controller of nonhonolomic robot in real-time based on support vector machine(SVM)is presented.The controller includes two parts:one is kinematic controller based on nonlinear law,and the other is dynamic controller based on SVM.The kinematic controller is aimed to provide desired velocity which can make the steering system stable.The dynamic controller is aimed to transform the desired velocity to control torque.The parameters of the dynamic system of the robot are estimated through SVM learning algorithm according to the training data of sliding windows in real time.The proposed controller can adapt to the changes in the robot model and uncertainties in the environment.Compared with artificial neural network(ANN)controller,SVM controller can converge to the reference trajectory more quickly and the tracking error is smaller.The simulation results verify the effectiveness of the method proposed.

Clickers and Deep Learning： An Oxymoron？

The use of clicker technology as a tool for promoting learning, let along deep learning, is hotly debated and has its proponents and opponents. With the question still open, this paper examines the use of this technology in fostering critical thinking and other higher-order learning and team-building skills, in the context of two entry-level psychology courses. The results obtained indicate that from the perspectives of both the student and the instructor, the integration of this technology was positively perceived to enhance the learning process and the acquisition of these skills. It would seem these concepts are, after all, not antithetical.

Learning the continuous-time optimal decision law from discrete-time rewards

The concept of reward is fundamental in reinforcement learning with a wide range of applications in natural and social sciences.Seeking an interpretable reward for decision-making that largely shapes the system's behavior has always been a challenge in reinforcement learning.In this work,we explore a discrete-time reward for reinforcement learning in continuous time and action spaces that represent many phenomena captured by applying physical laws.We find that the discrete-time reward leads to the extraction of the unique continuous-time decision law and improved computational efficiency by dropping the integrator operator that appears in classical results with integral rewards.We apply this finding to solve output-feedback design problems in power systems.The results reveal that our approach removes an intermediate stage of identifying dynamical models.Our work suggests that the discrete-time reward is efficient in search of the desired decision law,which provides a computational tool to understand and modify the behavior of large-scale engineering systems using the optimal learned decision.

On the Right to Learn as a Fundamental Human Right

With the advent of the learning age,the connotation of the“right to education”can no longer meet the requirements of personal and social development,so it is necessary to introduce the concept of the right to learn with a richer connotation.As a basic human right,the proposal and guarantee of the right to learn directly respond to the practical problems in the information age and the learning age.The right to learn is fundamental freedom enjoyed by everyone to acquire useful knowledge,skill,value,spirit,and attitude through reading,watching,listening,thinking,researching,practicing,being educated,and other learning methods to develop and perfect a personality.The philosophical basis,connotation,attribute,extraterritorial legislative guarantee of the right to learn and the development of the times de termine it should be regarded as a basic human right.Treating the right to learn as a basic human right and systematically guaranteeing it through the amendment of China’s Education Law and other supporting legislation will contribute to the realization of this right,the improvement of national knowledge,and the progress of China’s social civilization.

意蕴·藩篱·突破:数字法治时代视角下面向未来的复合型法治人才培养

在数字法治建设背景下,复合型法治人才培养是法学教育的核心使命。如何进一步创新与改革传统法治人才培养,培养面向数字法治的复合型法治人才,更好推进中国式法治现代化的实现,是当代法学教育和法治人才培养必须思考和回应的重大课题。从数字法治时代视角出发,以高等院校法学教育为研究主体,以“意蕴、藩篱、突破”思路探究面向数字法治时代的复合型法治人才培养路径,为新时代复合型法治人才培养提供有益参考。

体育学法概念辨析——兼与《体育学法论》作者商榷

文章通过对《体育学法论》的分析,明确体育学法概念。《体育学法论》把“体育学法”等同于“体育学习方法”,甚至是体育课堂上学生个别的行为表现,与书中教学生“学会学习”初衷相悖,由于体育学习方法界限具象且狭窄,造成《体育学法论》研究内卷化,教法和学法互为因果,真正的体育学法概念被遮蔽。把体育学法定义为学习体育知识与技能的规律与法则,可以区别体育学法与体育学习方法、学习行为、学习能力等相关概念。文章在学校体育教学中,依从全面深化课程改革要求,尤其是在基础教育阶段,把教学重心从项目教学转移到学法教学,符合发展学生核心素养的现代教学理念,为提升学生适应终身发展和社会发展需要的关键能力赋能。

基于分层强化学习的低过载比拦截制导律

为解决低过载比和纯角度量测等约束下的三维机动目标拦截制导问题,提出了一种基于分层强化学习的拦截制导律。首先将问题建模为马尔科夫决策过程模型,并考虑拦截能量消耗与弹目视线角速率,设计了一种启发式奖赏函数。其次通过构建具有双层结构的策略网络,并利用上层策略规划阶段性子目标来指导下层策略生成所需的制导指令,实现了拦截交战过程中的视线角速率收敛,以保证能成功拦截机动目标。仿真结果验证了所提出的方法较增强比例导引具有更高的拦截精度和拦截概率,且拦截过程的需用过载更低。

宋代民众的诉讼与官府的审理——读《名公书判清明集》的几点认识

“好讼”是宋代比较普遍的民情,这一现象在《名公书判清明集》中有具体印证。凭借大量的案例,再联系历史实情,可知“好讼”源于官吏贪腐、豪强奸恶,也是民众学法懂法、有文明维权意识的表现。江西农耕经济旺盛,耕地紧俏,因而争夺田产所有权的讼案很多。审理纠纷时因官员品德才干差异,有的贪财,甚至“自紊其法”,遂判决不公正。诸多关于教化与刑罚并举的判词,启发我们加深对《白鹿洞书院揭示》的理解,朱熹强调圣贤教人之道,也告诫生徒若不听教育,必将实施“学规”,予以惩罚。《名公书判清明集》是研究断代史、社会史、法制史等的珍贵资料,至今不失其借鉴警示意义。

“全民终身学习”视域下课程建设与教学改革研究——以“海运业务与海商法”为例

继续教育作为构建服务全民终身学习教育体系的重要内容,其课程的数字化改革、建设应走在教育数字化转型的前列。通过分析高校继续教育课程实施数字化改革的意义,以航运类专业继续教育课程“海运业务与海商法”为例,从基于“海洋强国”战略开展课程思政资源建设;建设完善课程数字化资源,服务“全民终身学习”;构建基于BOPPPS的线上线下“混合式教学”;实施基于大数据全程信息采集分析的考核评价办法四个方面探讨了课程的建设与教学改革,以期更好地服务全民终身学习,并为其他课程的建设提供借鉴。

神经网络架构轻量化搜索的飞行器控制律自学习方法

针对在运用Soft actor-critic(SAC)强化学习算法实现复杂的飞行器控制律自学习过程中,超参数设定高度依赖于人工经验进而造成设计难度大的问题,提出一种基于神经网络架构轻量化搜索策略的飞行器控制律自学习方法。该方法在将神经网络架构设计问题转化为图拓扑生成问题的基础上,结合LSTM循环神经网络的图拓扑生成算法、基于权重共享的深度强化学习参数轻量化训练与评估机制,以及基于策略梯度的图拓扑生成器参数学习算法,给出了一种面向深度强化学习的轻量化自动搜索框架,实现了SAC训练算法中神经网络架构超参数的自动优化,进而完成了控制律的自学习。以三维空间返回着陆控制为例,验证了所提方法的有效性和实用性。