Computational intelligence interception guidance law using online off-policy integral reinforcement learning

Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.

On the Integrated Learning of English and Law

This paper centers on the integrated learning of English and law in China.Firstly,it outlines the importance of English in the solution of the ever increasing legal disputes between China and the outside world,which inevitably involves an integrated learning of English and law.Secondly,it points out that the content of legal English reflects a combination of legal knowledge and English skills.Thirdly,it expounds on the difficulties that Chinese English majors are facing in the process of learning English and law simultaneously and furnishes some practical suggestions.

Conservation Laws of the Differential-Difference KP Equation

An infinite number of semi-discrete and continuous conservation laws for the differential-difference KP equation were obtained by using a solvable generalized Riccati equation.

Q-learning强化学习制导律

在未来的战场中,智能导弹将成为精确有效的打击武器,导弹智能化已成为一种主要的发展趋势。本文以传统的比例制导律为基础,提出基于强化学习的变比例系数制导算法。该算法以视线转率作为状态,依据脱靶量设计奖励函数,并设计离散化的行为空间,为导弹选择正确的制导指令。实验仿真验证了所提算法比传统的比例制导律拥有更好的制导精度,并使导弹拥有了自主决策能力。

Learning the continuous-time optimal decision law from discrete-time rewards

The concept of reward is fundamental in reinforcement learning with a wide range of applications in natural and social sciences.Seeking an interpretable reward for decision-making that largely shapes the system's behavior has always been a challenge in reinforcement learning.In this work,we explore a discrete-time reward for reinforcement learning in continuous time and action spaces that represent many phenomena captured by applying physical laws.We find that the discrete-time reward leads to the extraction of the unique continuous-time decision law and improved computational efficiency by dropping the integrator operator that appears in classical results with integral rewards.We apply this finding to solve output-feedback design problems in power systems.The results reveal that our approach removes an intermediate stage of identifying dynamical models.Our work suggests that the discrete-time reward is efficient in search of the desired decision law,which provides a computational tool to understand and modify the behavior of large-scale engineering systems using the optimal learned decision.

Iterative Learning Control for homing guidance design of missiles

This paper presents an Iterative Learning Control design applied to homing guidance of missiles against maneuvering targets. According to numerical experiments, although an increase of the control energies is appreciated with respect to a previous published base controller for comparison, this strategy, which is simple to realize, is able to reduce the time to reach the head-on condition to target destruction. This fact is important to minimize the missile lateral force-level to fulfill engaging in hyper-sonic target persecutions.

An Exploration on Adaptive Iterative Learning Control for a Class of Commensurate High-order Uncertain Nonlinear Fractional Order Systems

This paper explores the adaptive iterative learning control method in the control of fractional order systems for the first time. An adaptive iterative learning control(AILC) scheme is presented for a class of commensurate high-order uncertain nonlinear fractional order systems in the presence of disturbance.To facilitate the controller design, a sliding mode surface of tracking errors is designed by using sufficient conditions of linear fractional order systems. To relax the assumption of the identical initial condition in iterative learning control(ILC), a new boundary layer function is proposed by employing MittagLeffler function. The uncertainty in the system is compensated for by utilizing radial basis function neural network. Fractional order differential type updating laws and difference type learning law are designed to estimate unknown constant parameters and time-varying parameter, respectively. The hyperbolic tangent function and a convergent series sequence are used to design robust control term for neural network approximation error and bounded disturbance, simultaneously guaranteeing the learning convergence along iteration. The system output is proved to converge to a small neighborhood of the desired trajectory by constructing Lyapnov-like composite energy function(CEF)containing new integral type Lyapunov function, while keeping all the closed-loop signals bounded. Finally, a simulation example is presented to verify the effectiveness of the proposed approach.

Machine learning-based constitutive models for cement-grouted coal specimens under shearing

Cement-based grouting has been widely used in mining engineering;its constitutive law has not been comprehensively studied.In this study,a novel constitutive law of cement-grouted coal specimens(CGCS)was developed using hybrid machine learning(ML)algorithms.Shear tests were performed on CGCS for the analysis of stress-strain curves and the preparation of the dataset.To maintain the interpretation of the trained ML models,regression tree(RT)was used as the main technique.The effect of maximum RT depth(Maxdepth)on its performance was studied,and the hyperparameters of RT were tuned using the genetic algorithm(GA).The RT performance was also compared with ensemble learning techniques.The optimum correlation coefficient on the training set was determined as 0.835,0.946,0.981,and 0.985 for RT models with Maxdepth=3,5,7,and 9,respectively.The overall correlation coefficient was over 0.9 when the Maxdepth≥5,indicating that the constitutive law of CGCS can be well described.However,the failure type of CGCS could not be captured using the trained RT models.Random forest was found to be the optimum algorithm for the constitutive modeling of CGCS,while RT with the Maxdepth=3 performed the worst.

Q-learning强化学习协同拦截制导律

为实现多枚导弹协同拦截机动目标,提升拦截效能,提出了一种Q-learning强化学习协同拦截制导律。首先,基于逃逸域覆盖理论,建立了非线性多弹协同拦截模型。其次,以视线角速率为状态,依据脱靶量构造奖励函数,通过离线训练生成强化学习智能体,并结合传统比例制导控制方法,构建基于强化学习的变导引系数制导律,实时生成实现协同拦截的制导指令。最终,通过数值仿真验证了所提算法的有效性和优越性。

L_2-L_∞ learning of dynamic neural networks

This paper proposes an y2-y∞ learning law as a new learning method for dynamic neural networks with external disturbance. Based on linear matrix inequality （LMI） formulation, the y2-y∞ learning law is presented to not only guarantee asymptotical stability of dynamic neural networks but also reduce the effect of external disturbance to an y2-y∞ induced norm constraint. It is shown that the design of the y2-y∞ learning law for such neural networks can be achieved by solving LMIs, which can be easily facilitated by using some standard numerical packages. A numerical example is presented to demonstrate the validity of the proposed learning law.

Hierarchical reinforcement learning guidance with threat avoidance

The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchical deep deterministic policy gradient(DDPG)algorithm.The reward functions are constructed to minimize the line-of-sight(LOS)angle rate and avoid the threat caused by the opposed obstacles.To attenuate the chattering of the acceleration,a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward.The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.

Learning control of nonhonolomic robot based on support vector machine

A learning controller of nonhonolomic robot in real-time based on support vector machine(SVM)is presented.The controller includes two parts:one is kinematic controller based on nonlinear law,and the other is dynamic controller based on SVM.The kinematic controller is aimed to provide desired velocity which can make the steering system stable.The dynamic controller is aimed to transform the desired velocity to control torque.The parameters of the dynamic system of the robot are estimated through SVM learning algorithm according to the training data of sliding windows in real time.The proposed controller can adapt to the changes in the robot model and uncertainties in the environment.Compared with artificial neural network(ANN)controller,SVM controller can converge to the reference trajectory more quickly and the tracking error is smaller.The simulation results verify the effectiveness of the method proposed.

意蕴·藩篱·突破:数字法治时代视角下面向未来的复合型法治人才培养

在数字法治建设背景下,复合型法治人才培养是法学教育的核心使命。如何进一步创新与改革传统法治人才培养,培养面向数字法治的复合型法治人才,更好推进中国式法治现代化的实现,是当代法学教育和法治人才培养必须思考和回应的重大课题。从数字法治时代视角出发,以高等院校法学教育为研究主体,以“意蕴、藩篱、突破”思路探究面向数字法治时代的复合型法治人才培养路径,为新时代复合型法治人才培养提供有益参考。

体育学法概念辨析——兼与《体育学法论》作者商榷

文章通过对《体育学法论》的分析,明确体育学法概念。《体育学法论》把“体育学法”等同于“体育学习方法”,甚至是体育课堂上学生个别的行为表现,与书中教学生“学会学习”初衷相悖,由于体育学习方法界限具象且狭窄,造成《体育学法论》研究内卷化,教法和学法互为因果,真正的体育学法概念被遮蔽。把体育学法定义为学习体育知识与技能的规律与法则,可以区别体育学法与体育学习方法、学习行为、学习能力等相关概念。文章在学校体育教学中,依从全面深化课程改革要求,尤其是在基础教育阶段,把教学重心从项目教学转移到学法教学,符合发展学生核心素养的现代教学理念,为提升学生适应终身发展和社会发展需要的关键能力赋能。

基于分层强化学习的低过载比拦截制导律

为解决低过载比和纯角度量测等约束下的三维机动目标拦截制导问题,提出了一种基于分层强化学习的拦截制导律。首先将问题建模为马尔科夫决策过程模型,并考虑拦截能量消耗与弹目视线角速率,设计了一种启发式奖赏函数。其次通过构建具有双层结构的策略网络,并利用上层策略规划阶段性子目标来指导下层策略生成所需的制导指令,实现了拦截交战过程中的视线角速率收敛,以保证能成功拦截机动目标。仿真结果验证了所提出的方法较增强比例导引具有更高的拦截精度和拦截概率,且拦截过程的需用过载更低。

宋代民众的诉讼与官府的审理——读《名公书判清明集》的几点认识

“好讼”是宋代比较普遍的民情,这一现象在《名公书判清明集》中有具体印证。凭借大量的案例,再联系历史实情,可知“好讼”源于官吏贪腐、豪强奸恶,也是民众学法懂法、有文明维权意识的表现。江西农耕经济旺盛,耕地紧俏,因而争夺田产所有权的讼案很多。审理纠纷时因官员品德才干差异,有的贪财,甚至“自紊其法”,遂判决不公正。诸多关于教化与刑罚并举的判词,启发我们加深对《白鹿洞书院揭示》的理解,朱熹强调圣贤教人之道,也告诫生徒若不听教育,必将实施“学规”,予以惩罚。《名公书判清明集》是研究断代史、社会史、法制史等的珍贵资料,至今不失其借鉴警示意义。

“全民终身学习”视域下课程建设与教学改革研究——以“海运业务与海商法”为例

继续教育作为构建服务全民终身学习教育体系的重要内容,其课程的数字化改革、建设应走在教育数字化转型的前列。通过分析高校继续教育课程实施数字化改革的意义,以航运类专业继续教育课程“海运业务与海商法”为例,从基于“海洋强国”战略开展课程思政资源建设;建设完善课程数字化资源,服务“全民终身学习”;构建基于BOPPPS的线上线下“混合式教学”;实施基于大数据全程信息采集分析的考核评价办法四个方面探讨了课程的建设与教学改革,以期更好地服务全民终身学习,并为其他课程的建设提供借鉴。

神经网络架构轻量化搜索的飞行器控制律自学习方法

针对在运用Soft actor-critic(SAC)强化学习算法实现复杂的飞行器控制律自学习过程中,超参数设定高度依赖于人工经验进而造成设计难度大的问题,提出一种基于神经网络架构轻量化搜索策略的飞行器控制律自学习方法。该方法在将神经网络架构设计问题转化为图拓扑生成问题的基础上,结合LSTM循环神经网络的图拓扑生成算法、基于权重共享的深度强化学习参数轻量化训练与评估机制,以及基于策略梯度的图拓扑生成器参数学习算法,给出了一种面向深度强化学习的轻量化自动搜索框架,实现了SAC训练算法中神经网络架构超参数的自动优化,进而完成了控制律的自学习。以三维空间返回着陆控制为例,验证了所提方法的有效性和实用性。

基于项目式学习的高中物理实验教学——以“自制响箭”为例

项目式学习是一种全新的教学模式,是全面促进中学生核心素养发展的有效途径和手段,能够体现基础教育改革所倡导的“做中学”课程理念.在本研究中,项目式学习包含“主题选定、教学分析、项目实施、项目成果展示与评价”4个阶段,以此为基础在高中物理必修二“机械能守恒定律”一节中设计自制响箭项目.自制响箭项目不仅能够促进学生建构知识,还能够增强学生模型建构和科学推理能力,并能够弘扬中华优秀传统文化,使学生体会到物理学的生活性和文化内涵,从而有效提高物理教学质量.

基于多任务学习的民事案件判决预测方法

针对民事案件预测中法律法规组合多样化的问题,提出一种基于多任务学习的民事案件判决预测方法。该方法采用多CNN融合、阈值设定等多种策略,利用案由和法条之间的依赖关系,实现民事案件的案由纠纷和相关法律法规的联合判决预测。基于中国裁判文书公开网的民事案件,构造了10万篇民事文书进行判决预测实验。实验结果表明,相比于传统的预测模型,该方法针对有依赖关系的预测任务更加合理和有效。