Recorded recurrent deep reinforcement learning guidance laws for intercepting endoatmospheric maneuvering missiles

This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with uncertainties and observation noise.The attack-defense engagement scenario is modeled as a partially observable Markov decision process(POMDP).Given the benefits of recurrent neural networks(RNNs)in processing sequence information,an RNN layer is incorporated into the agent’s policy network to alleviate the bottleneck of traditional deep reinforcement learning methods while dealing with POMDPs.The measurements from the interceptor’s seeker during each guidance cycle are combined into one sequence as the input to the policy network since the detection frequency of an interceptor is usually higher than its guidance frequency.During training,the hidden states of the RNN layer in the policy network are recorded to overcome the partially observable problem that this RNN layer causes inside the agent.The training curves show that the proposed RRTD3 successfully enhances data efficiency,training speed,and training stability.The test results confirm the advantages of the RRTD3-based guidance laws over some conventional guidance laws.

Computational intelligence interception guidance law using online off-policy integral reinforcement learning

Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.

Six-Dimensional Guidance: The Strategies of Thinking Quality Cultivation in Senior High School English Discourse Learning

Taking the discourse learning of the new senior high school English textbook published by the People’s Education Press as an example,combined with the“six-dimensional guidance”deep reading strategy,and through the six-skill training strategies of“memory skill training,understanding skill training,application skill training,analytical skill training,evaluation skill training,creative skill training,”this paper aims to cultivate students’thinking profundity,logic,flexibility,sensitivity,criticality,and originality.It also promotes the real implementation of senior high school English deep reading that points to the cultivation of thinking quality in classroom teaching,and realizes the transformation from“conventional reading”to“deep reading”that reflects the core literacy of the discipline.

Iterative Learning Control for homing guidance design of missiles

This paper presents an Iterative Learning Control design applied to homing guidance of missiles against maneuvering targets. According to numerical experiments, although an increase of the control energies is appreciated with respect to a previous published base controller for comparison, this strategy, which is simple to realize, is able to reduce the time to reach the head-on condition to target destruction. This fact is important to minimize the missile lateral force-level to fulfill engaging in hyper-sonic target persecutions.

Hierarchical reinforcement learning guidance with threat avoidance

The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchical deep deterministic policy gradient(DDPG)algorithm.The reward functions are constructed to minimize the line-of-sight(LOS)angle rate and avoid the threat caused by the opposed obstacles.To attenuate the chattering of the acceleration,a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward.The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.

Impact point prediction guidance of ballistic missile in high maneuver penetration condition

An impact point prediction(IPP) guidance based on supervised learning is proposed to address the problem of precise guidance for the ballistic missile in high maneuver penetration condition.An accurate ballistic trajectory model is applied to generate training samples,and ablation experiments are conducted to determine the mapping relationship between the flight state and the impact point.At the same time,the impact point coordinates are decoupled to improve the prediction accuracy,and the sigmoid activation function is improved to ameliorate the prediction efficiency.Therefore,an IPP neural network model,which solves the contradiction between the accuracy and the speed of the IPP,is established.In view of the performance deviation of the divert control system,the mapping relationship between the guidance parameters and the impact deviation is analysed based on the variational principle.In addition,a fast iterative model of guidance parameters is designed for reference to the Newton iteration method,which solves the nonlinear strong coupling problem of the guidance parameter solution.Monte Carlo simulation results show that the prediction accuracy of the impact point is high,with a 3 σ prediction error of 4.5 m,and the guidance method is robust,with a 3 σ error of 7.5 m.On the STM32F407 singlechip microcomputer,a single IPP takes about 2.374 ms,and a single guidance solution takes about9.936 ms,which has a good real-time performance and a certain engineering application value.

IoV and Blockchain-Enabled Driving Guidance Strategy in Complex Traffic Environment

Diversified traffic participants and complex traffic environment(e.g.,roadblocks or road damage exist)challenge the decision-making accuracy of a single connected and autonomous vehicle(CAV)due to its limited sensing and computing capabilities.Using Internet of Vehicles(IoV)to share driving rules between CAVs can break limitations of a single CAV,but at the same time may cause privacy and safety issues.To tackle this problem,this paper proposes to combine IoV and blockchain technologies to form an efficient and accurate autonomous guidance strategy.Specifically,we first use reinforcement learning for driving decision learning,and give the corresponding driving rule extraction method.Then,an architecture combining IoV and blockchain is designed to ensure secure driving rule sharing.Finally,the shared rules will form an effective autonomous driving guidance strategy through driving rules selection and action selection.Extensive simulation proves that the proposed strategy performs well in complex traffic environment,mainly in terms of accuracy,safety,and robustness.

Identification of a Printed Anti-Counterfeiting Code Based on Feature Guidance Double Pool Attention Networks

The authenticity identification of anti-counterfeiting codes based on mobile phone platforms is affected by lighting environment,photographing habits,camera resolution and other factors,resulting in poor collection quality of anti-counterfeiting codes and weak differentiation of anti-counterfeiting codes for high-quality counterfeits.Developing an anticounterfeiting code authentication algorithm based on mobile phones is of great commercial value.Although the existing algorithms developed based on special equipment can effectively identify forged anti-counterfeiting codes,the anti-counterfeiting code identification scheme based on mobile phones is still in its infancy.To address the small differences in texture features,low response speed and excessively large deep learning models used in mobile phone anti-counterfeiting and identification scenarios,we propose a feature-guided double pool attention network(FG-DPANet)to solve the reprinting forgery problem of printing anti-counterfeiting codes.To address the slight differences in texture features in high-quality reprinted anti-counterfeiting codes,we propose a feature guidance algorithm that creatively combines the texture features and the inherent noise feature of the scanner and printer introduced in the reprinting process to identify anti-counterfeiting code authenticity.The introduction of noise features effectively makes up for the small texture difference of high-quality anti-counterfeiting codes.The double pool attention network(DPANet)is a lightweight double pool attention residual network.Under the condition of ensuring detection accuracy,DPANet can simplify the network structure as much as possible,improve the network reasoning speed,and run better on mobile devices with low computing power.We conducted a series of experiments to evaluate the FG-DPANet proposed in this paper.Experimental results show that the proposed FG-DPANet can resist highquality and small-size anti-counterfeiting code reprint forgery.By comparing with the existing algorithm based on texture,it is shown that the proposed method has a higher authentication accuracy.Last but not least,the proposed scheme has been evaluated in the anti-counterfeiting code blurring scene,and the results show that our proposed method can well resist slight blurring of anti-counterfeiting images.

Q-learning强化学习制导律

在未来的战场中,智能导弹将成为精确有效的打击武器,导弹智能化已成为一种主要的发展趋势。本文以传统的比例制导律为基础,提出基于强化学习的变比例系数制导算法。该算法以视线转率作为状态,依据脱靶量设计奖励函数,并设计离散化的行为空间,为导弹选择正确的制导指令。实验仿真验证了所提算法比传统的比例制导律拥有更好的制导精度,并使导弹拥有了自主决策能力。

E-learning对高校就业指导教师的挑战与对策

E-learning是一种全新的教育和学习模式,本文从e-learning的内涵和特点入手,分析e-learning在教学环境、知识结构、价值取向、教学方法和信息技能等方面给高校就业指导教师带来的挑战,并对高校就业指导师资培养的对策做了一些探讨。

Q-learning强化学习协同拦截制导律

为实现多枚导弹协同拦截机动目标,提升拦截效能,提出了一种Q-learning强化学习协同拦截制导律。首先,基于逃逸域覆盖理论,建立了非线性多弹协同拦截模型。其次,以视线角速率为状态,依据脱靶量构造奖励函数,通过离线训练生成强化学习智能体,并结合传统比例制导控制方法,构建基于强化学习的变导引系数制导律,实时生成实现协同拦截的制导指令。最终,通过数值仿真验证了所提算法的有效性和优越性。

Teaching the User By Learning From the User:Personalizing Movement Control in Physical Human-robot Interaction

This paper proposes a novel approach for physical human-robot interactions(pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior of each user in coping with different tasks, where lower performance results in higher intervention from the robot. This personalized physical human-robot interaction(p2HRI) method incorporates adaptive modeling of the interaction between the human and the robot as well as learning from demonstration(LfD) techniques to adapt to the users' performance. This approach is based on model predictive control where the system optimizes the rendered forces by predicting the performance of the user. Moreover, continuous learning of the user behavior is added so that the models and personalized considerations are updated based on the change of user performance over time. Applying this framework to a field such as haptic guidance for skill improvement, allows a more personalized learning experience where the interaction between the robot as the intelligent tutor and the student as the user,is better adjusted based on the skill level of the individual and their gradual improvement. The results suggest that the precision of the model of the interaction is improved using this proposed method,and the addition of the considered personalized factors to a more adaptive strategy for rendering of guidance forces.

Reinforcement learning-based missile terminal guidance of maneuvering targets with decoys

In this paper,a missile terminal guidance law based on a new Deep Deterministic Policy Gradient(DDPG)algorithm is proposed to intercept a maneuvering target equipped with an infrared decoy.First,to deal with the issue that the missile cannot accurately distinguish the target from the decoy,the energy center method is employed to obtain the equivalent energy center(called virtual target)of the target and decoy,and the model for the missile and the virtual decoy is established.Then,an improved DDPG algorithm is proposed based on a trusted-search strategy,which significantly increases the train efficiency of the previous DDPG algorithm.Furthermore,combining the established model,the network obtained by the improved DDPG algorithm and the reward function,an intelligent missile terminal guidance scheme is proposed.Specifically,a heuristic reward function is designed for training and learning in combat scenarios.Finally,the effectiveness and robustness of the proposed guidance law are verified by Monte Carlo tests,and the simulation results obtained by the proposed scheme and other methods are compared to further demonstrate its superior performance.

基于强化学习的多段连续体机器人轨迹规划

针对多段连续体机器人的轨迹规划问题,提出了一种基于深度确定性策略梯度强化学习的轨迹规划算法。首先,基于分段常曲率假设方法,建立连续体机器人的关节角速度和末端位姿的正向运动学模型。然后,采用强化学习算法,将机械臂的当前位姿和目标位姿等信息作为状态输入,将机械臂的关节角速度作为智能体的输出动作,设置合理的奖励函数,引导机器人从初始位姿向目标位姿移动。最后,在MATLAB中搭建仿真系统,仿真结果表明,强化学习算法成功对多段连续体机器人进行轨迹规划,控制连续体机器人的末端平稳运动到目标位姿。

运动能力形成过程阶段划分探究

运用文献研究、逻辑分析方法,依据相关理论,对运动能力形成过程的阶段划分进行研究。研究认为:运动能力形成是一个发展过程,分为有效激发兴趣、准确学习技术、熟练掌握技能与灵活运用技能4个阶段;每个阶段之间呈层级递进关系,即前一阶段的形成是后一阶段学习的前提。在落实“教会、勤练、常赛”的课程理念下,研究运动能力形成过程阶段划分,有助于实践者在培养过程中清晰其形成过程、明确各阶段培养目标,更有利于对学生运动能力培养,也为后续研究提供理论参考。

农业院校“思想+教学+科研”三维研究生导学思政教学机制——以“科技论文写作与研究方法论”课程为例

新时代要求进一步创新研究生的人才培养方式改革,以研究生的品质养成与专业成长为中心,努力构建三维研究生导学思政培养机制,主要包括具体化课程思政的建设目标、体现特色的课程思政建设方向和重点、课程思政教学的激励手段、形成迭代升级的导学团队研讨机制。为了实施上述机制打造“一核两翼三融合”的研究生导学思政育人模式,提出研究生线上线下混合教学模式。其特点是在关键研究路径上挖掘思政元素并融入规范的表达教学,包括具有中国经济和管理特点的科学问题、研究综述、概念框架和技术路线,有助于农业院校导师对研究生实施导学思政教学。

自主学习感觉生理学习模式的创立与实践

迄今,有关动物及人体感觉生理方面的研究,已获得7个诺贝尔生理学或医学奖。基于此情况,西北大学生命科学与医学部生理学教学团队通过对历年感觉生理教学过程与结果的研判,进行教学方法创新,提出“诺奖为基,议题为引,自主学习感觉生理”的自主学习模式。学生先通过团队的线上慕课学习感觉生理的基本知识,然后师生以诺奖成果为基础共同商定学习议题,由教师通过对议题的解析引导学生自主学习感觉生理。该自主学习模式显著提高了学生的学习主动性和学习兴趣,加深了学生对感觉生理内容的理解,提升了教师的教学水平,在实践中收到了良好的教学效果。

一种基于优化引导的无线联邦学习异步训练机制

针对在数据异构和资源异构的无线网络中联邦学习训练效率低及训练能耗高的问题,面向图像识别任务,提出了基于优化引导的异步联邦学习算法AFedGuide。利用较高样本多样性的客户端模型的引导作用,提高单轮聚合有效性。采用基于训练状态的模型增量异步更新机制,提高模型更新实时性以及信息整合能力。设计基于模型差异性的训练决策,修正优化方向。仿真结果显示,相较于对比算法,AFedGuide的训练时长平均减少67.78%,系统能耗平均节省65.49%,客户端的准确率方差平均减少25.5%,说明在客户端数据异构和资源异构的无线网络下,AFedGuide可以在较短的训练时间内以较小的训练能耗完成训练目标,并维持较高的训练公平性和模型适用性。

基于视频图像驱动的驾驶人注意力估计方法

驾驶人视觉注意力的深入研究对于预测不安全驾驶行为和理解驾驶行为具有重要意义。为此,提出一种基于视频图像驱动的驾驶人注意力估计方法,以估计驾驶人在行车时注意到视域内的行人或车辆等各种对象。该方法利用深度神经网络学习交通场景视频与驾驶员注意力特征之间的映射关系,并融入引导学习模块来提取与驾驶员注意力最相关的特征。考虑到驾驶的动态性,使用动态交通场景视频作为模型输入,设计时空特征提取模块。在稀疏、密集、低照度等常见的交通场景中,将估计的驾驶员注意力模型与收集的驾驶员注意力数据点进行对比。实验结果表明,所提方法能够准确估计驾驶员在驾驶过程中的注意力,对于预测不安全驾驶行为以及促进人们更好地理解驾驶行为具有重要的理论和实用价值。

基于实体复制和双粒度指导的抽象摘要

抽象神经网络在文本摘要领域取得了长足进步,展示了令人瞩目的成就.然而,由于抽象摘要的灵活性,它很容易造成生成的摘要忠实性差的问题,甚至偏离源文档的语义主旨.针对这一问题,本文提出了两种方法来提高摘要的保真度.(1)由于实体在摘要中起着重要作用,而且通常来自于原始文档,因此本文提出允许模型从源文档中复制实体,确保生成的实体与源文档中的实体相匹配,这有助于防止生成不一致的实体.(2)为了更好地防止生成的摘要与原文产生语义偏离,本文在摘要生成过程中使用关键实体和关键token作为两种不同粒度的指导信息以指导摘要的生成.本文使用ROUGE指标在两个广泛使用的文本摘要数据集CNNDM和XSum上评估了本文方法的性能,实验结果表明,这两种方法在提高模型性能方面都取得了显著的效果.此外,实验还证明了实体复制机制可以在一定程度上借助指导信息以纠正引入的语义噪声.