Gradient Optimizer Algorithm with Hybrid Deep Learning Based Failure Detection and Classification in the Industrial Environment

Failure detection is an essential task in industrial systems for preventing costly downtime and ensuring the seamlessoperation of the system. Current industrial processes are getting smarter with the emergence of Industry 4.0.Specifically, various modernized industrial processes have been equipped with quite a few sensors to collectprocess-based data to find faults arising or prevailing in processes along with monitoring the status of processes.Fault diagnosis of rotating machines serves a main role in the engineering field and industrial production. Dueto the disadvantages of existing fault, diagnosis approaches, which greatly depend on professional experienceand human knowledge, intellectual fault diagnosis based on deep learning (DL) has attracted the researcher’sinterest. DL reaches the desired fault classification and automatic feature learning. Therefore, this article designs a Gradient Optimizer Algorithm with Hybrid Deep Learning-based Failure Detection and Classification (GOAHDLFDC)in the industrial environment. The presented GOAHDL-FDC technique initially applies continuous wavelettransform (CWT) for preprocessing the actual vibrational signals of the rotating machinery. Next, the residualnetwork (ResNet18) model was exploited for the extraction of features from the vibration signals which are thenfed into theHDLmodel for automated fault detection. Finally, theGOA-based hyperparameter tuning is performedtoadjust the parameter valuesof theHDLmodel accurately.The experimental result analysis of the GOAHDL-FD Calgorithm takes place using a series of simulations and the experimentation outcomes highlight the better resultsof the GOAHDL-FDC technique under different aspects.

DDPG深度强化学习算法在无人船目标追踪与救援中的应用

为保证海上救援活动的高效性,研究结合深度确定性策略梯度算法(Deep Deterministic Policy Gradient,DDPG)从状态空间、动作空间、奖励函数方面对船只追踪救援目标算法进行设计,并实际应用到无人船追踪救援之中。结果显示DDPG算法的稳定成功率接近100%,性能优异。该设计的算法最终回合累积奖励值能够稳定在10左右,而平均时长则能稳定在80 s左右,能够根据周边环境的状态调整自己的运动策略,满足海上救援活动中的紧迫性要求,能为相关领域的研究提供一条新的思路。

A UAV collaborative defense scheme driven by DDPG algorithm

The deep deterministic policy gradient(DDPG)algo-rithm is an off-policy method that combines two mainstream reinforcement learning methods based on value iteration and policy iteration.Using the DDPG algorithm,agents can explore and summarize the environment to achieve autonomous deci-sions in the continuous state space and action space.In this paper,a cooperative defense with DDPG via swarms of unmanned aerial vehicle(UAV)is developed and validated,which has shown promising practical value in the effect of defending.We solve the sparse rewards problem of reinforcement learning pair in a long-term task by building the reward function of UAV swarms and optimizing the learning process of artificial neural network based on the DDPG algorithm to reduce the vibration in the learning process.The experimental results show that the DDPG algorithm can guide the UAVs swarm to perform the defense task efficiently,meeting the requirements of a UAV swarm for non-centralization,autonomy,and promoting the intelligent development of UAVs swarm as well as the decision-making process.

基于DDPG改进PID算法的堆肥翻堆作业反馈控制

在农业废弃物堆肥发酵过程中物料的含水率会发生变化,导致翻堆作业负荷的变化。而现有的翻堆作业主要通过人工操作,导致机器作业过程中作业效率低,容易产生故障。针对人工操作翻堆机作业时出现调控不精准的问题,通过试验构建翻堆机作业负荷与翻堆物料含水率、翻堆机行走速度关系模型,并结合翻堆机变频调速控制模型,利用DDPG(Deep Deterministic Policy Gradient)改进PID算法对翻堆作业调速系统进行优化控制。经过Simulink仿真结果表明,DDPG改进PID算法相比传统PID算法在超调量上减少6.7%,调节时间减少2.5 s,并且抗扰动与跟随性能均更优。翻堆作业现场测试结果表明:DDPG改进PID算法的控制方式相比传统PID算法超调量要降低4%、调节时间减少2 s,相比人工控制其调节时间减少6 s。

自动驾驶路径优化的RF-DDPG车辆控制算法研究

针对自动驾驶车辆在行使中对目标路径跟踪精度不高、鲁棒性能较差等问题,提出了一种深度确定性策略梯度RF-DDPG(reward function-deep deterministic policy gradient)路径跟踪算法。该算法是在深度强化学习DDPG的基础上,设计DDPG算法的奖励函数,以此优化DDPG的参数,达到所需跟踪精度及稳定性。并且采用aopllo自动驾驶仿真平台,对原始的DDPG算法和改进的RF-DDPG路径跟踪控制算法进行了仿真实验。研究结果表明,所提出的RF-DDPG算法在路径跟踪精度以及鲁棒性能等方面均优于DDPG算法。

基于DDPG算法的MISO通信系统优化

无线通信系统中部署大量天线时会增加硬件成本,且能耗和信号处理的复杂性加大,针对此问题提出了一种基于深度确定性策略梯度(DDPG)算法的多输入单输出(MISO)通信系统优化方案。研究智能反射面(RIS)辅助多用户的MISO系统,在连续状态和动作背景下,利用DDPG算法联合设计基站端波束成形矩阵和RIS处相移,以最大化系统加权和率。仿真实验结果表明,该方案与FP算法的加权和率性能相当且收敛性较好,相比未部署RIS的方案具有明显优势。

基于改进DDPG的变速抽蓄机组参与系统调频研究

在挖掘双馈型抽水蓄能(DFIM-PSH)机组调频能力的基础上,提出一种基于改进深度确定性策略梯度(DDPG)算法的系统频率控制方法。首先,基于所确定的DFIM-PSH机组在发电与抽水工况下的频率控制环节,构建考虑风电接入的含DFIMPSH单区域系统频率控制模型。其次,在考虑机组运行约束的基础上以最小化系统频率偏差及调频出力为目标,引入DDPG算法对各机组的AGC控制指令进行优化。通过在预学习中同时引入随机外部扰动与模型参数变化,提高AGC控制器在具有强不确定性环境中的适应性。最后,在仿真验证DFIM-PSH调频优势的基础上,在不同风电接入及扰动等多场景进行仿真分析,结果表明,所提频率控制方法能有效改善新型电力系统的频率特性且具有强鲁棒性。

Data-Driven Learning Control Algorithms for Unachievable Tracking Problems

For unachievable tracking problems, where the system output cannot precisely track a given reference, achieving the best possible approximation for the reference trajectory becomes the objective. This study aims to investigate solutions using the Ptype learning control scheme. Initially, we demonstrate the necessity of gradient information for achieving the best approximation.Subsequently, we propose an input-output-driven learning gain design to handle the imprecise gradients of a class of uncertain systems. However, it is discovered that the desired performance may not be attainable when faced with incomplete information.To address this issue, an extended iterative learning control scheme is introduced. In this scheme, the tracking errors are modified through output data sampling, which incorporates lowmemory footprints and offers flexibility in learning gain design.The input sequence is shown to converge towards the desired input, resulting in an output that is closest to the given reference in the least square sense. Numerical simulations are provided to validate the theoretical findings.

基于APF-LSTM-DDPG算法的移动机器人局部路径规划

针对深度强化学习算法存在训练时间长、收敛速度慢的问题,将深度确定性策略梯度(deep deterministic policy gradient,DDPG)算法和人工势场(artifical potential field,APF)法相融合,引入长短期记忆(long short-term memory,LSTM)神经网络结构,提出了APF-LSTM-DDPG算法。首先在DDPG算法中添加LSTM,通过记忆单元和遗忘单元将奖励较高的样本优先学习,使模型更快地收敛;其次引入人工势场法,设计相应势场函数,解决环境奖励稀疏的缺点,加速模型收敛进程;然后通过人工势场法修正算法的动作选择,缩短路径长度;最后在机器人仿真平台(robot operating system,ROS)中搭建不同障碍物环境对算法进行了仿真验证。仿真结果表明,APF-LSTM-DDPG算法在搭建的仿真环境中训练时平均奖励能够更快地稳定,提高了算法的成功率并减少了规划路径中的冗余。

Fractional Gradient Descent RBFNN for Active Fault-Tolerant Control of Plant Protection UAVs

With the increasing prevalence of high-order systems in engineering applications, these systems often exhibitsignificant disturbances and can be challenging to model accurately. As a result, the active disturbance rejectioncontroller (ADRC) has been widely applied in various fields. However, in controlling plant protection unmannedaerial vehicles (UAVs), which are typically large and subject to significant disturbances, load disturbances andthe possibility of multiple actuator faults during pesticide spraying pose significant challenges. To address theseissues, this paper proposes a novel fault-tolerant control method that combines a radial basis function neuralnetwork (RBFNN) with a second-order ADRC and leverages a fractional gradient descent (FGD) algorithm.We integrate the plant protection UAV model’s uncertain parameters, load disturbance parameters, and actuatorfault parameters and utilize the RBFNN for system parameter identification. The resulting ADRC exhibits loaddisturbance suppression and fault tolerance capabilities, and our proposed active fault-tolerant control law hasLyapunov stability implications. Experimental results obtained using a multi-rotor fault-tolerant test platformdemonstrate that the proposed method outperforms other control strategies regarding load disturbance suppressionand fault-tolerant performance.

基于DDPG的综合化航电系统多分区任务分配优化方法

综合化航电系统(Integrated Modular Avionics,IMA)通过时空分区机制实现共享资源平台下的多航电功能集成,分区间的任务分配方法的优劣决定着航电系统的整体效能。针对航电任务集合在多分区内的分配调度问题,提出了一种基于深度强化学习的优化方法。构建了航电系统模型与任务模型,以系统资源限制与任务实时性需求为约束,以提高系统资源利用率为优化目标,将任务分配过程描述为序贯决策问题。引入马尔科夫决策模型,建立基于深度确定性策略梯度(Deep Deterministic Policy Gradient,DDPG)法的IMA任务分配模型并提出通用分配架构;引入状态归一化、行为噪声等策略训练技巧,提高DDPG算法的学习性能和训练能力。仿真结果表明,提出的优化算法迭代次数达到500次时开始收敛,分析800次之后多分区内驻留任务方案在能满足约束要求的同时,最低处理效率提升20.55%。相较于传统分配方案和AC(Actor-Critic)算法,提出的DDPG算法在收敛能力、优化性能以及稳定性上均有显著优势。

基于MADDPG的多AGVs路径规划算法

针对多辆自动导引车系统(automated guided vehicle system,AGVs)在动态不确定环境下完成货物运送并进行路径规划的问题,提出一种基于多智能体深度确定性策略梯度(MADDPG)的多AGVs路径规划算法。本方法通过状态空间、动作空间、奖励函数和网络结构重新设计MADDPG算法的模型结构,通过OpenAI Gym接口搭建二维仿真环境用作多AGVs (agents)的训练平台。实验结果表明,相比于深度确定性策略梯度(DDPG)算法和双延迟深度确定性策略梯度(TD3)算法,基于MADDPG的多AGVs路径规划算法在智能仓储仿真环境下,多AGVs碰到货架的次数分别减少了21.49%、11.63%,碰到障碍物的次数分别减少了14.69%、10.12%,全部AGVs到达货物装卸点的成功率分别高出了17.22%、10.53%,表明学习后的AGV具有更高效的在线决策能力和自适应能力,能够找到较优的路径。

A-DDPG:多用户边缘计算系统的卸载研究

为了降低多边缘服务器多用户系统中用户的总成本,结合深度确定性策略梯度(deep deterministic policy gradient,DDPG)、长短期记忆网络(LSTM)和注意力机制,提出了一种基于DDPG的深度强化学习卸载算法(A-DDPG)。该算法采用二进制卸载策略,并且将任务的延迟敏感性和服务器负载的有限性以及任务迁移考虑在内,自适应地卸载任务,以最大限度减少由延迟敏感型任务超时造成的总损失。考虑时延和能耗两个指标并设定了不同的权重值,解决因用户类型不同带来的不公平问题,制定了任务卸载问题以最小化所有任务完成时延和能量消耗的总成本,以目标服务器的选择和数据卸载量为学习目标。实验结果表明,A-DDPG算法具有良好的稳定性和收敛性,与DDPG算法和双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)算法相比,A-DDPG算法的用户总成本分别降低了27%和26.66%,平均达到最优任务失败率的时间分别提前了57.14%和40%,其在奖励、总成本和任务失败率方面取得了较好的效果。

基于DDPG算法的微网负载端接口变换器自抗扰控制

直流微电网是新能源综合利用的重要形式,但其中的分布式接口往往存在着强随机性扰动,这给直流变换器的稳压控制带来了诸多问题。为了尽可能地抑制控制器参数固定时这种不确定性特征引起的不利影响,提出了一种利用深度确定性策略梯度(deep deterministic policy gradient,DDPG)算法整定线性自抗扰控制器参数的方法。依靠引入了智能算法的自抗扰微电网控制系统,实现了控制器参数的自适应调整,从而实现了微电网接口变换器的稳定运行。通过仿真对比了各类典型工况下,DDPG-LADRC与传统线性自抗扰控制器(linear active disturbance rejection control,LADRC)、双闭环比例-积分控制器的性能差异,验证了所提控制策略的有效性。而参数摄动下的鲁棒性分析结果结合多项指标下的系统整体性分析,充分体现了控制器参数的智能化调整所带来的多工况自适应性增益的优越性,具备较强的工程价值。

ADAPTIVE EXPONENT SMOOTHING GRADIENT ALGORITHM

由于ＬＭＳ算法具有权调节时间延迟和低通滤波的特性，故提出一种新的自适应指数平滑梯度算法。研究表明，当信号是一个高斯平稳过程时，在参数域｛Ω１：α∈（０，１）｝×｛Ω２：β∈（０，∞）｝上，该算法渐进无偏收敛于维纳解。本文给出了算法收敛性能和性能失调的理论分析以及计算公式。计算机模拟的数值结果表明，该算法是有效的。

基于DDPG算法的列车节能控制策略研究

随着城市范围的扩大和人口数量的增加,建设城市轨道交通成为解决交通拥堵的重要方式,其安全性、准点性和实时性也越来越受到重视。城市轨道交通运行能耗是列车运营成本的重要组成,而牵引能耗是城市轨道交通运行能耗的主要组成部分。合理的列车自动控制(Automatic Train Operation,ATO)策略能够在保证列车安全、准点运行的情况下,根据列车不同的运行状况,输出最优的牵引/制动级位,在一定程度上节省牵引能耗。为有效降低列车牵引能耗,采用深度强化学习中的深度确定性策略梯度算法(Deep Deterministic Policy Gradient,DDPG)研究城市轨道交通列车节能控制策略。首先根据动力学方程建立列车运行模型,然后以列车的位置、速度和剩余运行时间作为状态空间,以牵引/制动级位作为动作空间,根据ATO系统功能,以安全性、准点性、停车精度和节能性加权组合为奖励函数,实现列车在与仿真环境交互中学习到最优控制策略。最后以长沙地铁2号线为例进行仿真验证,结果表明该算法相比于DQN算法和动态规划算法,在满足安全、准点运行的情况下牵引能耗减少8.25%和21.7%,证明该算法的优越性,同时在临时调整列车进站时间和列车牵引系统故障后,算法也能够根据列车反馈的运行状态实时调整控制策略,证明该算法的有效性。

基于DDPG算法的双轮腿机器人运动控制研究

轮腿式机器人兼具轮式和足式机器人的机动性和灵活性,在多种场景中具有广泛的应用前景。针对双轮腿机器人在崎岖地形运动控制缺陷、高度依赖于精确动力学模型、无法自适应求解等问题,提出一种基于深度确定性策略梯度(deep deterministic policy gradient,DDPG)算法的双轮腿机器人控制方法。首先,分析了双轮腿机器人模型及其模糊动力学模型;然后,使用DDPG算法生成双轮腿机器人在崎岖地面的运动控制策略;最后,为了验证控制器性能,分别进行了3组运动控制对比实验。仿真实验表明,在缺少地面状况先验知识的条件下,采用DDPG算法生成的运动控制策略实现了双轮腿式机器人在崎岖地面快速稳定运动的功能,其平均速度相比双轮机器人提高了约29.2%,姿态角偏移峰值相比双足机器人分别减小了约43.9%、66%、50%。

改进DDPG算法在外骨骼机械臂轨迹运动中的应用

针对现阶段外骨骼机器人轨迹运动时出现效果不佳的问题,提出了基于优先经验回放与分区奖励(PERDA)融合的深度确定性策略梯度(DDPG)强化学习算法,即PERDA-DDPG。该方法利用时间差分误差(TD-errors)的大小对经验排序,改变了原始采样的策略。此外,相较于以往二值奖励函数,本文根据物理模型提出针对化的分区奖励。在Open AI Gym平台上实现仿真环境,实验结果表明:改进的算法收敛速度提升了约9.2%,学习过程更加稳定。

Improved gradient iterative algorithms for solving Lyapunov matrix equations

In this paper, an improved gradient iterative （GI） algorithm for solving the Lyapunov matrix equations is studied. Convergence of the improved method for any initial value is proved with some conditions. Compared with the GI algorithm, the improved algorithm reduces computational cost and storage. Finally, the algorithm is tested with GI several numerical examples.

Genetic Algorithm for the Thermal Stresses Optimum Design ofFunctionally Gradient Material Plate

Based on the thermal stress distribution for functionally gradient material (FGM) plates, a Genetic Algorithm (GA) method for the thermal stresses optimum design of FGM plate with computer technologies is given. The minimum thermal stresses combination distribution for FGM is obtained.