Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Decomposition of Mathematical Programming Models for Aircraft Wing Design Facilitating the Use of Dynamic Programming Approach

Aircraft designers strive to achieve optimal weight-reliability tradeoffs while designing an aircraft. Since aircraft wing skins account for more than fifty percent of their structural weight, aircraft wings must be designed with utmost care and attention in terms of material types and thickness configurations. In particular, the selection of thickness at each location of the aircraft wing skin is the most consequential task for aircraft designers. To accomplish this, we present discrete mathematical programming models to obtain optimal thicknesses either to minimize weight or to maximize reliability. We present theoretical results for the decomposition of these discrete mathematical programming models to reduce computer memory requirements and facilitate the use of dynamic programming for design purposes. In particular, a decomposed version of the weight minimization problem is solved for an aircraft wing with thirty locations (or panels) and fourteen thickness choices for each location to yield an optimal minimum weight design.

Optimal Design of High-Speed Partial Flow Pumps using Orthogonal Tests and Numerical Simulations

To investigate the influence of structural parameters on the performances and internal flow characteristics of partial flow pumps at a low specific speed of 10000 rpm,special attention was paid to the first and second stage impeller guide vanes.Moreover,the impeller blade outlet width,impeller inlet diameter,blade inclination angle,and number of blades were considered for orthogonal tests.Accordingly,nine groups of design solutions were formed,and then used as a basis for the execution of numerical simulations(CFD)aimed at obtaining the efficiency values and heads for each design solution group.The influence of impeller geometric parameters on the efficiency and head was explored,and the“weight”of each factor was obtained via a range analysis.Optimal structural parameters were finally chosen on the basis of the numerical simulation results,and the performances of the optimized model were verified accordingly(yet by means of CFD).Evidence is provided that the increase in the efficiency and head of the optimized model was 12.11%and 23.5 m,respectively,compared with those of the original model.

Suggestions on Improving China’s Breakthrough Therapy Designation System

Objective To analyze the characteristics of breakthrough therapy designation(BTD)and its implementation in China,and to provide reference for the optimization of BTD system.Methods A comparative research method was used to study the content and implementation effect of BTD system in China and the relevant policies and implementation of the same procedures of drug regulatory authorities in the United States,Japan and the European Union.Then,the differences in policies and implementation results among these countries were analyzed to provide suggestions for the implementation and optimization of this system in China.Results and Conclusion China’s BTD system is implemented late and a small number of drugs has been approved.At the same time,there are problems such as insufficient guidance and communication from the agency to applicants,a broad application condition,single review mode,and lack of full-time personnel.Both the agencies and the applicants have limited experience due to the short implementation time of BTD system in China.There are still some problems despite we have learned a lot from the experience of other drug regulatory agencies.Therefore,based on our national conditions,we should strengthen the guidance of evaluation agency to applicants,optimize the eligibility criteria of BTD system,introduce the rolling review,and increase the number of professional liaisons,which can accelerate the development and marketing process of drugs with obvious clinical value,and finally to address unmet medical need.

Improved design of reconfigurable frequency response masking filters based on second-order cone programming

In order to improve the design results for the reconfigurable frequency response masking FRM filters an improved design method based on second-order cone programming SOCP is proposed.Unlike traditional methods that separately design the proposed method takes all the desired designing modes into consideration when designing all the subfilters. First an initial solution is obtained by separately designing the subfilters and then the initial solution is updated by iteratively solving a SOCP problem. The proposed method is evaluated on a design example and simulation results demonstrate that jointly designing all the subfilters can obtain significantly lower minimax approximation errors compared to the conventional design method.

A Dynamic Programming Approach to the Design of Composite Aircraft Wings

A light and reliable aircraft has been the major goal of aircraft designers. It is imperative to design the aircraft wing skins as efficiently as possible since the wing skins comprise more than fifty percent of the structural weight of the aircraft wing. The aircraft wing skin consists of many different types of material and thickness configurations at various locations. Selecting a thickness for each location is perhaps the most significant design task. In this paper, we formulate discrete mathematical programming models to determine the optimal thicknesses for three different criteria: maximize reliability, minimize weight, and achieve a trade-off between maximizing reliability and minimizing weight. These three model formulations are generalized discrete resource-allocation problems, which lend themselves well to the dynamic programming approach. Consequently, we use the dynamic programming method to solve these model formulations. To illustrate our approach, an example is solved in which dynamic programming yields a minimum weight design as well as a trade-off curve for weight versus reliability for an aircraft wing with thirty locations (or panels) and fourteen thickness choices for each location.

波浪能发电装置垂荡浮子PTO系统的建模与计算

为了便捷求出垂荡式波浪能发电装置垂荡浮子的平均功率,对现有的设计计算方法进行了研究。基于波浪理论和Froude-Krylov假定法求解波浪力,应用单自由度有阻尼系统受迫振动理论,将波浪激励力作为对垂荡浮体的输入动力,将发电机的电磁阻力作为主要的阻尼力,建立了质量-弹簧-阻尼振动力学模型,确定了垂荡浮子响应的计算方法,并对实际波浪能发电装置垂荡浮子Power Take Off(PTO)系统列举出计算实例。在算例基础上,对假设条件进行了扩展,应用频率响应曲线、MATLAB软件做出不同参数条件下平均功率三维曲线和输出平均功率数值表,清晰地表明了系统响应状况。研究显示,在波高和频率比一定的条件下,PTO系统平均机械功率随着浮体质量和阻尼力矩的变化具有峰值,围绕峰值区域设计装置的各项参数,能够取得较理想的效果。该建模与计算方法可为实际工程优化设计提供参考。

面向乡村建筑分时分区用能需求的零碳综合能源系统设计优化

基于传统最大负荷原则设计的综合能源系统容量通常远大于建筑的实际用能需求,导致以户为单位的乡村综合能源系统设计缺乏针对性和适用性。该文通过现场调查和数值模拟的方法精细化乡村建筑的实际用能需求,基于农户用能行为制定4种用能策略。并构建考虑农户经济水平和零碳约束的综合能源系统设计优化模型,采用两阶段带精英策略的快速非支配排序遗传算法(Top-NSGA-II)进行多目标优化问题的求解,获得不同策略下的最佳设计方案。结果表明,与基于建筑全空间连续动态负荷的设计优化方案相比,基于分时分区用能需求的设计方案年总成本降低13.58%,系统购电比例降低7.89个百分点,实现全年净零碳排放。

基于双层规划的飞行区地势设计优化模型

为设计出一个符合标准且成本最低的机场飞行区地势表面,在飞行区内划分出骨架和扭曲面,上层规划从全局角度确定骨架具体位置,下层规划根据上层规划的结果,以局部表面跟天然地面的拟合程度为目标,合理衔接骨架并使得扭曲面设计最优,形成基于双层规划的飞行区地势设计优化模型,最后运用遗传算法求解。采用该模型计算得到的三明机场地势设计实例表明,优化模型的排水、坡度、坡段设计均符合既有要求,且与实际工程相比,更贴合天然地形,填方量下降7.50%,土方总计下降8.06%,工程成本降低10.73%。

中小型高桩梁板式码头结构优化研究

以某旅游休闲码头工程为依托,以结构安全、节省投资为目标,对常规的高桩梁板式结构,通过优化桩基布置、调整结构分缝型式、设置大刚度靠船块体和挡浪边板,提出一种适应于中小型码头的新型高桩梁板式结构。经技术经济比较,优化后的结构可节省工程投资达623万元。同时针对该结构空间特征显著的特点,对其三维计算模型模拟原则、单元类型选择及桩基与上部结构的连接拟合进行分析研究,给出指导性意见,可为类似项目的结构设计提供技术借鉴与参考。

大口径火炮弹协调器机构可靠性优化设计研究

为提高弹协调器的交弹效率,同时保障协调交弹动作具有高可靠度,开展某弹协调器机构的可靠性优化设计。考虑主要几何尺寸、制造误差、重要构件弹性变形等影响因素,建立某弹协调器的参数化刚柔耦合动力学模型,通过参数化动力学分析,复现协调器的协调交弹动作失效,并建立相对应的功能函数及协调器可靠性优化设计模型。针对协调器可靠性优化模型,为提高优化设计的效率和精度,构建新的Kriging模型自适应更新策略,并与序列二次规划(SQP)方法、功能函数度量法(PMA)/可靠度指标法(RIA)结合,提出协调器机构可靠性优化设计方法。研究结果表明,在协调交弹可靠度满足要求的情况下大幅提高了某协调器的协调效率,也验证了所提方法的有效性和工程价值。

基于CFD模拟与数学规划法的熔融聚合物流动管道设计优化

在满足产能和合理的管道压降范围下,如何降低熔融聚合物在熔体管道中黏度降问题是熔体流动管道设计的关键因素。本研究基于计算流体力学(CFD)的方法,对熔体流经管道发生热降解过程进行数值模拟,并在此基础上提出了混合整数非线性规划(MINLP)模型和非连续性非线性规划(DNLP)模型,分别对熔体管道进行设计与布局优化。分析实际算例的数值模拟结果发现,优化后的管道相较于原管道黏度降减小3.48%,同时出口的压降分布范围减小了60.47%,出口的压降分布更加均匀。结果表明,综合利用模拟方法与数学规划模型可实现熔融聚合物流动管道的自动化设计及优化。

基于SQP算法的厢舱类产品快速设计技术研究

针对大规模定制背景下产品设计制造过程中遇到的轻量化和强度校核等需求,提出了一种基于序列二次规划(SQP)算法的结构优化与快速设计方法。以方舱为实例对象,按照最优控制理论构建产品关键参数设计数学计算模型,并运用SQP算法实现参数的最优求解,同时通过产品族模块化划分和参数化变型配置技术实现产品结构的快速设计。最后通过二次开发构建了相应的数字化设计平台,验证了该方法的可行性。结果表明:该方法能有效提高产品结构设计的效率和合理性。

基于膜计算的扩散优化算法及工程应用

文中提出一种全局优化算法,称为基于膜计算的扩散优化算法(MC-DOA).该算法受到叶绿体和线粒体双膜细胞结构以及粒子扩散现象的启发,利用一个混合膜结构系统和四种不同的粒子搜索策略来寻找全局最优值.选取了两类基准测试函数对MC-DOA的性能进行测试试验,结果表明:MC-DOA能够精确快速的找到测试函数的最优值.将MC-DOA应用到实际工程问题——船舶螺旋桨参数优化设计当中,结果显示MC-DOA能够通过对螺旋桨参数的优化,有效地提高螺旋桨的敞水效率,表明了MC-DOA在船舶优化设计领域的适用性和有效性.

滑坡碎屑流消能桩结构动力计算与设计方法

滑坡碎屑流由于体积大、速度快和运动距离远的特性,常常在下游造成较大破坏形成严重的自然灾害。基于消能桩与滑坡碎屑流结构的动力相互作用过程,采用功能原理和结构力学方法,推导了消能桩对滑坡碎屑流能量耗散的理论计算模型。首先,通过冲击力计算模型获得滑坡碎屑流冲击力的垂向分布;其次,由悬臂梁挠曲变形的关系确定其抗弯反力;然后,根据滑坡碎屑流在坡面上流动的面积、厚度估算消能桩的反力,以及在反力方向上碎屑流通过的距离;最后,计算单根桩的动能耗散量。采用推导的公式进行消能桩的优化设计:综合考虑下游防护对象的位置和滑坡体的堆积位置,计算消能桩的“消能效益”,确定消能桩的数量;根据最大块石的冲击力作用,计算需要消能桩的排数,从而确定消能桩的排布设计。选取一个滑坡碎屑流实例,采用上述理论计算模型设计消能桩,计算出消能桩的数量和排数,并根据计算结果进行优化布设。结果表明,该模型适用于消能桩的优化设计与应用。

多学科设计优化方法在水下无人航行器中应用

为了提高水下无人航行器工作的可靠性和稳健性,本文采用多学科设计优化方法开展航行器的总体设计优化。首先开展水下航行器在总体设计阶段的学科设计,分析了各学科内部的模型和计算方法,明确了各学科输入、输出和耦合关系;开展了学科内参数建模,确立了总体设计的设计变量、系统参数和约束条件。然后以航行器质量最轻和机动性能最强为优化目标,获得设计变量计算最优解和优化设计主要输出结果值。设计变量优化计算值有效提高设计方案满足约束的能力,优化设计主要输出结果为水下无人航行器设计方案的选择提供了定量依据。

低水头宽变幅抽水蓄能机组水泵水轮机泵工况优化研究

本文针对国内某低水头抽水蓄能电站宽水头变幅特点,基于CFD优化方法对水泵水轮机模型进行水力性能研究。该电站运行水头变幅达到1.565,远超出同水头段定速抽水蓄能电站水头变幅,水力设计难度极大。在相对导叶高度b0/D1=0.185基础上对水泵水轮机进行优化设计,通过数值模拟方法对水泵水轮机整体性能进行预估。对比分析设计方案对宽水头变幅的适应性,同时兼顾水泵与水轮机工况的能量匹配关系,最终JFP20水泵水轮机方案达到了项目要求,研究成果可为今后中低水头宽变幅抽水蓄能机组水泵水轮机的参数选择和水力设计提供参考。

滑坡稳定性评价及治理工程设计软件Slope Designer

主要介绍了Slop eDesigne r1.2的设计开发思路和主要功能。该软件是基于Windows面向对象技术开发的滑坡稳定性评价与治理工程设计的一体化工程软件,采用Visual C++进行程序设计,包括稳定性分析子模块和抗滑桩设计子模块2部分,可对地震作用、暴雨等不同工况下的滑坡进行稳定性评价和推力计算,并能够对(锚拉)抗滑桩(弹性桩、刚性桩)的弯矩、剪力及侧应力进行计算,最后根据内力计算结果完成配筋计算。在三峡库区兴山县二里半滑坡的应用研究表明,该软件具有很好的工程实用性。

CN1/18道岔辙叉轨件力学响应特征及裂纹整治方法

针对CN系列道岔在不利条件下可能出现的心轨、翼轨及叉跟尖轨伤损等问题,建立CN1/18道岔辙叉轨件有限元模型,计算分析其力学响应特征;采用容许应力法分析验证病害产生原因,根据轨件受力状态提出了相应整治措施。结果表明:心轨和叉跟尖轨薄弱断面应力较大,有加工缺陷或划痕硌伤的条件下可能出现应力集中;准静态荷载作用下,翼轨强度满足要求,但安全余量较小,在长期服役条件下可能由于外部环境变化出现钢轨伤损,导致裂纹萌生和断裂。对于心轨,应加强凸缘和轨底交角位置的目视检查和涡流探伤,及时开展人工打磨整治或换轨处理;对于翼轨,应及时开展专项探伤,做好内倾的检查和调整;对于叉跟尖轨,应及时开展磁粉探伤,发现裂纹时须进行换轨处理。