Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Complex adaptive system theory,agent-based modeling,and simulation in dominant technology formation

Dominant technology formation is the key for the hightech industry to“cross the chasm”and gain an established foothold in the market(and hence disrupt the regime).Therefore,a stimulus-response model is proposed to investigate the dominant technology by exploring its formation process and mechanism.Specifically,based on complex adaptive system theory and the basic stimulus-response model,we use a combination of agent-based modeling and system dynamics modeling to capture the interactions between dominant technology and the socio-technical landscape.The results indicate the following:(i)The dynamic interaction is“stimulus-reaction-selection”,which promotes the dominant technology’s formation.(ii)The dominant technology’s formation can be described as a dynamic process in which the adaptation intensity of technology standards increases continuously until it becomes the leading technology under the dual action of internal and external mechanisms.(iii)The dominant technology’s formation in the high-tech industry is influenced by learning ability,the number of adopting users and adaptability.Therein,a“critical scale”of learning ability exists to promote the formation of leading technology:a large number of adopting users can promote the dominant technology’s formation by influencing the adaptive response of technology standards to the socio-technical landscape and the choice of technology standards by the socio-technical landscape.There is a minimum threshold and a maximum threshold for the role of adaptability in the dominant technology’s formation.(iv)The socio-technical landscape can promote the leading technology’s shaping in the high-tech industry,and different elements have different effects.This study promotes research on the formation mechanism of dominant technology in the high-tech industry,presents new perspectives and methods for researchers,and provides essential enlightenment for managers to formulate technology strategies.

CAS理论视角下我国智慧社区公共体育服务多元治理

采用文献资料、案例分析等研究方法,结合嘉兴市“运动家”智慧体育社区治理实践,从复杂适应系统(CAS)理论视角分析智慧社区公共体育服务治理系统的结构、特征和运行机制。认为:智慧社区公共体育服务治理结构由服务供需主体、政策制度系统、社区环境系统和社会环境系统等多元系统构成,具有聚集、多样性、“流”以及非线性特征,其运行机制包含以“刺激—反应”模型运行的内部自适应机制和以“回声”模型运行的外部交互机制。在此基础上,提出创新“积木块”组合方式、优化制度设计,拓宽资源“流”渠道、整合治理资源,建立创新引领新“标识”、聚合主体共识,重视主体间的“非线性”关系、营造协同治理环境等优化策略。

Model for cascading failures with adaptive defense in complex networks

This paper investigates cascading failures in networks by considering interplay between the flow dynamic and the network topology, where the fluxes exchanged between a pair of nodes can be adaptively adjusted depending on the changes of the shortest path lengths between them. The simulations on both an artificially created scale-free network and the real network structure of the power grid reveal that the adaptive adjustment of the fluxes can drastically enhance the robustness of complex networks against cascading failures. Particularly, there exists an optimal region where the propagation of the cascade is significantly suppressed and the fluxes supported by the network are maximal. With this understanding, a costless strategy of defense for preventing cascade breakdown is proposed. It is shown to be more effective for suppressing the propagation of the cascade than the recent proposed strategy of defense based on the intentional removal of nodes.

基于小型化Cas蛋白的CRISPR/Gal4BD-Cas供体适配基因编辑系统研究

在哺乳动物细胞中,利用同源引导修复(homology-directed repair,HDR)机制的基因编辑策略能够实现精准的点编辑和敲入,但是HDR的低效性严重制约了该策略在精准医疗和分子设计育种中的应用。鉴于HDR机制所需的供体DNA模板不能自主募集到基因组双链断裂(double-stranded break,DSB)处,本课题组提出了供体适配系统(donor adapting system,DAS)的概念并开发出CRISPR/SpCas9-Gal4BD供体适配基因编辑系统。由于SpCas9蛋白分子较大,与Gal4BD适配器结合不利于表达、病毒载体包装及活体递送等过程,因此本研究进一步采用两种小型化Cas蛋白——路邓葡萄球菌(Staphylococcus lugdunensis)源SlugCas9变体SlugCas9-HF与氨基酸球菌(Acidaminococcus sp.)源AsCas12a,开发了新型的CRISPR/Gal4BD-SlugCas9和CRISPR/Gal4BD-AsCas12a供体适配基因编辑系统。通过SSA活性报告实验初步证明了Gal4BD与SlugCas9、AsCas12a N-端融合对其打靶活性影响较小。通过HDR效率报告实验进行功能验证并优化供体设计方案,结果表明:对于CRISPR/Gal4BD-AsCas12a DAS,适配器结合序列(binding sequence,BS)与供体5′-端融合(BS-dsDNA)效果较好;对于CRISPR/Gal4BD-SlugCas9 DAS,则BS与供体3′-端融合(dsDNA-BS)较佳。最终,利用CRISPR/Gal4BD-SlugCas9 DAS对HEK293T细胞中的EMX1、NUDT5、AAVS1三个基因位点分别实现了24%、37%、31%的精准编辑,相比对照组得到了显著提高。本研究为供体适配基因编辑系统的进一步优化提供了参考和借鉴,为后续动物分子设计育种应用研究提供了新的基因编辑工具。

基于CAS理论的非遗传统技艺传承模式评价指标体系设计

本文旨在探讨非遗传统技艺的传承模式受物理场景的局限以及传承主体的固化,导致传统技艺传承体验受限和传播目标缺失,已难适应新时代的需求现状。通过以复杂适应系统(Complex Adaptive System,CAS)理论为指导,借助“自适应主体”的引入,从“主体”与“环境”交互的角度审视复杂系统的行为特征,设计一个“体验+传播”的非物质文化遗产(非遗)传统技艺传承模式评价体系,为非遗保护、传承及其可持续发展提供更为科学、全面和动态的评价工具和策略,有助于实现非遗的活态保护和创造性转化。

基于复杂适应系统(CAS)理论的编辑出版人才培养机制研究

全面考察编辑出版人才培养中的各个主体及其所处的外部环境,探索新时代下编辑出版人才培养机制。文章引入复杂适应系统理论,明确系统内部的各个主体,并梳理微观层面上的主体适应性和宏观层面上的多元主体互动。依循核心主体—其他多元主体—主体与主体之间—主体与外部环境之间的各个系统层次,立足于对编辑出版人才主体地位的明确,提出新时代下政府、高校、出版单位、行业协会多主体联合的编辑出版人才培养机制。提出应尊重编辑出版人才主体及其个人发展,激发主体活力以释放新动能,促进主体间交互以实现整体涌现,并以多样化的人才培养成就多元化的编辑出版人才,以助力出版业的高质量发展。

APPLICATION STUDY ON ADAPTIVE NEURAL FUZZY INFERENCE MODEL IN COMPLEX SOCIAL-TECHNICAL SYSTEM

The adaptive neural fuzzy inference system （ANFIS） is used to make a ease study considering features of complex social-technical system with the target of increasing organizational efficiency of public scientific research institutions. An integrated ANFIS model is built and the effectiveness of the model is verified by means of investigation data and their processing results. The model merges the learning mechanism of neural network and the language inference ability of fuzzy system, and thereby remedies the defects of neural network and fuzzy logic system. Result of this case study shows that the model is suitable for complicated socio-technical systems and has bright application perspective to solve such problems of prediction, evaluation and policy-making in managerial fields.

Patterns of Interactions of the Complex City System:Emotional Urban Objects as Triggering Agents-A Secondary Publication

This article presents an analysis of the patterns of interactions resulting from the positive and negative emotional events that occur in cities,considering them as complex systems.It explores,from the imaginaries,how certain urban objects can act as emotional agents and how these events affect the urban system as a whole.An adaptive complex systems perspective is used to analyze these patterns.The results show patterns in the processes and dynamics that occur in cities based on the objects that affect the emotions of the people who live there.These patterns depend on the characteristics of the emotional charge of urban objects,but they can be generalized in the following process:(1)immediate reaction by some individuals;(2)emotions are generated at the individual level which begins to generalize,permuting to a collective emotion;(3)a process of reflection is detonated in some individuals from the reading of collective emotions;(4)integration/significance in the community both at the individual and collective level,on the concepts,roles and/or functions that give rise to the process in the system.Therefore,it is clear that emotions play a significant role in the development of cities and these aspects should be considered in the design strategies of all kinds of projects for the city.Future extensions of this work could include a deeper analysis of specific emotional events in urban environments,as well as possible implications for urban policy and decision making.

职业本科教育适应性的逻辑意蕴——基于CAS理论的视角

职业本科教育是跨界教育。跨界属性造就了职业本科教育的复杂性,复杂性催生了职业本科教育的适应性,适应性推动了职业本科教育的守正创新。适应性创新是铸就职业本科教育高质量发展的重要驱动力。基于CAS理论,提出职业本科教育适应性的本质逻辑是为职业本科教育打造一种跨界、集聚、融合、涌现、创新的社会契约。鉴于增强职业本科教育适应性还面临“办学历史较短,办学规模较小;院校类型复杂,办学实力不足;产教融合不够,政策执行掣肘;办学经费不足,影响适应发展”等现实困境,从多样性、互动性、涌现性、自组织性等维度,提出了增强职业本科教育适应性的策略建议。

A Method for Reducing Ocean Wave-Induced Magnetic Noises in Shallow-Water MT Data Using a Complex Adaptive Filter

In shallow-water areas,the marine magnetotelluric(MT)method faces a challenge in the investigation of seabed conductivity structures due to electrical and magnetic noises induced by ocean waves,which seriously contaminate MT data.Ocean waves can affect electric and magnetic fields to different extents.In general,their influence on magnetic fields is considerably greater than that on electric fields.In this paper,a complex adaptive filter is adopted to reduce wave-induced magnetic noises in the frequency domain.The processing results of synthetic and measured MT data indicate that the proposed method can effectively reduce wave-induced magnetic noises and provide reliable apparent resistivity and phase data.

Adaptive generalized matrix projective lag synchronization between two different complex networks with non-identical nodes and different dimensions

The adaptive generalized matrix projective lag synchronization between two different complex networks with non-identical nodes and different dimensions is investigated in this paper. Based on Lyapunov stability theory and Barbalat＇s lemma, generalized matrix projective lag synchronization criteria are derived by using the adaptive control method. Furthermore, each network can be undirected or directed, connected or disconnected, and nodes in either network may have identical or different dynamics. The proposed strategy is applicable to almost all kinds of complex networks. In addition, numerical simulation results are presented to illustrate the effectiveness of this method, showing that the synchronization speed is sensitively influenced by the adaptive law strength, the network size, and the network topological structure.

Robust adaptive synchronization of uncertain and delayed dynamical complex networks with faulty network

This paper presents a new robust adaptive synchronization method for a class of uncertain dynamical complex networks with network failures and coupling time-varying delays. Adaptive schemes are proposed to adjust controller parameters for the faulty network compensations, as well as to estimate the upper and lower bounds of delayed state errors and perturbations to compensate the effects of delay and perturbation on-line without assuming symmetry or irreducibility of networks. It is shown that, through Lyapunov stability theory, distributed adaptive controllers con- structed by the adaptive schemes are successful in ensuring the achievement of asymptotic synchronization of networks in the present of faulty and delayed networks, and perturbation inputs. A Chua＇s circuit network example is finally given to show the effectiveness of the proposed synchronization criteria.

Adaptive function projective synchronization of uncertain complex dynamical networks with disturbance

We investigate the problem of function projective synchronization （FPS） in drive–response dynamical networks with non-identical nodes. An adaptive controller is proposed for the FPS of complex dynamical networks with uncertain parameters and disturbance. Not only are the unknown parameters of the networks estimated by the adaptive laws obtained from the Lyapunov stability theory and Taylor expansions, but the unknown bounded disturbances are also simultaneously conquered by the proposed control. Finally, a numerical simulation is provided to illustrate the feasibility and effectiveness of the obtained result.

Decentralized adaptive synchronization of an uncertain complex delayed dynamical network

In this paper, we investigate the locally and globally adaptive synchronization problem for an uncertain complex dynamical network with time-varying coupling delays based on the decentralized control. The coupling terms here are bounded by high-order polynomials with known gains that are ubiquitous in a large class of complex dynamical networks. We generalize the usual technology of searching for an appropriate coordinates transformation to change the network dynamics into a series of decoupled lower-dimensional systems. Several adaptive synchronization criteria are derived by constructing the Lyapunov-Krasovskii functional and Barbalat lemma, and the proposed criteria are simple in form and convenient for the practical engineering design. Numerical simulations illustrated by a nearest-neighbor coupling network verify the effectiveness of the proposed synchronization scheme.

Theory and model of water resources complex adaptive allocation system

Complex adaptive sys tem theory is a new and important embranchment of system science, which prov ides a new thought to research water resources allocation system. Based on the a nalysis of complexity and complex adaptive mechanism of water resources allocat ion system, a fire-new analysis model is presented in this paper. With t he description of dynamical mechanism of system, behavior characters of agents and the evaluation method of system status, an integrity research system is built to analyse the evolvement rule of water resources allocation system. A nd a brief research for the impact of water resources allocation in benefi cial regions of the Water Transfer from South to North China Project is conducted.

Dynamic modeling and aperiodically intermittent strategy for adaptive finite-time synchronization control of the multi-weighted complex transportation networks with multiple delays

The idea of network splitting according to time delay and weight is introduced.Based on the cyber physical systems(CPS),a class of multi-weighted complex transportation networks with multiple delays is modeled.The finite-time synchronization of the proposed complex transportation networks model is studied systematically.On the basis of the theory of stability,the technique of adaptive control,aperiodically intermittent control and finite-time control,the aperiodically intermittent adaptive finite-time synchronization controller is designed.The controller designed in this paper is beneficial for understanding the synchronization in multi-weighted complex transportation networks with multiple delays.In addition,the conditions for the existence of finite time synchronization have been discussed in detail.And the specific value of the settling finite time for synchronization is obtained.Moreover,the outer coupling configuration matrices are not required to be irreducible or symmetric.Finally,simulation results of the finite-time synchronization problem are given to illustrate the correctness of the results obtained.

Linguistic Reflection on the Online Catchword of Appreciation —From the Perspective of Usage-based Language Model and Complex Adaptive System Theory

Language is a special social phenomenon and is always on the changing process with the development of society. During the evolving process of language, new language varieties will continuously emerge due to the changes of some social and cultural factors. Cyber language is universally accepted as one type of the social language varieties. Basically, cyber language can be treated as a complex adaptive system which is influenced by the interaction between users’ cognition, social culture and the surrounding environments. Thus it is safe to say that cyber language is always undergoing a dynamic evolving process. With the usage-based language model as the theoretical foundation, this paper proposes a Complex Adaptive System (CAS) approach to analyze the expression of Appreciation to explore the complex, dynamic and nonlinear development of cyber language from the angle of meaning construction, grammaticalization and functional adaption respectively. It is found that the expression of Appreciation is experiencing adaptively a semantic connotations development and a process of grammatical functions expansion as well. This paper suggests that the emergence and development of cyber language is a novel and trendy social language phenomenon. Network language can achieve its process and evolution under the huge impact of social changes and social promotions. When faced with the changing surroundings, cyber language itself enjoys a timely adaption and responsive development to keep up with the new environments, which reflects the basic principle of language development, namely, language changes with the development of society.

CAS理论下的职业教育强适应性EMSV指标模型研究

增强职业教育适应性,仅有愿景描绘是不够的,亟需进一步标定“强”的靶向,用以指明“增强”路向,为此构建强适应性概念框架。创新之处在于突破“群氓”运动思维,从CAS理论视角重新审视强适应性生成逻辑,提出“强”的能量蕴含在多元适应性主体趋利而聚的复杂适应系统中,是利益共同体内部合作势能在时间轴上从隐性量变到显性质变并最终集中爆发的涌现结果这一全新观点,并创建EMSV指标模型这一强适应性概念框架:激励(E)为各方聚集提供支点,动力(M)是夯实利益共同体的基础,合力(S)是合作剩余的现实呈现,活力(V)是合力的双向持续表达。EMSV指标模型为“增强职业教育适应性”提供新理念、总原则、元规则、主标准的路向指引。

气候模式CAS-ESM-C对1月热带太平洋流场模态的模拟评估

利用气候模式CAS-ESM-C从1922年起84年的模拟资料,对1月份热带太平洋上层流场作复EOF分解及小波分析,并与实况以及理论解析解作对比讨论,以考察模式对赤道大洋上层流场的模拟能力,得到主要结论:(1)复EOF分解前3个模态的方差贡献为53.5%、12.9%、9.5%,累积方差贡献为75.9%,累积方差贡献比实况更高。(2)第一、二模态空间场与实况相比总体相像,流场都为赤道所俘获,在俘获区内的流场均以偏纬向流为主;差异在于模拟资料分析的赤道俘获区范围较实况要大,流场的经向流分量及越赤道流也较实况明显。(3)第一、二模态实时间系数序列无线性变化趋势,而实况则有。复EOF模态年际及年代际变化与实况相同或相近;第一、二模态中3~7年的年际变化是厄尔尼诺与南方涛动(ENSO)的反映;第一模态22~23年的年代际变化受北太平洋主要气候模态北太平洋年代际振荡(PDO)对热带太平洋的影响,而第二模态13年的年代际变化是受北太平洋次要气候模态北太平洋环流振荡(NPGO)对热带太平洋的影响;第一、二模态还都有峰值16年的年代际变化,这可能与印尼穿越流有关。(4)模拟资料分析的结果具有理论解析解中流场为赤道所俘获及流场为纬向流的特点,只是解析解中因无风应力强迫,流场呈纯纬向流。(5)第一(二)模态在赤道太平洋东部(中部)有海温动力异常,并可称之为东(中)部型ENSO模态。气候模式CAS-ESM-C对热带太平洋上层流场的模拟表现较佳。