Reinforcement Learning Behavioral Control for Nonlinear Autonomous System

Behavior-based autonomous systems rely on human intelligence to resolve multi-mission conflicts by designing mission priority rules and nonlinear controllers.In this work,a novel twolayer reinforcement learning behavioral control(RLBC)method is proposed to reduce such dependence by trial-and-error learning.Specifically,in the upper layer,a reinforcement learning mission supervisor(RLMS)is designed to learn the optimal mission priority.Compared with existing mission supervisors,the RLMS improves the dynamic performance of mission priority adjustment by maximizing cumulative rewards and reducing hardware storage demand when using neural networks.In the lower layer,a reinforcement learning controller(RLC)is designed to learn the optimal control policy.Compared with existing behavioral controllers,the RLC reduces the control cost of mission priority adjustment by balancing control performance and consumption.All error signals are proved to be semi-globally uniformly ultimately bounded(SGUUB).Simulation results show that the number of mission priority adjustment and the control cost are significantly reduced compared to some existing mission supervisors and behavioral controllers,respectively.

Multi-agent reinforcement learning behavioral control for nonlinear second-order systems

Reinforcement learning behavioral control(RLBC)is limited to an individual agent without any swarm mission,because it models the behavior priority learning as a Markov decision process.In this paper,a novel multi-agent reinforcement learning behavioral control(MARLBC)method is proposed to overcome such limitations by implementing joint learning.Specifically,a multi-agent reinforcement learning mission supervisor(MARLMS)is designed for a group of nonlinear second-order systems to assign the behavior priorities at the decision layer.Through modeling behavior priority switching as a cooperative Markov game,the MARLMS learns an optimal joint behavior priority to reduce dependence on human intelligence and high-performance computing hardware.At the control layer,a group of second-order reinforcement learning controllers are designed to learn the optimal control policies to track position and velocity signals simultaneously.In particular,input saturation constraints are strictly implemented via designing a group of adaptive compensators.Numerical simulation results show that the proposed MARLBC has a lower switching frequency and control cost than finite-time and fixed-time behavioral control and RLBC methods.

Behavioral control task supervisor with memory based on reinforcement learning for human-multi-robot coordination systems

In this study,a novel reinforcement learning task supervisor(RLTS)with memory in a behavioral control framework is proposed for human–multi-robot coordination systems(HMRCSs).Existing HMRCSs suffer from high decision-making time cost and large task tracking errors caused by repeated human intervention,which restricts the autonomy of multi-robot systems(MRSs).Moreover,existing task supervisors in the null-space-based behavioral control(NSBC)framework need to formulate many priority-switching rules manually,which makes it difficult to realize an optimal behavioral priority adjustment strategy in the case of multiple robots and multiple tasks.The proposed RLTS with memory provides a detailed integration of the deep Q-network(DQN)and long short-term memory(LSTM)knowledge base within the NSBC framework,to achieve an optimal behavioral priority adjustment strategy in the presence of task conflict and to reduce the frequency of human intervention.Specifically,the proposed RLTS with memory begins by memorizing human intervention history when the robot systems are not confident in emergencies,and then reloads the history information when encountering the same situation that has been tackled by humans previously.Simulation results demonstrate the effectiveness of the proposed RLTS.Finally,an experiment using a group of mobile robots subject to external noise and disturbances validates the effectiveness of the proposed RLTS with memory in uncertain real-world environments.

Behavior Measurement Model Based on Prediction and Control of Trusted Network

In order to construct the trusted network and realize the trust of network behavior,a new multi-dimensional behavior measurement model based on prediction and control is presented.By using behavior predictive equation,individual similarity function,group similarity function,direct trust assessment function,and generalized predictive control,this model can guarantee the trust of an end user and users in its network.Compared with traditional measurement model,the model considers different characteristics of various networks.The trusted measurement policies established according to different network environments have better adaptability.By constructing trusted group,the threats to trusted group will be reduced greatly.Utilizing trusted group to restrict individuals in network can ensure the fault tolerance of trustworthiness of trusted individuals and group.The simulation shows that this scheme can support behavior measurement more efficiently than traditional ones and the model resists viruses and Trojans more efficiently than older ones.

Multi-boid Flocking with Formation Control Using Local Sensing and Communication

Behavior-based flocking has got remarkable attention in the recent past. The flocking algorithms can have inherent properties like organizing,healing and re-configuring for a distributed system. In this research we presented the emergent flocking behavior-based control. We defined the basis behavior and with variety of combination, and obtained a complex group behavior flocking. Unlike classical flocking, we implemented additional rules obstacle avoidance,formation and seek target which results in V-formation flocking while avoiding obstacles. We performed the visual simulation of our flocking algorithm using MATLAB. The results concluded that the multi-boid flock could successfully navigate to the target while avoiding collisions. This can be applied to areas where we need to maximize the coverage of sensors or minimize the risk of combative attack,both in military and civilian scenarios.

Highly Efficient and Stable Hybrid White Organic Light Emitting Diodes with Controllable Exciton Behavior by a Mixed Bipolar Interlayer

Highly efficient and stable hybrid white organic light-emitting diodes （HWOLEDs） with a mixed bipolar interlayer between fluorescent blue and phosphorescent yellow emitting layers are demonstrated. The bipolar interlayer is a mixture of p-type diphenyl （l0-phenyl-lOH-spiro [acridine-9,9＇-fluoren]-3Lyl） phosphine oxide and n-type 2＇,2- （1,3,5-benzinetriyl）-tris（1-phenyl-l-H-benzimidazole）. The electroluminance and Commission Internationale de l＇Eclairage （CIE1931） coordinates＇ characteristics can be modulated easily by adjusting the ratio of the hole- predominated material to the electron-predominated material in the interlayer. The hybrid WOLED with a p-type：n-type ratio of 1：3 shows a maximum current efficiency and power efficiency of 61.1 ed/A and 55.8 lm/W, respectively, with warm white CIE coordinates of （0.34, 0.43）. The excellent efficiency and adaptive CIE coordi- nates are attributed to the mixed interlayer with improved charge carrier balance, optimized exciton distribution, and enhanced harvesting of singlet and triplet excitons.

Control of Fracture Behavior of Carbon Fiber/Pitch-based CarbonMatrix Composites with Microspace Modification Concept

Theoretical consideration was conducted on a relation between pore diameter and interfacialarea between pores and fibers when pores uniforinly distribute in C/C composites. It was shownthat bonding at the fiber/matrix interface apparently decreased with decreasing a pore diameter,and consequently a new idea of microspace modification concept was proposed for controllingfracture behavior of C/C composites. Four types of C/C composites with various pore structureswere fabricated by hot-pressing, and their fracture behavior was investigated by three pointbending tests. The fracture behavior of the C/C composites was changed from brittle one topseudo ductile one with decreasing the pore diameter. This result supported the validity of themicrospace modification concept proposed in this paper.

Neural substrates of behavioral inhibitory control during the two-choice oddball task:functional neuroimaging evidence

Background:Behavioral inhibitory control(BIC)depicts a cognitive function of inhibiting inappropriate dominant responses to meet the context requirement.Despite abundant research into neural substrates of BIC during the go/no-go and stop signal tasks,these tasks were consistently shown hard to isolate neural processes of response inhibition,which is of primary interest,from those of response generation.Therefore,it is necessary to explore neural substrates of BIC using the two-choice oddball(TCO)task,whose design of dual responses is thought to produce an inhibition effect free of the confounds of response generation.Objective:The current study aims at depicting neural substrates of performing behavioral inhibitory control in the two-choice oddball task,which designs dual responses to balance response generation.Also,neural substrates of performing BIC during this task are compared with those in the go/no-go task,which designs a motor response in a single condition.Methods:The present study integrated go/no-go(GNG)and TCO tasks into a new Three-Choice BIC paradigm,which consists of stan-dard(75%),deviant(12.5%),and no-go(12.5%)conditions simultaneously.Forty-eight college students participated in this experiment,which required them to respond to standard(frequent)and deviant stimuli by pressing different keys,while inhibiting motor response to no-go stimuli.Conjunction analysis and ROI(region of interest)analysis were adopted to identify the unique neural mechanisms that subserve the processes of BIC.Results:Both tasks are effective in assessing BIC function,reflected by the significantly lower accuracy of no-go compared to standard condition in GNG,and the significantly lower accuracy and longer reaction time of deviant compared to standard condition in TCO.However,there were no significant differences between deviant and no-go conditions in accuracy.Moreover,functional neuroimaging has demonstrated that the anterior cingulate cortex(ACC)activation was observed for no-go vs.standard contrast in the GNG task,but not in deviant vs.standard contrast in the TCO task,suggesting that ACC involvement is not a necessary component of BIC.Second,ROI analysis of areas that were co-activated in TCO and GNG showed co-activations in the right inferior frontal cortex(triangle and orbital),with the signals in the TCO task significantly higher than those in the GNG task.Conclusions:These findings show that the designed responses to both standard and deviant stimuli in the TCO task,compared to the GNG task,produced a more prominent prefrontal inhibitory processing and extinguished an unnecessary component of ACC activation during BIC.This implies that prefrontal involvement,but not that of ACC,is mandatory for the successful performance of inhibiting prepotent behaviors.

A sensor-based motion planner: Situated-Bug

Propose a new sensor-based motion planning approach of Situated-Bug, which is composed of goal-oriented behavior, boundary following behavior and goal-oriented obstacle avoidance behavior, which are based on fuzzy control. The situated-Bug selects its behaviors according to robot orientation, instead of positions and hit points like other Bug algorithms, and its convergence proves robust to sensor noise, and it can guide the robots running for long rang traverse. At the same time, the design of the Situated-Bug is presented. Simulation results show that the approach is effective and practical.

Frequency-Agile WLAN Notch UWB Antenna for URLLC Applications

This paper introduces a compact dual notched UWB antenna with an independently controllable WLAN notched band integrated with fixed WiMAX band-notch.The proposed antenna utilizes a slot resonator placed in the main radiator of the antenna for fixed WiMAX band notch,while an inverted L-shaped resonator in the partial ground plane for achieving frequency agility within WLAN notched band.The inverted L-shaped resonator is also loaded with fixed and variable capacitors to control and adjust the WLAN notch.The WLAN notched band can be controlled independently with a wide range of tunability without disturbing the WiMAX bandnotch performance.Step by step design approach of the proposed antenna is discussed and the corresponding mathematical analysis of the proposed resonators are provided in both cases.Simulation of the proposed antenna is performed utilizing commercially available 3D-EM simulator,Ansoft High Frequency Structure Simulator(HFSS).The proposed antenna has high selectivity with experimental validation in terms of reflection coefficient,radiation characteristics,antenna gain,and percentage radiation efficiency.The corresponding measured frequency response of the input port corresponds quite well with the calculations and simulations in both cases.The proposed antenna is advantageous and can adjust according to the device requirements and be one of the attractive candidates for overlay cognitive radio UWB applications and URLLC service in 5G tactile internet.The proposed multifunctional antenna can also be used for wireless vital signs monitoring,sensing applications,and microwave imaging techniques.

Encapsulation of MnS Nanocrystals into N,S-Co-doped Carbon as Anode Material for Full Cell Sodium-Ion Capacitors

Sodium-ion capacitors(SICs)have received increasing interest for grid stationary energy storage application due to their affordability,high power,and energy densities.The major challenge for SICs is to overcome the kinetics imbalance between faradaic anode and nonfaradaic cathode.To boost the Na+reaction kinetics,the present work demonstrated a high-rate MnS-based anode by embedding the MnS nanocrystals into the N,S-co-doped carbon matrix(MnS@NSC).Benefiting from the fast pseudocapacitive Na+storage behavior,the resulting composite exhibits extraordinary rate capability(205.6 mAh g−1 at 10 A g−1)and outstanding cycling stability without notable degradation after 2000 cycles.A prototype SIC was demonstrated using MnS@NSC anode and N-doped porous carbon(NC)cathode;the obtained hybrid SIC device can display a high energy density of 139.8 Wh kg−1 and high power density of 11,500 W kg−1,as well as excellent cyclability with 84.5%capacitance retention after 3000 cycles.The superior electrochemical performance is contributed to downsizing of MnS and encapsulation of conductive N,S-co-doped carbon matrix,which not only promote the Na+and electrons transport,but also buffer the volume variations and maintain the structure integrity during Na+insertion/extraction,enabling its comparable fast reaction kinetics and cyclability with NC cathode.

A Review for Model Plant Mismatch Measures in Process Monitoring

Model is usually necessary for the design of a control loop. Due to simplification and unknown dynamics, model plant mismatch is inevitable in the control loop. In process monitoring, detection of mismatch and evaluation of its influences are demanded. In this paper several mismatch measures are presented based on different model descriptions. They are categorized into different groups from different perspectives and their potential in detection and diagnosis is evaluated. Two case studies on mixing process and distillation process demonstrate the efficacy of the framework of mismatch monitoring.

Security enhancement of artificial neural network using physically transient form of heterogeneous memristors with tunable resistive switching behaviors

As a critical command center in organisms,the brain can execute multiple intelligent interactions through neural networks,including memory,learning and cognition.Recently,memristive-based neuromorphic devices have been widely developed as promising technologies to build artificial synapses and neurons for neural networks.However,multiple information interactions in artificial intelligence devices potentially pose threats to information security.Herein,a transient form of heterogeneous memristor with a stacked structure of Ag/MgO/SiN_(x)/W is proposed,in which both the reconfigurable resistive switching behavior and volatile threshold switching characteristics could be realized by adjusting the thickness of the SiN_(x)layer.The underlying resistive switching mechanism of the device was elucidated in terms of filamentary and interfacial effects.Representative neural functions,including short-term plasticity(STP),the transformation from STP to long-term plasticity,and integrate-and-fire neuron functions,have been successfully emulated in memristive devices.Moreover,the dissolution kinetics associated with underlying transient behaviors were explored,and the water-assisted transfer printing technique was exploited to build transient neuromorphic device arrays on the water-dissolvable poly(vinyl alcohol)substrate,which were able to formless disappear in deionized water after 10-s dissolution at room temperature.This transient form of memristive-based neuromorphic device provides an important step toward information security reinforcement for artificial neural network applications.

Factors influencing physician's behavioral intention to use Traditional Chinese Medicine to treat coronavirus disease 2019 based on the theory of planned behavior

OBJECTIVE:To explore the factors influencing physicians’intentions to use Traditional Chinese Medicine(TCM)to treat coronavirus disease 2019(COVID-19).METHODS:A cross-sectional,self-report online survey was conducted from March 16,2020,to April 2,2020,in China.Participants were recruited through convenience and snowball sampling.Data were collected by using a self-designed questionnaire based on the Theory of Planned Behavior.Structural equation modeling was used for data analysis.RESULTS:A total of 494 physicians were enrolled in this study.Overall,the model explained 75.4%and 75.5%of the total variance in intention and attitude,respectively.Specifically,attitudes(β=0.467,P<0.001),past behavior(β=0.384,P<0.05),subjective norms(SN)(β=0.177,P<0.001),and perceived behavioral control(PBC)(β=0.133,P<0.05)significantly affected physicians’intention to use TCM.Cognition(β=0.606,P<0.001)and PBC(β=0.569,P<0.01)significantly influenced physicians’attitudes toward using TCM.SN(β=0.064,P=0.263)was not a factor affecting attitude.CONCLUSION:Physicians’intention to use TCM was significantly associated with attitude,past behavior,PBC,and SN.The findings may not only be useful for understanding the influencing factors and paths of physicians’intention to use TCM to treat COVID-19 but also provide a reference for health authorities and policymakers to promote physicians to utilize TCM.

Identifying the key purchase factors for organic food among Chinese consumers

The current study primarily aims to identify the critical purchase factors that affect Chinese consumer purchase intention and purchase decision with regard to organic food consumption,in accordance with a modified theory of planned behavior and the alphabet theory.Specifically,this study builds a conceptual research framework by which to delve into the relationships between purchase factors and purchase intention,and elucidate the mediating roles of purchase factors in the relationships between purchase intention and purchase decision.Moreover,by leveraging a modified theory of planned behavior and the alphabet theory,the current study also determines the critical roles of subjective norms and reveals the information and knowledge that impact consumer attitude toward the purchase of organic food.The current study leverages the purposive sampling method and captures 310 records within Beijing,China.The results indicate that purchase attitude correlates positively with subjective norms and knowledge,while purchase intention correlates positively with purchase attitude,perceived behavior control,and food therapy culture.Furthermore,purchase intention can significantly mediate relationships between each of purchase attitude,perceived behavior control,food therapy culture,and purchase decision.Finally,we discuss the theoretical and practical significance of the framework,and propose subsequent research directions regarding organic food purchase behavior.

Rational designed isostructural MOF for the charge-discharge behavior study of super capacitors

In recent years,the rapid charge-discharge property of super capacitors based on metal-organic frameworks(MOFs)has seen excellent applications in energy storage equipment.However,the purposeful design of high-performance electrodes for MOFderived super capacitors is still an urgent problem that needs to be solved.Herein,we rationally design and prepare three MOFs with the same crystal configuration and controllable functional groups.Through the combination of rigorous experiment and calculation,we have verified the effects of the specific surface area of the electrode material as well as the binding energy between the electrode material and the electrolyte ions on the performance of the super capacitor.This work not only extends the application of MOFs,but also provides a model-material platform for the study of charge–discharge behavior of MOF-based super capacitors,creating a way of thinking for the selection and design of MOF materials for energy storage applications.

Technology improvements and management innovations in construction of Xiluodu hydropower station on Jinsha River

Developer and owner:China Three Gorges Corporation(CTG)Engineering management:China Three Gorges Projects Development Corporation(CTGPC)Designer:Chengdu Engineering Corporation Co.,Ltd.,Power

Fuzzy Logic Based Behavior Fusion for Navigation of an Intelligent Mobile Robot

This paper presents a new method for behavior fusion control of a mobile robot in uncertain environments. Using behavior fusion by fuzzy logic, a mobile robot is able to directly execute its motion according to range information about environments, acquired by ultrasonic sensorst without the need for trajectory planning. Based on low-level behavior control, an efficient strategy for integrating high-level global planning for robot motion can be formulated, since,in most applications, some information on environments is prior knowledge. Aglobal planner, therefore, only needs to generate some subgoal positions ratherthan exact geometric paths. Because such subgoals can be easily removed from or added into the planner, this. strategy reduces computational time for globalplanning and is flekible for replanning in dynamic environments. Simulationresults demonstrate that the proposed strategy can be applied to robot motion in complex and dynamic environments.

An applied multi-agent system within the framework of a player-centered probabilistic computer game

The probabilistic model to represent the behavior of an applied multi-agent system that introduces the interaction between a set of agents and Player has been developed within the framework of player-centered probabilistic computer games.The approach features are given with the aid of a game developed for testing cognitive abilities.The agent’s behavior is nondeterministic and therefore unpredictable from Player viewpoint.The system allows both coordinated and autonomous agent’s behavior that depends on availability of information about the presence and position of workable agents for each other.Agent’s behavior is determined with the aid of the algorithm that includes identification of the probabilistic model parameters using maximized objective functions representing individual and group probabilities for Player defeating.Both the model and algorithm ensure the behavior control for relevant applied multi-agent systems.

Forecasting macro parameters representing the behavior of an applied multi-agent system

Under consideration is the method of probabilistic forecasting of macro parameters representing general patterns of an applied multi-agent system behavior.This system introduces the game interaction between a set of agents and a target.The behavior dynamics of such a system is described by a discrete-state discrete-time Markov random process and represented in the terms,which are considered convenient for interpretation and practical control of macro parameters.The corresponding calculations are based on common characteristics of the initial conditions.The probabilistic model of behavior of the system is generalized for the case of the mobile target and supplemented with formulas for the dynamical calculation of probabilistic distributions of the target’s defeat by the agents as well as the agent’s defeat by the target.