Mutual information oriented deep skill chaining for multi‐agent reinforcement learning

Multi‐agent reinforcement learning relies on reward signals to guide the policy networks of individual agents.However,in high‐dimensional continuous spaces,the non‐stationary environment can provide outdated experiences that hinder convergence,resulting in ineffective training performance for multi‐agent systems.To tackle this issue,a novel reinforcement learning scheme,Mutual Information Oriented Deep Skill Chaining(MioDSC),is proposed that generates an optimised cooperative policy by incorporating intrinsic rewards based on mutual information to improve exploration efficiency.These rewards encourage agents to diversify their learning process by engaging in actions that increase the mutual information between their actions and the environment state.In addition,MioDSC can generate cooperative policies using the options framework,allowing agents to learn and reuse complex action sequences and accelerating the convergence speed of multi‐agent learning.MioDSC was evaluated in the multi‐agent particle environment and the StarCraft multi‐agent challenge at varying difficulty levels.The experimental results demonstrate that MioDSC outperforms state‐of‐the‐art methods and is robust across various multi‐agent system tasks with high stability.

Similarity Intelligence:Similarity Based Reasoning,Computing,and Analytics

Similarity has been playing an important role in computer science,artificial intelligence(AI)and data science.However,similarity intelligence has been ignored in these disciplines.Similarity intelligence is a process of discovering intelligence through similarity.This article will explore similarity intelligence,similarity-based reasoning,similarity computing and analytics.More specifically,this article looks at the similarity as an intelligence and its impact on a few areas in the real world.It explores similarity intelligence accompanying experience-based intelligence,knowledge-based intelligence,and data-based intelligence to play an important role in computer science,AI,and data science.This article explores similarity-based reasoning(SBR)and proposes three similarity-based inference rules.It then examines similarity computing and analytics,and a multiagent SBR system.The main contributions of this article are:1)Similarity intelligence is discovered from experience-based intelligence consisting of data-based intelligence and knowledge-based intelligence.2)Similarity-based reasoning,computing and analytics can be used to create similarity intelligence.The proposed approach will facilitate research and development of similarity intelligence,similarity computing and analytics,machine learning and case-based reasoning.

Knowledge Reasoning Method Based on Deep Transfer Reinforcement Learning:DTRLpath

In recent years,with the continuous development of deep learning and knowledge graph reasoning methods,more and more researchers have shown great interest in improving knowledge graph reasoning methods by inferring missing facts through reasoning.By searching paths on the knowledge graph and making fact and link predictions based on these paths,deep learning-based Reinforcement Learning(RL)agents can demonstrate good performance and interpretability.Therefore,deep reinforcement learning-based knowledge reasoning methods have rapidly emerged in recent years and have become a hot research topic.However,even in a small and fixed knowledge graph reasoning action space,there are still a large number of invalid actions.It often leads to the interruption of RL agents’wandering due to the selection of invalid actions,resulting in a significant decrease in the success rate of path mining.In order to improve the success rate of RL agents in the early stages of path search,this article proposes a knowledge reasoning method based on Deep Transfer Reinforcement Learning path(DTRLpath).Before supervised pre-training and retraining,a pre-task of searching for effective actions in a single step is added.The RL agent is first trained in the pre-task to improve its ability to search for effective actions.Then,the trained agent is transferred to the target reasoning task for path search training,which improves its success rate in searching for target task paths.Finally,based on the comparative experimental results on the FB15K-237 and NELL-995 datasets,it can be concluded that the proposed method significantly improves the success rate of path search and outperforms similar methods in most reasoning tasks.

Smart Healthcare Activity Recognition Using Statistical Regression and Intelligent Learning

In this present time,Human Activity Recognition(HAR)has been of considerable aid in the case of health monitoring and recovery.The exploitation of machine learning with an intelligent agent in the area of health informatics gathered using HAR augments the decision-making quality and significance.Although many research works conducted on Smart Healthcare Monitoring,there remain a certain number of pitfalls such as time,overhead,and falsification involved during analysis.Therefore,this paper proposes a Statistical Partial Regression and Support Vector Intelligent Agent Learning(SPR-SVIAL)for Smart Healthcare Monitoring.At first,the Statistical Partial Regression Feature Extraction model is used for data preprocessing along with the dimensionality-reduced features extraction process.Here,the input dataset the continuous beat-to-beat heart data,triaxial accelerometer data,and psychological characteristics were acquired from IoT wearable devices.To attain highly accurate Smart Healthcare Monitoring with less time,Partial Least Square helps extract the dimensionality-reduced features.After that,with these resulting features,SVIAL is proposed for Smart Healthcare Monitoring with the help of Machine Learning and Intelligent Agents to minimize both analysis falsification and overhead.Experimental evaluation is carried out for factors such as time,overhead,and false positive rate accuracy concerning several instances.The quantitatively analyzed results indicate the better performance of our proposed SPR-SVIAL method when compared with two state-of-the-art methods.

A function-based behavioral modeling method for air combat simulation

Today’s air combat has reached a high level of uncertainty where continuous or discrete variables with crisp values cannot be properly represented using fuzzy sets. With a set of membership functions, fuzzy logic is well-suited to tackle such complex states and actions. However, it is not necessary to fuzzify the variables that have definite discrete semantics.Hence, the aim of this study is to improve the level of model abstraction by proposing multiple levels of cascaded hierarchical structures from the perspective of function, namely, the functional decision tree. This method is developed to represent behavioral modeling of air combat systems, and its metamodel,execution mechanism, and code generation can provide a sound basis for function-based behavioral modeling. As a proof of concept, an air combat simulation is developed to validate this method and the results show that the fighter Alpha built using the proposed framework provides better performance than that using default scripts.

Research on Negotiating Agent Development

The paper presents a flexible and effective method of development of negotiating agents. A strategy specification, which is specified by a state chart and defeasible rules, can be dynamically inserted into an agent shell incorporating a state chart interpreter and a defeasible logic inference engine, in order to yield a desirable agent. The set of desirable criteria and rules is required to be justified with different context of the application. Key words automatic negotiation - intelligence agent - defeasible logic - state chart CLC number TP 391 Foundation item: Supported by the National Natural Science Foundation of China (60173051)Biography: WEI Ding-guo (1964-), male, Associate professor, research direction: computer network protocal etc.

Optimization Management of Industrial Organizations Based on Performance Indicators

This paper proposes an intelligent management system (IMS) to help managers in their delicate and tedious task of exploiting the plethora of data (indicators) contained in management dashboards. This system is based on intelligent agents, ontologies and data mining. It is implemented by PASSI (Process for Agent Societies Specification and Implementation) methods for agent design and implementation, the Methodology for Knowledge Modeling and Hot-Winters for data prediction. Intelligent agents not only track indicators but also store the knowledge of managers within the company. Ontologies are used to manage the representation and presentation aspects of knowledge. Data mining makes it possible to: make the most of all available data;model the industrial process of data selection, exploration and modeling;and transform behaviors into predictive indicators. An instance of the IMS named SYGISS, currently in operation within a large brewery organization, allows us to observe very interesting results: the extraction of indicators is done in less than 5 minutes whereas manual extraction used to take 14 days;the generation of dashboards is instantaneous whereas it used to take 12 hours;the interpretation of indicators is instantaneous whereas it used to take a day;forecasts are possible and are done in less than 5 minutes whereas they did not exist with the old management. These important contributions help to optimize the management of this organization.

An Intelligent Agent Based on Virtual Geographic Environment System

On the basis of previous work, this paper designs an intelligent agent based on virtual geographic environment (VGE) system that is characterized by huge data, rapid computation, multi\|user, multi\|thread and intelligence and issues challenges to traditional GIS models and algorithms. The new advances in software and hardware technology lay a reliable basis for system design, development and application.

Design of an Intelligent Self-Healing Smart Grid using a Hybrid Multi-Agent Framework

This paper discusses the applications of a hybrid multi-agent framework for self-healing applications in an intelligent smart grid system following catastrophic disturbances such as loss of generators or during system fault.The proposed hybrid multi-agent framework is a hybrid of both centralized and decentralized scheme to allow distributed intelligent agent in the smart grid system to make fast local decision while allowing the slower central controller to judge the effectiveness of the decision made by the local agents and to suggest more optimal solutions.

Intelligent Agent-Based System for Digital Library Information Retrieval

A new information search model is reported and the design and implementation of a system based on intelligent agent is presented. The system is an assistant information retrieval system which helps users to search what they need. The system consists of four main components: interface agent, information retrieval agent, broker agent and learning agent. They collaborate to implement system functions. The agents apply learning mechanisms based on an improved ID3 algorithm.

Integration of Agent System with Legacy Software

Agent technique is a new method that can analyze, design and realize a distributed open system. It has been used in almost every field. But if act for the real practical words in technique, it must integrate with legacy software, such as database system etc, and control them. This paper introduces the specification of agent software integration, ontology, instances database as implementing agent software integration with CORBA technique and takes XML, ACL as language communicating among agents.

Multi-agent Systems and Cloud Computing for Controlling and Managing Chemical and Food Processes

Cloud computing can offer a very powerful, reliable, predictable and scalable computing infrastructure for the execution of MAS （multi-agent systems） implementing complex agent-based applications such when modelling, simulation and real-time running of complex systems must be provided. Multi-agent systems appears as an adequate approach to current challenges in many areas. Between important qualities of MAS also belongs to, that they are open, interoperable, and heterogenous systems. The agent is active, a program entity, has its own ideas how to perform the tasks of the own agenda. Agents： perceive, behave ＂reasonably＂, act in the environment, communicate with other agents. Cloud infrastructures can offer an ideal platform where run MAS systems simulations, applications and real-time running because of its large amount of processing and memory resources that can be dynamically configured for executing large agent-based software at unprecedented scale. Cloud computing can help chemical and food companies drive operational excellence; meet growing and changing customer demands; accelerate new product innovation and ramp-to-volume manufacturing in key markets; reduce IT spending; manage and mitigate supply chain risks; and enable faster and more flexible delivery of new IT system. Production type of SOC （service-oriented computing） can be inspired by a ＂Cloud＂, for the production of ＂Cloud＂ offers an attractive and natural solutions in several computing trends such as delivery system over the Internet, use of utilities, flexibility, virtualization, a ＂grid＂ distributed computing, outsourcing, Web 2.0, etc.. Production of the ＂Cloud＂ is also considered as a new multidisciplinary field that includes ＂network＂ production, virtual manufacturing, agile manufacturing, and of course cloud computing. Examples of cloud computing and MAS applications in food and chemistry development and industry, proposition of using multi-agent systems in the control of batch processes, modified ACO （ant colony optimization） approach for the diversified service allocation and scheduling mechanism in cloud paradigma, examples of applications in a business area were studied in the paper.

MULTI-AGENT COMPUTER AIDED ASSEMBLY PROCESS PLANNING SYSTEM FOR SHIP HULL

A multi agent computer aided assembly process planning system (MCAAPP) for ship hull is presented. The system includes system framework, global facilitator, the macro agent structure, agent communication language, agent oriented programming language, knowledge representation and reasoning strategy. The system can produce the technological file and technological quota, which can satisfy the production needs of factory.

Design and Application of an Intelligent Agent for Web Information Discovery

With the propagation of applications on the internet, the internet has become a great information source which supplies users with valuable information. But it is hard for users to quickly acquire the right information on the web. This paper an intelligent agent for internet applications to retrieve and extract web information under user's guidance. The intelligent agent is made up of a retrieval script to identify web sources, an extraction script based on the document object model to express extraction process, a data translator to export the extracted information into knowledge bases with frame structures, and a data reasoning to reply users' questions. A GUI tool named Script Writer helps to generate the extraction script visually, and knowledge rule databases help to extract wanted information and to generate the answer to questions.

Coordinated dynamic mission planning scheme for intelligent multi-agent systems

Mission planning was thoroughly studied in the areas of multiple intelligent agent systems,such as multiple unmanned air vehicles,and multiple processor systems.However,it still faces challenges due to the system complexity,the execution order constraints,and the dynamic environment uncertainty.To address it,a coordinated dynamic mission planning scheme is proposed utilizing the method of the weighted AND/OR tree and the AOE-Network.In the scheme,the mission is decomposed into a time-constraint weighted AND/OR tree,which is converted into an AOE-Network for mission planning.Then,a dynamic planning algorithm is designed which uses task subcontracting and dynamic re-decomposition to coordinate conflicts.The scheme can reduce the task complexity and its execution time by implementing real-time dynamic re-planning.The simulation proves the effectiveness of this approach.

Optimizing Alzheimer’s Disease Therapy Using a Neural Intelligent Agent-Based Platform

Alzheimer’s disease affects millions of persons every year. Negative emotions such as stress and frustration have a negative impact on memory function and Alzheimer's patients experience more negative emotions than healthy adults. Non-pharmacological treatment such as immersion in virtual environments could help Alzheimer patients by reducing their negative emotions, but it has restrictions and requirements. In this work, we present three virtual reality relaxing systems in which the patients are immersed in relaxing environments. We propose to use intelligent agents in order to adapt the relaxing environment to each participant and optimize its relaxation effect. The intelligent agents track the emotions of patients using electroencephalography as input in order to adapt the environments. We designed each system with different levels of intelligence in order to analyze the impact of the adaptation on the patients. Experiments were performed for each system on participants with subjective cognitive decline. Results show that these relaxing systems can reduce negative emotions and improve participants’ memory performance. The positive effects on affective state and memory persisted for a longer period of time and were generally more effective for the systems with more intelligence. We believe that the combination of a relaxing environment, virtual reality, intelligent agents for adapting the environment, and brain assessment is a promising method for helping Alzheimer’s patients.

Rumi: An Intelligent Agent Enhancing Learning Management Systems Using Machine Learning Techniques

The dramatic improvement of information and communication technology (ICT) has made an evolution in learning management systems (LMS). The rapid growth in LMSs has caused users to demand more advanced, automated, and intelligent services. This paper discusses how Artificial Intelligence and Machine Learning techniques are adopted to fulfill users’ needs in a social learning management system named “CourseNetworking”. The paper explains how machine learning contributed to developing an intelligent agent called “Rumi” as a personal assistant in CourseNetworking platform to add personalization, gamification, and more dynamics to the system. This paper aims to introduce machine learning to traditional learning platforms and guide the developers working in LMS field to benefit from advanced technologies in learning platforms by offering customized services.

Research on active defense decision-making method for cloud boundary networks based on reinforcement learning of intelligent agent

The cloud boundary network environment is characterized by a passive defense strategy,discrete defense actions,and delayed defense feedback in the face of network attacks,ignoring the influence of the external environment on defense decisions,thus resulting in poor defense effectiveness.Therefore,this paper proposes a cloud boundary network active defense model and decision method based on the reinforcement learning of intelligent agent,designs the network structure of the intelligent agent attack and defense game,and depicts the attack and defense game process of cloud boundary network;constructs the observation space and action space of reinforcement learning of intelligent agent in the non-complete information environment,and portrays the interaction process between intelligent agent and environment;establishes the reward mechanism based on the attack and defense gain,and encourage intelligent agents to learn more effective defense strategies.the designed active defense decision intelligent agent based on deep reinforcement learning can solve the problems of border dynamics,interaction lag,and control dispersion in the defense decision process of cloud boundary networks,and improve the autonomy and continuity of defense decisions.

Agents with four categories of understanding abilities and their role in two-stage(deep)emotional intelligence simulation

Understanding is the essence of any intelligent system.We revise our four machine understanding paradigms which are:(i)basic understanding,(ii)rich understanding,(iii)exploratory understanding,and(iv)theory-based understanding;and we elaborate on the first two of them.We then introduce the concept of two-stage(or deep)machine understanding which provides descriptive understandings,as well as evaluations of these understandings.After a brief systematization of emotions,we cover the following paradigms for agents with two-stage(deep)understanding abilities for emotional intelligence simulation:(i)basic understanding,(ii)rich-understanding,and(iii)switchable understanding.

e-Automation,an Architecture for Distributed Industrial Automation Systems

Due to the complexity of modern industrial systems, a conventional automation system is not capable of providing sufficient information management and high-level intelligent approaches, as achieving these functionalities requires the support of comprehensive data management and coordination between system devices and heterogenous information. This paper proposes the concept of e-Automation, in which computer networking and distributed intelligence agent technologies are applied to industrial automation systems, and presents a hardware and software architecture that implements this concept. An open infrastructure based on multi-agent systems is employed in the proposed architecture of e-Automation, which aims to allow the implementation of diverse tasks and to permit greater configurability than can be obtained from a traditional system. To evaluate our proposed e-Automation concept, this paper presents a case study of substation information management which adopts the proposed e-Automation architecture in power system domain.