Collision-free parking recommendation based on multi-agent reinforcement learning in vehicular crowdsensing

The recent proliferation of Fifth-Generation(5G)networks and Sixth-Generation(6G)networks has given rise to Vehicular Crowd Sensing(VCS)systems which solve parking collisions by effectively incentivizing vehicle participation.However,instead of being an isolated module,the incentive mechanism usually interacts with other modules.Based on this,we capture this synergy and propose a Collision-free Parking Recommendation(CPR),a novel VCS system framework that integrates an incentive mechanism,a non-cooperative VCS game,and a multi-agent reinforcement learning algorithm,to derive an optimal parking strategy in real time.Specifically,we utilize an LSTM method to predict parking areas roughly for recommendations accurately.Its incentive mechanism is designed to motivate vehicle participation by considering dynamically priced parking tasks and social network effects.In order to cope with stochastic parking collisions,its non-cooperative VCS game further analyzes the uncertain interactions between vehicles in parking decision-making.Then its multi-agent reinforcement learning algorithm models the VCS campaign as a multi-agent Markov decision process that not only derives the optimal collision-free parking strategy for each vehicle independently,but also proves that the optimal parking strategy for each vehicle is Pareto-optimal.Finally,numerical results demonstrate that CPR can accomplish parking tasks at a 99.7%accuracy compared with other baselines,efficiently recommending parking spaces.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

An Adaptive Program Recommendation System for Multi-User Sharing Environment

More and more accounts or devices are shared by multiple users in video applications,which makes it difficult to provide recommendation service.Existing recommendation schemes overlook multiuser sharing scenarios,and they cannot make effective use of the mixed information generated by multi-user when exploring users’potential interests.To solve these problems,this paper proposes an adaptive program recommendation system for multi-user sharing environment.Specifically,we first design an offline periodic identification module by building multi-user features and periodically predicting target user in future sessions,which can separate the profile of target user from mixed log records.Subsequently,an online recommendation module with adaptive timevarying exploration strategy is constructed by jointly using personal information and multi-user social information provided by identification module.On one hand,to learn the dynamic changes in user-interest,a time-varying linear upper confidence bound(LinUCB)based on personal information is designed.On the other hand,to reduce the risk of exploration,a timeinvariant LinUCB based on separated multi-user social information from one account/device is proposed to compute the quality scores of programs for each user,which is integrated into the time-varying LinUCB by cross-weighting strategy.Finally,experimental results validate the efficiency of the proposed scheme.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

CoopAI-Route: DRL Empowered Multi-Agent Cooperative System for Efficient QoS-Aware Routing for Network Slicing in Multi-Domain SDN

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.

Improving Diversity with Multi-Loss Adversarial Training in Personalized News Recommendation

Users’interests are often diverse and multi-grained,with their underlying intents even more so.Effectively captur-ing users’interests and uncovering the relationships between diverse interests are key to news recommendation.Meanwhile,diversity is an important metric for evaluating news recommendation algorithms,as users tend to reject excessive homogeneous information in their recommendation lists.However,recommendation models themselves lack diversity awareness,making it challenging to achieve a good balance between the accuracy and diversity of news recommendations.In this paper,we propose a news recommendation algorithm that achieves good performance in both accuracy and diversity.Unlike most existing works that solely optimize accuracy or employ more features to meet diversity,the proposed algorithm leverages the diversity-aware capability of the model.First,we introduce an augmented user model to fully capture user intent and the behavioral guidance they might undergo as a result.Specifically,we focus on the relationship between the original clicked news and the augmented clicked news.Moreover,we propose an effective adversarial training method for diversity(AT4D),which is a pluggable component that can enhance both the accuracy and diversity of news recommendation results.Extensive experiments on real-world datasets confirm the efficacy of the proposed algorithm in improving both the accuracy and diversity of news recommendations.

Urban Traffic Control Meets Decision Recommendation System:A Survey and Perspective

Urban traffic control is a multifaceted and demanding task that necessitates extensive decision-making to ensure the safety and efficiency of urban transportation systems.Traditional approaches require traffic signal professionals to manually intervene on traffic control devices at the intersection level,utilizing their knowledge and expertise.However,this process is cumbersome,labor-intensive,and cannot be applied on a large network scale.Recent studies have begun to explore the applicability of recommendation system for urban traffic control,which offer increased control efficiency and scalability.Such a decision recommendation system is complex,with various interdependent components,but a systematic literature review has not yet been conducted.In this work,we present an up-to-date survey that elucidates all the detailed components of a recommendation system for urban traffic control,demonstrates the utility and efficacy of such a system in the real world using data and knowledgedriven approaches,and discusses the current challenges and potential future directions of this field.

Recommendation Method for Contrastive Enhancement of Neighborhood Information

Knowledge graph can assist in improving recommendation performance and is widely applied in various person-alized recommendation domains.However,existing knowledge-aware recommendation methods face challenges such as weak user-item interaction supervisory signals and noise in the knowledge graph.To tackle these issues,this paper proposes a neighbor information contrast-enhanced recommendation method by adding subtle noise to construct contrast views and employing contrastive learning to strengthen supervisory signals and reduce knowledge noise.Specifically,first,this paper adopts heterogeneous propagation and knowledge-aware attention networks to obtain multi-order neighbor embedding of users and items,mining the high-order neighbor informa-tion of users and items.Next,in the neighbor information,this paper introduces weak noise following a uniform distribution to construct neighbor contrast views,effectively reducing the time overhead of view construction.This paper then performs contrastive learning between neighbor views to promote the uniformity of view information,adjusting the neighbor structure,and achieving the goal of reducing the knowledge noise in the knowledge graph.Finally,this paper introduces multi-task learning to mitigate the problem of weak supervisory signals.To validate the effectiveness of our method,experiments are conducted on theMovieLens-1M,MovieLens-20M,Book-Crossing,and Last-FM datasets.The results showthat compared to the best baselines,our method shows significant improvements in AUC and F1.

The use of oral human immunodeficiency virus pre-exposure prophylaxis in pregnant and lactating women in sub-Saharan Africa:considerations,barriers,and recommendations

In sub-Saharan Africa(SSA),63%of new human immunodeficiency virus(HIV)infections in 2021 were among women,particularly adolescent girls,and young women.There is a high incidence of HIV among pregnant and lactating women(PLW)in SSA.It is estimated that the risk of HIV-acquisition during pregnancy and the postpartum period more than doubles.In this article,we discuss the safety and effectiveness of drugs used for oral HIV pre-exposure prophylaxis(PrEP),considerations for initiating PrEP in PLW,the barriers to initiating and adhering to PrEP among them and suggest recommendations to address these barriers.Tenofovir/emtricitabine,the most widely used combination in SSA,is safe,clinically effective,and cost-effective among PLW.Any PLW who requests PrEP and has no medical contraindications should receive it.PrEP users who are pregnant or lactating may experience barriers to starting and adhering for a variety of reasons,including personal,pill-related,and healthcare facility-related issues.To address the barriers,we recommend an increased provision of information on PrEP to the women and the communities,increasing and/or facilitating access to PrEP among the PLW,and developing strategies to increase adherence.

Recommendation System Based on Perceptron and Graph Convolution Network

The relationship between users and items,which cannot be recovered by traditional techniques,can be extracted by the recommendation algorithm based on the graph convolution network.The current simple linear combination of these algorithms may not be sufficient to extract the complex structure of user interaction data.This paper presents a new approach to address such issues,utilizing the graph convolution network to extract association relations.The proposed approach mainly includes three modules:Embedding layer,forward propagation layer,and score prediction layer.The embedding layer models users and items according to their interaction information and generates initial feature vectors as input for the forward propagation layer.The forward propagation layer designs two parallel graph convolution networks with self-connections,which extract higher-order association relevance from users and items separately by multi-layer graph convolution.Furthermore,the forward propagation layer integrates the attention factor to assign different weights among the hop neighbors of the graph convolution network fusion,capturing more comprehensive association relevance between users and items as input for the score prediction layer.The score prediction layer introduces MLP(multi-layer perceptron)to conduct non-linear feature interaction between users and items,respectively.Finally,the prediction score of users to items is obtained.The recall rate and normalized discounted cumulative gain were used as evaluation indexes.The proposed approach effectively integrates higher-order information in user entries,and experimental analysis demonstrates its superiority over the existing algorithms.

Bipartite consensus problems of Lurie multi-agent systems over signed graphs: A contraction approach

This paper examines the bipartite consensus problems for the nonlinear multi-agent systems in Lurie dynamics form with cooperative and competitive communication between different agents. Based on the contraction theory, some new conditions for the nonlinear Lurie multi-agent systems reaching bipartite leaderless consensus and bipartite tracking consensus are presented. Compared with the traditional methods, this approach degrades the dimensions of the conditions, eliminates some restrictions of the system matrix, and extends the range of the nonlinear function. Finally, two numerical examples are provided to illustrate the efficiency of our results.

Hyperbolic Tangent Function-Based Protocols for Global/Semi-Global Finite-Time Consensus of Multi-Agent Systems

This paper investigates the problem of global/semi-global finite-time consensus for integrator-type multi-agent sys-tems.New hyperbolic tangent function-based protocols are pro-posed to achieve global and semi-global finite-time consensus for both single-integrator and double-integrator multi-agent systems with leaderless undirected and leader-following directed commu-nication topologies.These new protocols not only provide an explicit upper-bound estimate for the settling time,but also have a user-prescribed bounded control level.In addition,compared to some existing results based on the saturation function,the pro-posed approach considerably simplifies the protocol design and the stability analysis.Illustrative examples and an application demonstrate the effectiveness of the proposed protocols.

Designing Proportional-Integral Consensus Protocols for Second-Order Multi-Agent Systems Using Delayed and Memorized State Information

This paper is concerned with consensus of a secondorder linear time-invariant multi-agent system in the situation that there exists a communication delay among the agents in the network.A proportional-integral consensus protocol is designed by using delayed and memorized state information.Under the proportional-integral consensus protocol,the consensus problem of the multi-agent system is transformed into the problem of asymptotic stability of the corresponding linear time-invariant time-delay system.Note that the location of the eigenvalues of the corresponding characteristic function of the linear time-invariant time-delay system not only determines the stability of the system,but also plays a critical role in the dynamic performance of the system.In this paper,based on recent results on the distribution of roots of quasi-polynomials,several necessary conditions for Hurwitz stability for a class of quasi-polynomials are first derived.Then allowable regions of consensus protocol parameters are estimated.Some necessary and sufficient conditions for determining effective protocol parameters are provided.The designed protocol can achieve consensus and improve the dynamic performance of the second-order multi-agent system.Moreover,the effects of delays on consensus of systems of harmonic oscillators/double integrators under proportional-integral consensus protocols are investigated.Furthermore,some results on proportional-integral consensus are derived for a class of high-order linear time-invariant multi-agent systems.

Combined CNN-LSTM Deep Learning Algorithms for Recognizing Human Physical Activities in Large and Distributed Manners:A Recommendation System

Recognizing human activity(HAR)from data in a smartphone sensor plays an important role in the field of health to prevent chronic diseases.Daily and weekly physical activities are recorded on the smartphone and tell the user whether he is moving well or not.Typically,smartphones and their associated sensing devices operate in distributed and unstable environments.Therefore,collecting their data and extracting useful information is a significant challenge.In this context,the aimof this paper is twofold:The first is to analyze human behavior based on the recognition of physical activities.Using the results of physical activity detection and classification,the second part aims to develop a health recommendation system to notify smartphone users about their healthy physical behavior related to their physical activities.This system is based on the calculation of calories burned by each user during physical activities.In this way,conclusions can be drawn about a person’s physical behavior by estimating the number of calories burned after evaluating data collected daily or even weekly following a series of physical workouts.To identify and classify human behavior our methodology is based on artificial intelligence models specifically deep learning techniques like Long Short-Term Memory(LSTM),stacked LSTM,and bidirectional LSTM.Since human activity data contains both spatial and temporal information,we proposed,in this paper,to use of an architecture allowing the extraction of the two types of information simultaneously.While Convolutional Neural Networks(CNN)has an architecture designed for spatial information,our idea is to combine CNN with LSTM to increase classification accuracy by taking into consideration the extraction of both spatial and temporal data.The results obtained achieved an accuracy of 96%.On the other side,the data learned by these algorithms is prone to error and uncertainty.To overcome this constraint and improve performance(96%),we proposed to use the fusion mechanisms.The last combines deep learning classifiers tomodel non-accurate and ambiguous data to obtain synthetic information to aid in decision-making.The Voting and Dempster-Shafer(DS)approaches are employed.The results showed that fused classifiers based on DS theory outperformed individual classifiers(96%)with the highest accuracy level of 98%.Also,the findings disclosed that participants engaging in physical activities are healthy,showcasing a disparity in the distribution of physical activities between men and women.

Diversified and compatible web APIs recommendation based on game theory in IoT

With the ever-increasing popularity of Internet of Things(IoT),massive enterprises are attempting to encapsulate their developed outcomes into various lightweight Web Application Programming Interfaces(APIs)that can be accessible remotely.In this context,finding and writing a list of existing Web APIs that can collectively meet the functional needs of software developers has become a promising approach to economically and easily develop successful mobile applications.However,the number and diversity of candidate IoT Web APIs places an additional burden on application developers’Web API selection decisions,as it is often a challenging task to simultaneously ensure the diversity and compatibility of the final set of Web APIs selected.Considering this challenge and latest successful applications of game theory in IoT,a Diversified and Compatible Web APIs Recommendation approach,namely DivCAR,is put forward in this paper.First of all,to achieve API diversity,DivCAR employs random walk sampling technique on a pre-built“API-API”correlation graph to generate diverse“API-API”correlation subgraphs.Afterwards,with the diverse“API-API”correlation subgraphs,the compatible Web APIs recommendation problem is modeled as a minimum group Steiner tree search problem.A sorted set of multiple compatible and diverse Web APIs are returned to the application developer by solving the minimum group Steiner tree search problem.At last,a set of experiments are designed and implemented on a real dataset crawled from www.programmableweb.com.Experimental results validate the effectiveness and efficiency of our proposed DivCAR approach in balancing the Web APIs recommendation diversity and compatibility.