Task Offloading and Resource Allocation in NOMA-VEC:A Multi-Agent Deep Graph Reinforcement Learning Algorithm

Vehicular edge computing(VEC)is emerging as a promising solution paradigm to meet the requirements of compute-intensive applications in internet of vehicle(IoV).Non-orthogonal multiple access(NOMA)has advantages in improving spectrum efficiency and dealing with bandwidth scarcity and cost.It is an encouraging progress combining VEC and NOMA.In this paper,we jointly optimize task offloading decision and resource allocation to maximize the service utility of the NOMA-VEC system.To solve the optimization problem,we propose a multiagent deep graph reinforcement learning algorithm.The algorithm extracts the topological features and relationship information between agents from the system state as observations,outputs task offloading decision and resource allocation simultaneously with local policy network,which is updated by a local learner.Simulation results demonstrate that the proposed method achieves a 1.52%∼5.80%improvement compared with the benchmark algorithms in system service utility.

Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

A Multi-Task Deep Learning Framework for Simultaneous Detection of Thoracic Pathology through Image Classification

Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.

Distributed Graph Database Load Balancing Method Based on Deep Reinforcement Learning

This paper focuses on the scheduling problem of workflow tasks that exhibit interdependencies.Unlike indepen-dent batch tasks,workflows typically consist of multiple subtasks with intrinsic correlations and dependencies.It necessitates the distribution of various computational tasks to appropriate computing node resources in accor-dance with task dependencies to ensure the smooth completion of the entire workflow.Workflow scheduling must consider an array of factors,including task dependencies,availability of computational resources,and the schedulability of tasks.Therefore,this paper delves into the distributed graph database workflow task scheduling problem and proposes a workflow scheduling methodology based on deep reinforcement learning(DRL).The method optimizes the maximum completion time(makespan)and response time of workflow tasks,aiming to enhance the responsiveness of workflow tasks while ensuring the minimization of the makespan.The experimental results indicate that the Q-learning Deep Reinforcement Learning(Q-DRL)algorithm markedly diminishes the makespan and refines the average response time within distributed graph database environments.In quantifying makespan,Q-DRL achieves mean reductions of 12.4%and 11.9%over established First-fit and Random scheduling strategies,respectively.Additionally,Q-DRL surpasses the performance of both DRL-Cloud and Improved Deep Q-learning Network(IDQN)algorithms,with improvements standing at 4.4%and 2.6%,respectively.With reference to average response time,the Q-DRL approach exhibits a significantly enhanced performance in the scheduling of workflow tasks,decreasing the average by 2.27%and 4.71%when compared to IDQN and DRL-Cloud,respectively.The Q-DRL algorithm also demonstrates a notable increase in the efficiency of system resource utilization,reducing the average idle rate by 5.02%and 9.30%in comparison to IDQN and DRL-Cloud,respectively.These findings support the assertion that Q-DRL not only upholds a lower average idle rate but also effectively curtails the average response time,thereby substantially improving processing efficiency and optimizing resource utilization within distributed graph database systems.

Task offloading mechanism based on federated reinforcement learning in mobile edge computing

With the arrival of 5G,latency-sensitive applications are becoming increasingly diverse.Mobile Edge Computing(MEC)technology has the characteristics of high bandwidth,low latency and low energy consumption,and has attracted much attention among researchers.To improve the Quality of Service(QoS),this study focuses on computation offloading in MEC.We consider the QoS from the perspective of computational cost,dimensional disaster,user privacy and catastrophic forgetting of new users.The QoS model is established based on the delay and energy consumption and is based on DDQN and a Federated Learning(FL)adaptive task offloading algorithm in MEC.The proposed algorithm combines the QoS model and deep reinforcement learning algorithm to obtain an optimal offloading policy according to the local link and node state information in the channel coherence time to address the problem of time-varying transmission channels and reduce the computing energy consumption and task processing delay.To solve the problems of privacy and catastrophic forgetting,we use FL to make distributed use of multiple users’data to obtain the decision model,protect data privacy and improve the model universality.In the process of FL iteration,the communication delay of individual devices is too large,which affects the overall delay cost.Therefore,we adopt a communication delay optimization algorithm based on the unary outlier detection mechanism to reduce the communication delay of FL.The simulation results indicate that compared with existing schemes,the proposed method significantly reduces the computation cost on a device and improves the QoS when handling complex tasks.

多无人机辅助边缘计算场景下基于Q-learning的任务卸载优化

引入多无人机辅助边缘计算系统,由多个无人机和原有边缘服务器共同为移动用户提供通信和计算资源;将优化问题建模为资源竞争和卸载决策约束下的系统总效用最大化问题,系统总效用由用户满意度、任务延迟和系统能耗3个因素组成.由于优化模型是一个具有NP难属性的非凸问题,故采用强化学习方法求解得到使系统总效用最大的最优任务卸载决策集.仿真实验结果表明,与贪心顺序调优卸载方案和随机选择卸载方案相比,该文提出的Q-learning方案的系统总效用分别提高了15%和43%以上.

Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning

The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.

Multi-tasking to Address Diversity in Language Learning

With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately in varied contexts and with different uses of the language. To attain this, the teacher is tasked with designing, monitoring and processing language learning activities for students to carry out and in the process learn by doing and reflecting on the learning process they went through as they interacted socially with each other. This paper describes a task named"The Fishbowl Technique"and found to be effective in large ESL classes in the secondary level in the Philippines.

On the Task-based Collaborative Learning

Task-based language teaching(TBLT) has been a prevalent teaching practice in the TEFL field in the recent years and its momentum for striving to be the legitimate one has never ceased. The present study tries to provide a theoretical foundation for its application in the communicative learning approach of English as the second language(ESL),namely the collaborative learning mode.

Task-based Teaching and Learning in English Listening Class

In technical college English listening class,task-based teaching and learning method can not only create harmonious environment for students' learning,but also motivate students' enthusiasm in listening class,thus students can benefit a great deal in listening class and the listening can be carried out successfully.

Remote Sensing Data Processing Process Scheduling Based on Reinforcement Learning in Cloud Environment

Task scheduling plays a crucial role in cloud computing and is a key factor determining cloud computing performance.To solve the task scheduling problem for remote sensing data processing in cloud computing,this paper proposes a workflow task scheduling algorithm—Workflow Task Scheduling Algorithm based on Deep Reinforcement Learning(WDRL).The remote sensing data process modeling is transformed into a directed acyclic graph scheduling problem.Then,the algorithm is designed by establishing a Markov decision model and adopting a fitness calculation method.Finally,combine the advantages of reinforcement learning and deep neural networks to minimize make-time for remote sensing data processes from experience.The experiment is based on the development of CloudSim and Python and compares the change of completion time in the process of remote sensing data.The results showthat compared with several traditionalmeta-heuristic scheduling algorithms,WDRL can effectively achieve the goal of optimizing task scheduling efficiency.

Deep reinforcement learning for UAV swarm rendezvous behavior

The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the main trends of UAV development in the future.This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network(DDQN)algorithm.We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning(DRL)for the long period task.We also propose the concept of temporary storage area,optimizing the memory playback unit of the traditional DDQN algorithm,improving the convergence speed of the algorithm,and speeding up the training process of the algorithm.Different from traditional task environment,this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment.Based on the DDQN algorithm,the collaborative tasks of UAV swarm in different task scenarios are trained.The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy,and improving the intelligence of UAV swarm collaborative task execution.The simulation results show that after training,the proposed UAV swarm can carry out the rendezvous task well,and the success rate of the mission reaches 90%.

Deep reinforcement learning-based optimization of lightweight task offloading for multi-user mobile edge computing

To improve the quality of computation experience for mobile devices,mobile edge computing(MEC)is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network,which supports both traditional communication and MEC services.However,this kind of intensive computing problem is a high dimensional NP hard problem,and some machine learning methods do not have a good effect on solving this problem.In this paper,the Markov decision process model is established to find the excellent task offloading scheme,which maximizes the long-term utility performance,so as to make the best offloading decision according to the queue state,energy queue state and channel quality between mobile users and BS.In order to explore the curse of high dimension in state space,a candidate network is proposed based on edge computing optimize offloading(ECOO)algorithm with the application of deep deterministic policy gradient algorithm.Through simulation experiments,it is proved that the ECOO algorithm is superior to some deep reinforcement learning algorithms in terms of energy consumption and time delay.So the ECOO is good at dealing with high dimensional problems.

Multi-task Coalition Parallel Formation Strategy Based on Reinforcement Learning

代理人联盟是代理人协作和合作的一种重要方式。形成一个联盟，代理人能提高他们的能力解决问题并且获得更多的实用程序。在这份报纸，新奇多工联盟平行形成策略被介绍，并且多工联盟形成的过程是一个 Markov 决定过程的结论理论上被证明。而且，学习的加强被用来解决多工联盟平行的代理人行为策略，和这个过程形成被描述。在多工面向的领域，策略罐头有效地并且平行形式多工联盟。

Promotion of interaction in cooperative learning task

How to promote interaction in cooperative learning tasks is discussed from a theoretical perspective in order to maximize the benefits of cooperative learning. A classroom instructional model is presented and examined to illustrate how successful and effective interaction is carried out to create the optimal conditions for second language acquisition.

A Distributed Algorithm for Parallel Multi-task Allocation Based on Profit Sharing Learning

经由联盟形成的任务分配是在多代理人系统(妈) 的几应用程序域的基本研究挑战，例如资源分配，灾难反应管理等等。怎么以一种分布式的方式分配许多未解决的任务到一些代理人，主要处理。在这篇论文，我们在自我组织、自我学习的代理人之中建议一个分布式的平行多工分配算法。处理状况，我们在二维的房间地理上驱散代理人和任务，然后介绍为寻找它的任务由的一个单个代理人的分享学习的利润(PSL ) 不断自我学习。我们也在代理人之中为通讯和协商介绍策略分配真实工作量到每个 tasked 代理人。最后，评估建议算法的有效性，我们把它与 Shehory 和 Krau 被许多研究人员在最近的年里讨论的分布式的任务分配算法作比较。试验性的结果证明建议算法罐头快速为每项任务形成一个解决的联盟。而且，建议算法罐头明确地告诉我们每个 tasked 代理人的真实工作量，并且能因此为实际控制任务提供一本特定、重要的参考书。

MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

Multi-Agent Deep Deterministic Policy Gradien-Based Task Offloading Resource Allocation Joint Offloading

With the advancement of technology and the continuous innovation of applications, low-latency applications such as drones, online games and virtual reality are gradually becoming popular demands in modern society. However, these applications pose a great challenge to the traditional centralized mobile cloud computing paradigm, and it is obvious that the traditional cloud computing model is already struggling to meet such demands. To address the shortcomings of cloud computing, mobile edge computing has emerged. Mobile edge computing provides users with computing and storage resources by offloading computing tasks to servers at the edge of the network. However, most existing work only considers single-objective performance optimization in terms of latency or energy consumption, but not balanced optimization in terms of latency and energy consumption. To reduce task latency and device energy consumption, the problem of joint optimization of computation offloading and resource allocation in multi-cell, multi-user, multi-server MEC environments is investigated. In this paper, a dynamic computation offloading algorithm based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to obtain the optimal policy. The experimental results show that the algorithm proposed in this paper reduces the delay by 5 ms compared to PPO, 1.5 ms compared to DDPG and 10.7 ms compared to DQN, and reduces the energy consumption by 300 compared to PPO, 760 compared to DDPG and 380 compared to DQN. This fully proves that the algorithm proposed in this paper has excellent performance.

Multi-Task Learning for Semantic Relatedness and Textual Entailment

Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task supervised learning and do not consider the correlation between the tasks. Based on this observation, in this paper, we implemented a multi-task learning model to joint learn two related NLP tasks simultaneously and conducted experiments to evaluate if learning these tasks jointly can improve the system performance compared with learning them individually. In addition, a comparison of our model with the state-of-the-art learning models, including multi-task learning, transfer learning, unsupervised learning and feature based traditional machine learning models is presented. This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task.