Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

A Multi-Task Deep Learning Framework for Simultaneous Detection of Thoracic Pathology through Image Classification

Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.

A Robust Approach for Multi Classification-Based Intrusion Detection through Stacking Deep Learning Models

Intrusion detection is a predominant task that monitors and protects the network infrastructure.Therefore,many datasets have been published and investigated by researchers to analyze and understand the problem of intrusion prediction and detection.In particular,the Network Security Laboratory-Knowledge Discovery in Databases(NSL-KDD)is an extensively used benchmark dataset for evaluating intrusion detection systems(IDSs)as it incorporates various network traffic attacks.It is worth mentioning that a large number of studies have tackled the problem of intrusion detection using machine learning models,but the performance of these models often decreases when evaluated on new attacks.This has led to the utilization of deep learning techniques,which have showcased significant potential for processing large datasets and therefore improving detection accuracy.For that reason,this paper focuses on the role of stacking deep learning models,including convolution neural network(CNN)and deep neural network(DNN)for improving the intrusion detection rate of the NSL-KDD dataset.Each base model is trained on the NSL-KDD dataset to extract significant features.Once the base models have been trained,the stacking process proceeds to the second stage,where a simple meta-model has been trained on the predictions generated from the proposed base models.The combination of the predictions allows the meta-model to distinguish different classes of attacks and increase the detection rate.Our experimental evaluations using the NSL-KDD dataset have shown the efficacy of stacking deep learning models for intrusion detection.The performance of the ensemble of base models,combined with the meta-model,exceeds the performance of individual models.Our stacking model has attained an accuracy of 99%and an average F1-score of 93%for the multi-classification scenario.Besides,the training time of the proposed ensemble model is lower than the training time of benchmark techniques,demonstrating its efficiency and robustness.

Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning

The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.

Distributed Graph Database Load Balancing Method Based on Deep Reinforcement Learning

This paper focuses on the scheduling problem of workflow tasks that exhibit interdependencies.Unlike indepen-dent batch tasks,workflows typically consist of multiple subtasks with intrinsic correlations and dependencies.It necessitates the distribution of various computational tasks to appropriate computing node resources in accor-dance with task dependencies to ensure the smooth completion of the entire workflow.Workflow scheduling must consider an array of factors,including task dependencies,availability of computational resources,and the schedulability of tasks.Therefore,this paper delves into the distributed graph database workflow task scheduling problem and proposes a workflow scheduling methodology based on deep reinforcement learning(DRL).The method optimizes the maximum completion time(makespan)and response time of workflow tasks,aiming to enhance the responsiveness of workflow tasks while ensuring the minimization of the makespan.The experimental results indicate that the Q-learning Deep Reinforcement Learning(Q-DRL)algorithm markedly diminishes the makespan and refines the average response time within distributed graph database environments.In quantifying makespan,Q-DRL achieves mean reductions of 12.4%and 11.9%over established First-fit and Random scheduling strategies,respectively.Additionally,Q-DRL surpasses the performance of both DRL-Cloud and Improved Deep Q-learning Network(IDQN)algorithms,with improvements standing at 4.4%and 2.6%,respectively.With reference to average response time,the Q-DRL approach exhibits a significantly enhanced performance in the scheduling of workflow tasks,decreasing the average by 2.27%and 4.71%when compared to IDQN and DRL-Cloud,respectively.The Q-DRL algorithm also demonstrates a notable increase in the efficiency of system resource utilization,reducing the average idle rate by 5.02%and 9.30%in comparison to IDQN and DRL-Cloud,respectively.These findings support the assertion that Q-DRL not only upholds a lower average idle rate but also effectively curtails the average response time,thereby substantially improving processing efficiency and optimizing resource utilization within distributed graph database systems.

Task offloading mechanism based on federated reinforcement learning in mobile edge computing

With the arrival of 5G,latency-sensitive applications are becoming increasingly diverse.Mobile Edge Computing(MEC)technology has the characteristics of high bandwidth,low latency and low energy consumption,and has attracted much attention among researchers.To improve the Quality of Service(QoS),this study focuses on computation offloading in MEC.We consider the QoS from the perspective of computational cost,dimensional disaster,user privacy and catastrophic forgetting of new users.The QoS model is established based on the delay and energy consumption and is based on DDQN and a Federated Learning(FL)adaptive task offloading algorithm in MEC.The proposed algorithm combines the QoS model and deep reinforcement learning algorithm to obtain an optimal offloading policy according to the local link and node state information in the channel coherence time to address the problem of time-varying transmission channels and reduce the computing energy consumption and task processing delay.To solve the problems of privacy and catastrophic forgetting,we use FL to make distributed use of multiple users’data to obtain the decision model,protect data privacy and improve the model universality.In the process of FL iteration,the communication delay of individual devices is too large,which affects the overall delay cost.Therefore,we adopt a communication delay optimization algorithm based on the unary outlier detection mechanism to reduce the communication delay of FL.The simulation results indicate that compared with existing schemes,the proposed method significantly reduces the computation cost on a device and improves the QoS when handling complex tasks.

Multi-Agent Deep Deterministic Policy Gradien-Based Task Offloading Resource Allocation Joint Offloading

With the advancement of technology and the continuous innovation of applications, low-latency applications such as drones, online games and virtual reality are gradually becoming popular demands in modern society. However, these applications pose a great challenge to the traditional centralized mobile cloud computing paradigm, and it is obvious that the traditional cloud computing model is already struggling to meet such demands. To address the shortcomings of cloud computing, mobile edge computing has emerged. Mobile edge computing provides users with computing and storage resources by offloading computing tasks to servers at the edge of the network. However, most existing work only considers single-objective performance optimization in terms of latency or energy consumption, but not balanced optimization in terms of latency and energy consumption. To reduce task latency and device energy consumption, the problem of joint optimization of computation offloading and resource allocation in multi-cell, multi-user, multi-server MEC environments is investigated. In this paper, a dynamic computation offloading algorithm based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to obtain the optimal policy. The experimental results show that the algorithm proposed in this paper reduces the delay by 5 ms compared to PPO, 1.5 ms compared to DDPG and 10.7 ms compared to DQN, and reduces the energy consumption by 300 compared to PPO, 760 compared to DDPG and 380 compared to DQN. This fully proves that the algorithm proposed in this paper has excellent performance.

Machine Learning Assisted Design of Natural Rubber Composites with Multi⁃Performance Optimization

Multi⁃performance optimization of tread rubber composites is a key issue of great concern in automotive industry.Traditional experimental design approach via“trial and error”or intuition is ineffective due to mutual inhibition among multiple properties.A“Uniform design⁃Machine learning”strategy for performance prediction and multi⁃performance optimization of tread rubber composites was proposed.The wear resistance,rolling resistance,tensile strength and wet skid resistance were simultaneously optimized.A series of feasible optimization designs were screened via statistical analysis and machine learning analysis,and were experimentally prepared.The verification experiments demonstrate that the optimization design via machine learning analysis meets the optimization requirements of all target performance,especially for Akron abrasion and 60℃tanδ(about 21%and 9%lower than the design targets,respectively)due to the inhibition of mechanical degradation and good dispersion of fillers.

Multi-Agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

Due to the fading characteristics of wireless channels and the burstiness of data traffic,how to deal with congestion in Ad-hoc networks with effective algorithms is still open and challenging.In this paper,we focus on enabling congestion control to minimize network transmission delays through flexible power control.To effectively solve the congestion problem,we propose a distributed cross-layer scheduling algorithm,which is empowered by graph-based multi-agent deep reinforcement learning.The transmit power is adaptively adjusted in real-time by our algorithm based only on local information(i.e.,channel state information and queue length)and local communication(i.e.,information exchanged with neighbors).Moreover,the training complexity of the algorithm is low due to the regional cooperation based on the graph attention network.In the evaluation,we show that our algorithm can reduce the transmission delay of data flow under severe signal interference and drastically changing channel states,and demonstrate the adaptability and stability in different topologies.The method is general and can be extended to various types of topologies.

Multi-tasking to Address Diversity in Language Learning

With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately in varied contexts and with different uses of the language. To attain this, the teacher is tasked with designing, monitoring and processing language learning activities for students to carry out and in the process learn by doing and reflecting on the learning process they went through as they interacted socially with each other. This paper describes a task named"The Fishbowl Technique"and found to be effective in large ESL classes in the secondary level in the Philippines.

Multi-task Coalition Parallel Formation Strategy Based on Reinforcement Learning

代理人联盟是代理人协作和合作的一种重要方式。形成一个联盟，代理人能提高他们的能力解决问题并且获得更多的实用程序。在这份报纸，新奇多工联盟平行形成策略被介绍，并且多工联盟形成的过程是一个 Markov 决定过程的结论理论上被证明。而且，学习的加强被用来解决多工联盟平行的代理人行为策略，和这个过程形成被描述。在多工面向的领域，策略罐头有效地并且平行形式多工联盟。

A Distributed Algorithm for Parallel Multi-task Allocation Based on Profit Sharing Learning

经由联盟形成的任务分配是在多代理人系统(妈) 的几应用程序域的基本研究挑战，例如资源分配，灾难反应管理等等。怎么以一种分布式的方式分配许多未解决的任务到一些代理人，主要处理。在这篇论文，我们在自我组织、自我学习的代理人之中建议一个分布式的平行多工分配算法。处理状况，我们在二维的房间地理上驱散代理人和任务，然后介绍为寻找它的任务由的一个单个代理人的分享学习的利润(PSL ) 不断自我学习。我们也在代理人之中为通讯和协商介绍策略分配真实工作量到每个 tasked 代理人。最后，评估建议算法的有效性，我们把它与 Shehory 和 Krau 被许多研究人员在最近的年里讨论的分布式的任务分配算法作比较。试验性的结果证明建议算法罐头快速为每项任务形成一个解决的联盟。而且，建议算法罐头明确地告诉我们每个 tasked 代理人的真实工作量，并且能因此为实际控制任务提供一本特定、重要的参考书。

Multi-Task Learning for Semantic Relatedness and Textual Entailment

Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task supervised learning and do not consider the correlation between the tasks. Based on this observation, in this paper, we implemented a multi-task learning model to joint learn two related NLP tasks simultaneously and conducted experiments to evaluate if learning these tasks jointly can improve the system performance compared with learning them individually. In addition, a comparison of our model with the state-of-the-art learning models, including multi-task learning, transfer learning, unsupervised learning and feature based traditional machine learning models is presented. This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task.

基于Q-learning的多业务网络选择博弈策略

为了增加网络吞吐量并改善用户体验,提出一种基于Q学习(Q-learning)的多业务网络选择博弈(Multi-Service Network Selection Game based on Q-learning,QSNG)策略。该策略通过模糊推理和综合属性评估获得多业务网络效用函数,并将其用作Q-learning的奖励。用户通过博弈算法预测网络选择策略收益,避免访问负载较重的网络。同时,使用二进制指数退避算法减少多个用户并发访问某个网络的概率。仿真结果表明,所提策略可以根据用户的QoS需求和价格偏好自适应地切换到最合适的网络,将其与基于强化学习的网络辅助反馈(Reinforcement Learning with Network-Assisted Feedback,RLNF)策略和无线网络选择博弈(Radio Network Selection Games,RSG)策略相比,所提策略可以分别减少总切换数量的80%和60%,使网络吞吐量分别提高了7%和8%,并且可以保证系统的公平性。

Q-Learning-Based Pesticide Contamination Prediction in Vegetables and Fruits

Pesticides have become more necessary in modern agricultural production.However,these pesticides have an unforeseeable long-term impact on people's wellbeing as well as the ecosystem.Due to a shortage of basic pesticide exposure awareness,farmers typically utilize pesticides extremely close to harvesting.Pesticide residues within foods,particularly fruits as well as veggies,are a significant issue among farmers,merchants,and particularly consumers.The residual concentrations were far lower than these maximal allowable limits,with only a few surpassing the restrictions for such pesticides in food.There is an obligation to provide a warning about this amount of pesticide use in farming.Previous technologies failed to forecast the large number of pesticides that were dangerous to people,necessitating the development of improved detection and early warning systems.A novel methodology for verifying the status and evaluating the level of pesticides in regularly consumed veggies as well as fruits has been identified,named as the Hybrid Chronic Multi-Residual Framework(HCMF),in which the harmful level of used pesticide residues has been predicted for contamination in agro products using Q-Learning based Recurrent Neural Network and the predicted contamination levels have been analyzed using Complex Event Processing(CEP)by processing given spatial and sequential data.The analysis results are used to minimize and effectively use pesticides in the agricultural field and also ensure the safety of farmers and consumers.Overall,the technique is carried out in a Python environment,with the results showing that the proposed model has a 98.57%accuracy and a training loss of 0.30.

On the Task-based Collaborative Learning

Task-based language teaching(TBLT) has been a prevalent teaching practice in the TEFL field in the recent years and its momentum for striving to be the legitimate one has never ceased. The present study tries to provide a theoretical foundation for its application in the communicative learning approach of English as the second language(ESL),namely the collaborative learning mode.

Task-based Teaching and Learning in English Listening Class

In technical college English listening class,task-based teaching and learning method can not only create harmonious environment for students' learning,but also motivate students' enthusiasm in listening class,thus students can benefit a great deal in listening class and the listening can be carried out successfully.

Remote Sensing Data Processing Process Scheduling Based on Reinforcement Learning in Cloud Environment

Task scheduling plays a crucial role in cloud computing and is a key factor determining cloud computing performance.To solve the task scheduling problem for remote sensing data processing in cloud computing,this paper proposes a workflow task scheduling algorithm—Workflow Task Scheduling Algorithm based on Deep Reinforcement Learning(WDRL).The remote sensing data process modeling is transformed into a directed acyclic graph scheduling problem.Then,the algorithm is designed by establishing a Markov decision model and adopting a fitness calculation method.Finally,combine the advantages of reinforcement learning and deep neural networks to minimize make-time for remote sensing data processes from experience.The experiment is based on the development of CloudSim and Python and compares the change of completion time in the process of remote sensing data.The results showthat compared with several traditionalmeta-heuristic scheduling algorithms,WDRL can effectively achieve the goal of optimizing task scheduling efficiency.

Covid-19 Diagnosis Using a Deep Learning Ensemble Model with Chest X-Ray Images

Covid-19 is a deadly virus that is rapidly spread around the world towards the end of the 2020.The consequences of this virus are quite frightening,especially when accompanied by an underlying disease.The novelty of the virus,the constant emergence of different variants and its rapid spread have a negative impact on the control and treatment process.Although the new test kits provide almost certain results,chest X-rays are extremely important to detect the progression and degree of the disease.In addition to the Covid-19 virus,pneumonia and harmless opacity of the lungs also complicate the diagnosis.Considering the negative results caused by the virus and the treatment costs,the importance of fast and accurate diagnosis is clearly seen.In this context,deep learning methods appear as an extremely popular approach.In this study,a hybrid model design with superior properties of convolutional neural networks is presented to correctly classify the Covid-19 disease.In addition,in order to contribute to the literature,a suitable dataset with balanced case numbers that can be used in all artificial intelligence classification studies is presented.With this ensemble model design,quite remarkable results are obtained for the diagnosis of three and four-class Covid-19.The proposed model can classify normal,pneumonia,and Covid-19 with 92.6%accuracy and 82.6%for normal,pneumonia,Covid-19,and lung opacity.

Exploring Deep Reinforcement Learning with Multi Q-Learning

Q-learning is a popular temporal-difference reinforcement learning algorithm which often explicitly stores state values using lookup tables. This implementation has been proven to converge to the optimal solution, but it is often beneficial to use a function-approximation system, such as deep neural networks, to estimate state values. It has been previously observed that Q-learning can be unstable when using value function approximation or when operating in a stochastic environment. This instability can adversely affect the algorithm’s ability to maximize its returns. In this paper, we present a new algorithm called Multi Q-learning to attempt to overcome the instability seen in Q-learning. We test our algorithm on a 4 × 4 grid-world with different stochastic reward functions using various deep neural networks and convolutional networks. Our results show that in most cases, Multi Q-learning outperforms Q-learning, achieving average returns up to 2.5 times higher than Q-learning and having a standard deviation of state values as low as 0.58.