Task Offloading and Resource Allocation in NOMA-VEC:A Multi-Agent Deep Graph Reinforcement Learning Algorithm

Vehicular edge computing(VEC)is emerging as a promising solution paradigm to meet the requirements of compute-intensive applications in internet of vehicle(IoV).Non-orthogonal multiple access(NOMA)has advantages in improving spectrum efficiency and dealing with bandwidth scarcity and cost.It is an encouraging progress combining VEC and NOMA.In this paper,we jointly optimize task offloading decision and resource allocation to maximize the service utility of the NOMA-VEC system.To solve the optimization problem,we propose a multiagent deep graph reinforcement learning algorithm.The algorithm extracts the topological features and relationship information between agents from the system state as observations,outputs task offloading decision and resource allocation simultaneously with local policy network,which is updated by a local learner.Simulation results demonstrate that the proposed method achieves a 1.52%∼5.80%improvement compared with the benchmark algorithms in system service utility.

Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

A Multi-Task Deep Learning Framework for Simultaneous Detection of Thoracic Pathology through Image Classification

Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.

High throughput screening of single atomic catalysts with optimized local structures for the electrochemical oxygen reduction by machine learning

Single atomic catalysts(SACs),especially metal-nitrogen doped carbon(M-NC)catalysts,have been extensively explored for the electrochemical oxygen reduction reaction(ORR),owing to their high activity and atomic utilization efficiency.However,there is still a lack of systematic screening and optimization of local structures surrounding active centers of SACs for ORR as the local coordination has an essential impact on their electronic structures and catalytic performance.Herein,we systematic study the ORR catalytic performance of M-NC SACs with different central metals and environmental atoms in the first and second coordination sphere by using density functional theory(DFT)calculation and machine learning(ML).The geometric and electronic informed overpotential model(GEIOM)based on random forest algorithm showed the highest accuracy,and its R^(2) and root mean square errors(RMSE)were 0.96 and 0.21,respectively.30 potential high-performance catalysts were screened out by GEIOM,and the RMSE of the predicted result was only 0.12 V.This work not only helps us fast screen high-performance catalysts,but also provides a low-cost way to improve the accuracy of ML models.

Identification of navigation characteristics of single otter trawl vessel using four machine learning models

Fishing logbook records the fishing behaviors and other information of fishing vessels.However,the accuracy of the recorded information is often difficult to guarantee due to the misreport and concealment.The fishing vessel monitoring system(VMS)can monitor and record the navigation information of fishing vessels in real time,and it may be used to improve the accuracy of identifying the state of fishing vessels.If the VMS data and fishing logbook are combined to establish their relationships,then the navigation characteristics and fishing behavior of fishing vessels can be more accurately identified.Therefore,first,a method for determining the state of VMS data points using fishing log data was proposed.Secondly,the relationship between VMS data and the different states of fishing vessels was further explored.Thirdly,the state of the fishing vessel was predicted using VMS data by building machine learning models.The speed,heading,longitude,latitude,and time as features from the VMS data were extracted by matching the VMS and logbook data of three single otter trawl vessels from September 2012 to January 2013,and four machine learning models were established,i.e.,Random Forest(RF),Adaptive Boosting(AdaBoost),K-Nearest Neighbor(KNN),and Gradient Boosting Decision Tree(GBDT)to predict the behavior of fishing vessels.The prediction performances of the models were evaluated by using normalized confusion matrix and receiver operator characteristic curve.Results show that the importance rankings of spatial(longitude and latitude)and time features were higher than those of speed and heading.The prediction performances of the RF and AdaBoost models were higher than those of the KNN and GBDT models.RF model showed the highest prediction performance for fishing state.Meanwhile,AdaBoost model exhibited the highest prediction performance for non-fishing state.This study offered a technical basis for judging the navigation characteristics of fishing vessels,which improved the algorithm for judging the behavior of fishing vessels based on VMS data,enhanced the prediction accuracy,and upgraded the fishery management being more scientific and efficient.

Distributed Graph Database Load Balancing Method Based on Deep Reinforcement Learning

This paper focuses on the scheduling problem of workflow tasks that exhibit interdependencies.Unlike indepen-dent batch tasks,workflows typically consist of multiple subtasks with intrinsic correlations and dependencies.It necessitates the distribution of various computational tasks to appropriate computing node resources in accor-dance with task dependencies to ensure the smooth completion of the entire workflow.Workflow scheduling must consider an array of factors,including task dependencies,availability of computational resources,and the schedulability of tasks.Therefore,this paper delves into the distributed graph database workflow task scheduling problem and proposes a workflow scheduling methodology based on deep reinforcement learning(DRL).The method optimizes the maximum completion time(makespan)and response time of workflow tasks,aiming to enhance the responsiveness of workflow tasks while ensuring the minimization of the makespan.The experimental results indicate that the Q-learning Deep Reinforcement Learning(Q-DRL)algorithm markedly diminishes the makespan and refines the average response time within distributed graph database environments.In quantifying makespan,Q-DRL achieves mean reductions of 12.4%and 11.9%over established First-fit and Random scheduling strategies,respectively.Additionally,Q-DRL surpasses the performance of both DRL-Cloud and Improved Deep Q-learning Network(IDQN)algorithms,with improvements standing at 4.4%and 2.6%,respectively.With reference to average response time,the Q-DRL approach exhibits a significantly enhanced performance in the scheduling of workflow tasks,decreasing the average by 2.27%and 4.71%when compared to IDQN and DRL-Cloud,respectively.The Q-DRL algorithm also demonstrates a notable increase in the efficiency of system resource utilization,reducing the average idle rate by 5.02%and 9.30%in comparison to IDQN and DRL-Cloud,respectively.These findings support the assertion that Q-DRL not only upholds a lower average idle rate but also effectively curtails the average response time,thereby substantially improving processing efficiency and optimizing resource utilization within distributed graph database systems.

Pyramid Separable Channel Attention Network for Single Image Super-Resolution

Single Image Super-Resolution(SISR)technology aims to reconstruct a clear,high-resolution image with more information from an input low-resolution image that is blurry and contains less information.This technology has significant research value and is widely used in fields such as medical imaging,satellite image processing,and security surveillance.Despite significant progress in existing research,challenges remain in reconstructing clear and complex texture details,with issues such as edge blurring and artifacts still present.The visual perception effect still needs further enhancement.Therefore,this study proposes a Pyramid Separable Channel Attention Network(PSCAN)for the SISR task.Thismethod designs a convolutional backbone network composed of Pyramid Separable Channel Attention blocks to effectively extract and fuse multi-scale features.This expands the model’s receptive field,reduces resolution loss,and enhances the model’s ability to reconstruct texture details.Additionally,an innovative artifact loss function is designed to better distinguish between artifacts and real edge details,reducing artifacts in the reconstructed images.We conducted comprehensive ablation and comparative experiments on the Arabidopsis root image dataset and several public datasets.The experimental results show that the proposed PSCAN method achieves the best-known performance in both subjective visual effects and objective evaluation metrics,with improvements of 0.84 in Peak Signal-to-Noise Ratio(PSNR)and 0.017 in Structural Similarity Index(SSIM).This demonstrates that the method can effectively preserve high-frequency texture details,reduce artifacts,and have good generalization performance.

Task offloading mechanism based on federated reinforcement learning in mobile edge computing

With the arrival of 5G,latency-sensitive applications are becoming increasingly diverse.Mobile Edge Computing(MEC)technology has the characteristics of high bandwidth,low latency and low energy consumption,and has attracted much attention among researchers.To improve the Quality of Service(QoS),this study focuses on computation offloading in MEC.We consider the QoS from the perspective of computational cost,dimensional disaster,user privacy and catastrophic forgetting of new users.The QoS model is established based on the delay and energy consumption and is based on DDQN and a Federated Learning(FL)adaptive task offloading algorithm in MEC.The proposed algorithm combines the QoS model and deep reinforcement learning algorithm to obtain an optimal offloading policy according to the local link and node state information in the channel coherence time to address the problem of time-varying transmission channels and reduce the computing energy consumption and task processing delay.To solve the problems of privacy and catastrophic forgetting,we use FL to make distributed use of multiple users’data to obtain the decision model,protect data privacy and improve the model universality.In the process of FL iteration,the communication delay of individual devices is too large,which affects the overall delay cost.Therefore,we adopt a communication delay optimization algorithm based on the unary outlier detection mechanism to reduce the communication delay of FL.The simulation results indicate that compared with existing schemes,the proposed method significantly reduces the computation cost on a device and improves the QoS when handling complex tasks.

多无人机辅助边缘计算场景下基于Q-learning的任务卸载优化

引入多无人机辅助边缘计算系统,由多个无人机和原有边缘服务器共同为移动用户提供通信和计算资源;将优化问题建模为资源竞争和卸载决策约束下的系统总效用最大化问题,系统总效用由用户满意度、任务延迟和系统能耗3个因素组成.由于优化模型是一个具有NP难属性的非凸问题,故采用强化学习方法求解得到使系统总效用最大的最优任务卸载决策集.仿真实验结果表明,与贪心顺序调优卸载方案和随机选择卸载方案相比,该文提出的Q-learning方案的系统总效用分别提高了15%和43%以上.

PEPFL:A framework for a practical and efficient privacy-preserving federated learning

As an emerging joint learning model,federated learning is a promising way to combine model parameters of different users for training and inference without collecting users’original data.However,a practical and efficient solution has not been established in previous work due to the absence of efficient matrix computation and cryptography schemes in the privacy-preserving federated learning model,especially in partially homomorphic cryptosystems.In this paper,we propose a Practical and Efficient Privacy-preserving Federated Learning(PEPFL)framework.First,we present a lifted distributed ElGamal cryptosystem for federated learning,which can solve the multi-key problem in federated learning.Secondly,we develop a Practical Partially Single Instruction Multiple Data(PSIMD)parallelism scheme that can encode a plaintext matrix into single plaintext for encryption,improving the encryption efficiency and reducing the communication cost in partially homomorphic cryptosystem.In addition,based on the Convolutional Neural Network(CNN)and the designed cryptosystem,a novel privacy-preserving federated learning framework is designed by using Momentum Gradient Descent(MGD).Finally,we evaluate the security and performance of PEPFL.The experiment results demonstrate that the scheme is practicable,effective,and secure with low communication and computation costs.

Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning

The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.

Single-atom catalysts based on polarization switching of ferroelectric In_(2)Se_(3) for N_(2) reduction

The polarization switching plays a crucial role in controlling the final products in the catalytic pro-cess.The effect of polarization orientation on nitrogen reduction was investigated by anchoring transition metal atoms to form active centers on ferroelectric material In_(2)Se_(3).During the polariza-tion switching process,the difference in surface electrostatic potential leads to a redistribution of electronic states.This affects the interaction strength between the adsorbed small molecules and the catalyst substrate,thereby altering the reaction barrier.In addition,the surface states must be considered to prevent the adsorption of other small molecules(such as *O,*OH,and *H).Further-more,the V@↓-In_(2)Se_(3) possesses excellent catalytic properties,high electrochemical and thermody-namic stability,which facilitates the catalytic process.Machine learning also helps us further ex-plore the underlying mechanisms.The systematic investigation provides novel insights into the design and application of two-dimensional switchable ferroelectric catalysts for various chemical processes.

Learning curve of transumbilical suture-suspension single-incision laparoscopic cholecystectomy

AIM: To investigate the learning curve of transumbilical suture-suspension single-incision laparoscopic cholecystectomy (SILC). METHODS: The clinical data of 180 consecutive transumbilical suture-suspension SILCs performed by a team in our department during the period from August 2009 to March 2011 were retrospectively analyzed. Patients were divided into nine groups according to operation dates, and each group included 20 patients operated on consecutively in each time period. The surgical outcome was assessed by comparing operation time, blood loss during operation, and complications between groups in order to evaluate the improvement in technique.RESULTS: A total of 180 SILCs were successfully performed by five doctors. The average operation time was 53.58 ± 30.08 min (range: 20.00-160.00 min) and average blood loss was 12.70 ± 11.60 mL (range: 0.00-100.00 mL). None of the patients were converted to laparotomy or multi-port laparoscopic cholecystectomy. There were no major complications such as hemorrhage or biliary system injury during surgery. Eight postoperative complications occurred mainly in the first three groups (n = 6), and included ecchymosis around the umbilical incision (n = 7) which resolved without special treatment, and one case of delayed bile leakage in group 8, which was treated by ultrasound-guided puncture and drainage. There were no differences in intraoperative blood loss, postoperative complications and length of postoperative hospital stay among the groups. Bonferroni's test showed that the operation time in group 1 was significantly longer than that in the other groups (F = 7.257, P = 0.000). The majority of patients in each group were discharged within 2 d, with an average postoperative hospital stay of 1.9 ± 1.2 d. CONCLUSION: Following scientific principles and standard procedures, a team experienced in multi-port laparoscopic cholecystectomy can master the technique of SILC after 20 cases.

R2N: A Novel Deep Learning Architecture for Rain Removal from Single Image

Visual degradation of captured images caused by rainy streaks under rainy weather can adversely affect the performance of many open-air vision systems.Hence,it is necessary to address the problem of eliminating rain streaks from the individual rainy image.In this work,a deep convolution neural network(CNN)based method is introduced,called Rain-Removal Net(R2N),to solve the single image de-raining issue.Firstly,we decomposed the rainy image into its high-frequency detail layer and lowfrequency base layer.Then,we used the high-frequency detail layer to input the carefully designed CNN architecture to learn the mapping between it and its corresponding derained high-frequency detail layer.The CNN architecture consists of four convolution layers and four deconvolution layers,as well as three skip connections.The experiments on synthetic and real-world rainy images show that the performance of our architecture outperforms the compared state-of-the-art de-raining models with respects to the quality of de-rained images and computing efficiency.

Single and Mitochondrial Gene Inheritance Disorder Prediction Using Machine Learning

One of the most difficult jobs in the post-genomic age is identifying a genetic disease from a massive amount of genetic data.Furthermore,the complicated genetic disease has a very diverse genotype,making it challenging to find genetic markers.This is a challenging process since it must be completed effectively and efficiently.This research article focuses largely on which patients are more likely to have a genetic disorder based on numerous medical parameters.Using the patient’s medical history,we used a genetic disease prediction algorithm that predicts if the patient is likely to be diagnosed with a genetic disorder.To predict and categorize the patient with a genetic disease,we utilize several deep and machine learning techniques such as Artificial neural network(ANN),K-nearest neighbors(KNN),and Support vector machine(SVM).To enhance the accuracy of predicting the genetic disease in any patient,a highly efficient approach was utilized to control how the model can be used.To predict genetic disease,deep and machine learning approaches are performed.The most productive tool model provides more precise efficiency.The simulation results demonstrate that by using the proposed model with the ANN,we achieve the highest model performance of 85.7%,84.9%,84.3%accuracy of training,testing and validation respectively.This approach will undoubtedly transform genetic disorder prediction and give a real competitive strategy to save patients’lives.

Machine Learning to Instruct Single Crystal Growth by Flux Method

Growth of high-quality single crystals is of great significance for research of condensed matter physics. The exploration of suitable growing conditions for single crystals is expensive and time-consuming, especially for ternary compounds because of the lack of ternary phase diagram. Here we use machine learning(ML) trained on our experimental data to predict and instruct the growth. Four kinds of ML methods, including support vector machine(SVM), decision tree, random forest and gradient boosting decision tree, are adopted. The SVM method is relatively stable and works well, with an accuracy of 81% in predicting experimental results. By comparison,the accuracy of laboratory reaches 36%. The decision tree model is also used to reveal which features will take critical roles in growing processes.

Learning a discriminative high-fidelity dictionary for single channel source separation

Sparse-representation-based single-channel source separation,which aims to recover each source’s signal using its corresponding sub-dictionary,has attracted many scholars’attention.The basic premise of this model is that each sub-dictionary possesses discriminative information about its corresponding source,and this information can be used to recover almost every sample from that source.However,in a more general sense,the samples from a source are composed not only of discriminative information but also common information shared with other sources.This paper proposes learning a discriminative high-fidelity dictionary to improve the separation performance.The innovations are threefold.Firstly,an extra sub-dictionary was combined into a conventional union dictionary to ensure that the source-specific sub-dictionaries can capture only the purely discriminative information for their corresponding sources because the common information is collected in the additional sub-dictionary.Secondly,a task-driven learning algorithm is designed to optimize the new union dictionary and a set of weights that indicate how much of the common information should be allocated to each source.Thirdly,a source separation scheme based on the learned dictionary is presented.Experimental results on a human speech dataset yield evidence that our algorithm can achieve better separation performance than either state-of-the-art or traditional algorithms.

CT reconstruction from a single X-ray image for a particular patient via progressive learning

Computed tomography(CT)has enjoyed widespread applications,especially in the assistance of clinical diagnosis and treatment.However,fast CT imaging is not available for guiding adaptive precise radiotherapy in the current radiation treatment process because the conventional CT reconstruction requires numerous projections and rich computing resources.This paper mainly studies the challenging task of 3 D CT reconstruction from a single 2 D X-ray image of a particular patient,which enables fast CT imaging during radiotherapy.It is widely known that the transformation from a 2 D projection to a 3 D volumetric CT image is a highly nonlinear mapping problem.In this paper,we propose a progressive learning framework to facilitate 2 D-to-3 D mapping.The proposed network starts training from low resolution and then adds new layers to learn increasing high-resolution details as the training progresses.In addition,by bridging the distribution gap between an X-ray image and a CT image with a novel attention-based 2 D-to-3 D feature transform module and an adaptive instance normalization layer,our network obtains enhanced performance in recovering a 3 D CT volume from a single X-ray image.We demonstrate the effectiveness of our approach on a ten-phase 4 D CT dataset including 20 different patients created from a public medical database and show its outperformance over some baseline methods in image quality and structure preservation,achieving a PSNR value of 22.76±0.708 dB and FSIM value of 0.871±0.012 with the ground truth as a reference.This method may promote the application of CT imaging in adaptive radiotherapy and provide image guidance for interventional surgery.

Evaluation of the Single-Port Laparoscopic Right Hemicolectomy Learning Curve

Background: The use of single-port laparoscopy has gained popularity within recent years. Part of the appeal in learning this approach is that it draws heavily from concepts mastered through conventional laparoscopy. Various studies have shown the efficacy and feasibility of the single-port laparoscopic approach, but there are few that examine the learning curve in adopting this new technique. Objective: Our goal was to better define the learning curve in performing a single-port laparoscopic right hemicolectomy. Design: A review of prospectively gathered operative data was performed to analyze the results of single-port laparoscopic right hemicolectomies performed within our institution by experienced laparoscopic surgeons. The first 100 cases were divided into quintiles. Comparisons were made among the cohorts regarding patient demographics, operative time, length of stay, conversions, and complications. Results: There was no difference among quintiles with regard to age, sex, BMI, or ASA class. Operative time, conversions, length of stay, and number of complications did not significantly vary among each group of patients. There was a significant difference in estimated blood loss and length of stay between the fifth cohort and the others due to one patient’s poor outcome. Conclusions: The single-port laparoscopic right hemicolectomy learning curve for surgeons already skilled in laparoscopy is short. There are few differences in various outcome measures among groups at any stage in the learning curve. The skills utilized to perform conventional laparoscopic colorectal surgery readily translate to the single-port approach and result in proficiency from nearly the start.