Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

Multi-access performance of DS UWB systemunder indoor dense multi-path channel

The performance of multi-user code to direct spreading bi-phase shift keying （DS-BPSK） direct impulse ultra wideband （UWB） systems under indoor multi-user and multi-path environment is analyzed and simulated. The system output signals with Rake receiver are derived, then a simple and practical code selection scheme is given; i. e., with a large occupation to empty ratio of the repeating pulses, directly choosing those random or pseudo-random user codes with enough length and good co-relative orthogonal features will make the performance of DS-BPSK approximate the optimum and, so there is no need to carefully design the code or its type. The system multi-access performances are simulated using Gold sequence and PN codes as multi-user codes under CMI-CM4 multi-path channels. Simulation results prove that the proposed scheme is feasible.

Deep Reinforcement Learning-Based Computation Offloading for 5G Vehicle-Aware Multi-Access Edge Computing Network

Multi-access Edge Computing(MEC)is one of the key technologies of the future 5G network.By deploying edge computing centers at the edge of wireless access network,the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios.Meanwhile,with the development of IOV(Internet of Vehicles)technology,various delay-sensitive and compute-intensive in-vehicle applications continue to appear.Compared with traditional Internet business,these computation tasks have higher processing priority and lower delay requirements.In this paper,we design a 5G-based vehicle-aware Multi-access Edge Computing network(VAMECN)and propose a joint optimization problem of minimizing total system cost.In view of the problem,a deep reinforcement learningbased joint computation offloading and task migration optimization(JCOTM)algorithm is proposed,considering the influences of multiple factors such as concurrent multiple computation tasks,system computing resources distribution,and network communication bandwidth.And,the mixed integer nonlinear programming problem is described as a Markov Decision Process.Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption,optimize computing offloading and resource allocation schemes,and improve system resource utilization,compared with other computing offloading policies.

Intelligent Immunity Based Security Defense System for Multi-Access Edge Computing Network

In this paper,the security problem for the multi-access edge computing(MEC)network is researched,and an intelligent immunity-based security defense system is proposed to identify the unauthorized mobile users and to protect the security of whole system.In the proposed security defense system,the security is protected by the intelligent immunity through three functions,identification function,learning function,and regulation function,respectively.Meanwhile,a three process-based intelligent algorithm is proposed for the intelligent immunity system.Numerical simulations are given to prove the effeteness of the proposed approach.

Integration of Communication and Computing in Blockchain-Enabled Multi-Access Edge Computing Systems

Blockchain and multi-access edge com-puting(MEC)are two emerging promising tech-nologies that have received extensive attention from academia and industry.As a brand-new information storage,dissemination and management mechanism,blockchain technology achieves the reliable transmis-sion of data and value.While as a new computing paradigm,multi-access edge computing enables the high-frequency interaction and real-time transmission of data.The integration of communication and com-puting in blockchain-enabled multi-access edge com-puting networks has been studied without a systemat-ical view.In the survey,we focus on the integration of communication and computing,explores the mu-tual empowerment and mutual promotion effects be-tween the blockchain and MEC,and introduces the resource integration architecture of blockchain and multi-access edge computing.Then,the paper sum-marizes the applications of the resource integration ar-chitecture,resource management,data sharing,incen-tive mechanism,and consensus mechanism,and ana-lyzes corresponding applications in real-world scenar-ios.Finally,future challenges and potentially promis-ing research directions are discussed and present in de-tail.

A Review in the Core Technologies of 5G: Device-to-Device Communication, Multi-Access Edge Computing and Network Function Virtualization

5G is a new generation of mobile networking that aims to achieve unparalleled speed and performance. To accomplish this, three technologies, Device-to-Device communication (D2D), multi-access edge computing (MEC) and network function virtualization (NFV) with ClickOS, have been a significant part of 5G, and this paper mainly discusses them. D2D enables direct communication between devices without the relay of base station. In 5G, a two-tier cellular network composed of traditional cellular network system and D2D is an efficient method for realizing high-speed communication. MEC unloads work from end devices and clouds platforms to widespread nodes, and connects the nodes together with outside devices and third-party providers, in order to diminish the overloading effect on any device caused by enormous applications and improve users’ quality of experience (QoE). There is also a NFV method in order to fulfill the 5G requirements. In this part, an optimized virtual machine for middle-boxes named ClickOS is introduced, and it is evaluated in several aspects. Some middle boxes are being implemented in the ClickOS and proved to have outstanding performances.

New strategies for collision resolution of multi-access channel

The truncated binary exponential back-off algorithm is one of the most effective methods applied in collision resolution process of random multi-access channel.In this study,two new strategies are presented to improve the capability of the truncated binary exponential back-off algorithm.In the new strategies,the sizes of the initial window size or the operating window sizes are adjusted dynamically,which always bring a significant improvement for the self-adaptability of the original algorithm.A series of experiments are simulated and the results verify that the new strategies can make the implementation more stable and effective than the original algorithm.

DQN-Based Proactive Trajectory Planning of UAVs in Multi-Access Edge Computing

The main aim of future mobile networks is to provide secure,reliable,intelligent,and seamless connectivity.It also enables mobile network operators to ensure their customer’s a better quality of service(QoS).Nowadays,Unmanned Aerial Vehicles(UAVs)are a significant part of the mobile network due to their continuously growing use in various applications.For better coverage,cost-effective,and seamless service connectivity and provisioning,UAVs have emerged as the best choice for telco operators.UAVs can be used as flying base stations,edge servers,and relay nodes in mobile networks.On the other side,Multi-access EdgeComputing(MEC)technology also emerged in the 5G network to provide a better quality of experience(QoE)to users with different QoS requirements.However,UAVs in a mobile network for coverage enhancement and better QoS face several challenges such as trajectory designing,path planning,optimization,QoS assurance,mobilitymanagement,etc.The efficient and proactive path planning and optimization in a highly dynamic environment containing buildings and obstacles are challenging.So,an automated Artificial Intelligence(AI)enabled QoSaware solution is needed for trajectory planning and optimization.Therefore,this work introduces a well-designed AI and MEC-enabled architecture for a UAVs-assisted future network.It has an efficient Deep Reinforcement Learning(DRL)algorithm for real-time and proactive trajectory planning and optimization.It also fulfills QoS-aware service provisioning.A greedypolicy approach is used to maximize the long-term reward for serving more users withQoS.Simulation results reveal the superiority of the proposed DRL mechanism for energy-efficient and QoS-aware trajectory planning over the existing models.

基于高空平台的边缘计算卸载:网络、算法和展望

高空平台(high altitude platform,HAP)技术与多接入边缘计算(multi-access edge computing,MEC)技术的结合将MEC服务器部署区域由地面扩展到空中,打破传统地面MEC网络的局限性,为用户提供无处不在的计算卸载服务。针对基于HAP的MEC卸载研究进行综述,首先,从HAP计算节点的优势、网络组成部分、网络结构、主要挑战及其应对技术4个方面分析基于HAP的MEC网络;其次,分别从图论、博弈论、机器学习、联邦学习等理论的角度对基于HAP的MEC卸载算法进行横向分析和纵向对比;最后,指出基于HAP的MEC卸载技术目前存在的问题,并对该技术的未来研究方向进行展望。

Two-Stage IoT Computational Task Offloading Decision-Making in MEC with Request Holding and Dynamic Eviction

The rapid development of Internet of Things(IoT)technology has led to a significant increase in the computational task load of Terminal Devices(TDs).TDs reduce response latency and energy consumption with the support of task-offloading in Multi-access Edge Computing(MEC).However,existing task-offloading optimization methods typically assume that MEC’s computing resources are unlimited,and there is a lack of research on the optimization of task-offloading when MEC resources are exhausted.In addition,existing solutions only decide whether to accept the offloaded task request based on the single decision result of the current time slot,but lack support for multiple retry in subsequent time slots.It is resulting in TD missing potential offloading opportunities in the future.To fill this gap,we propose a Two-Stage Offloading Decision-making Framework(TSODF)with request holding and dynamic eviction.Long Short-Term Memory(LSTM)-based task-offloading request prediction and MEC resource release estimation are integrated to infer the probability of a request being accepted in the subsequent time slot.The framework learns optimized decision-making experiences continuously to increase the success rate of task offloading based on deep learning technology.Simulation results show that TSODF reduces total TD’s energy consumption and delay for task execution and improves task offloading rate and system resource utilization compared to the benchmark method.

Associative Tasks Computing Offloading Scheme in Internet of Medical Things with Deep Reinforcement Learning

The Internet of Medical Things(Io MT) is regarded as a critical technology for intelligent healthcare in the foreseeable 6G era. Nevertheless, due to the limited computing power capability of edge devices and task-related coupling relationships, Io MT faces unprecedented challenges. Considering the associative connections among tasks, this paper proposes a computing offloading policy for multiple-user devices(UDs) considering device-to-device(D2D) communication and a multi-access edge computing(MEC)technique under the scenario of Io MT. Specifically,to minimize the total delay and energy consumption concerning the requirement of Io MT, we first analyze and model the detailed local execution, MEC execution, D2D execution, and associated tasks offloading exchange model. Consequently, the associated tasks’ offloading scheme of multi-UDs is formulated as a mixed-integer nonconvex optimization problem. Considering the advantages of deep reinforcement learning(DRL) in processing tasks related to coupling relationships, a Double DQN based associative tasks computing offloading(DDATO) algorithm is then proposed to obtain the optimal solution, which can make the best offloading decision under the condition that tasks of UDs are associative. Furthermore, to reduce the complexity of the DDATO algorithm, the cacheaided procedure is intentionally introduced before the data training process. This avoids redundant offloading and computing procedures concerning tasks that previously have already been cached by other UDs. In addition, we use a dynamic ε-greedy strategy in the action selection section of the algorithm, thus preventing the algorithm from falling into a locally optimal solution. Simulation results demonstrate that compared with other existing methods for associative task models concerning different structures in the Io MT network, the proposed algorithm can lower the total cost more effectively and efficiently while also providing a tradeoff between delay and energy consumption tolerance.

Mathematical demonstration of correlation of optical prime code

Starting from the configuration of the optical prime code, a kind of key signature code for the optical code-division multiple access （OCDMA） system, based on the linear congruence theory in the finite Galois field, the correlation properties of the basic prime code, the extended prime code and the modified prime code are mathematically analyzed, the distribution of cross-correlation values is given and the overlap area of ＂1＂s in the case of periodically circularly shifting is indicated. It is mathematically demonstrated that the maximum cross-correlation of the basic prime code is 2, and that of the extended prime code and the modified prime code is 1. The integrated correlation analysis process is proposed. The signal-interfere ratio （SIR） and the BER performance of the systems employing different signature codes are calculated, respectively, and the performances of OCDMA systems employing different signature codes mode are compared.

Jointly Optimized Request Dispatching and Service Placement for MEC in LEO Network

Integrating Multi-access Edge Computing(MEC) in Low Earth Orbit(LEO) network is an important way to provide globally seamless low-delay service. In this paper, we consider the scenario that MEC platforms with computation and storage resource are deployed on LEO satellites, which is called "LEO-MEC". Service request dispatching decision is very important for resource utilization of the whole LEO-MEC system and Qo E of MEC users. Another important problem is service placement that is closely coupled with request dispatching. This paper models the joint service request dispatching and service placement problem as an optimization problem, which is a Mixed Integer Linear Programming(MILP). Our proposed mechanism solves this problem and uses the solved decision variables to dispatch requests and place services. Simulation results show that our proposed mechanism can achieve better performance in terms of ratio of served users and average hop count compared with baseline mechanism.

New Parallel Interference Cancellation Scheme for DS/CDMA Systems

This paper proposes a new multi-stage parallel interference cancellation scheme by modifying the conventional multi-stage parallel interference canceller (PIC). At each stage, it first converts the interference-cancelled outputs from previous stage into thea prior information, in terms of which the bit mean values are computed and the multi-access interference (MAI) for each user is evaluated, and then an interference cancellation is performed to obtain further interference suppression. To reduce the implementation complexity, we give an approximation expression for bit mean value. The performance over AWGN channel is analyzed and compared to the conventional PIC. The user numberK=7 and spreading factorN=13 are chosen as simulation parameters. The computer simulation results show that the proposed PIC has better performance than the conventional PIC both with 2 interference cancellation (IC) stages, at bit error rate of 10?3, for example, about 3 dB performance gain is obtained by using the proposed PIC. It is also shown that our proposed PIC with 1-stage is superior to the conventional PIC with 2-stage in performance, which is of practical value because PIC with fewer stages can bring about shorter processing delay. Key words CDMA - parallel interference cancellation - multi-access interference CLC number TN 914 Foundation item: Supported by the National Natural Science Foundation of China (69772015)Biography: Xu Guo-xiong (1967-), male, Ph. D candidate, research direction: wireless communication.

Joint resource allocation for hybrid NOMA-assisted MEC in 6G networks

Multi-access Edge Computing(MEC)is an essential technology for expanding computing power of mobile devices,which can combine the Non-Orthogonal Multiple Access(NOMA)in the power domain to multiplex signals to improve spectral efficiency.We study the integration of the MEC with the NOMA to improve the computation service for the Beyond Fifth-Generation(B5G)and the Sixth-Generation(6G)wireless networks.This paper aims to minimize the energy consumption of a hybrid NOMA-assisted MEC system.In a hybrid NOMA system,a user can offload its task during a time slot shared with another user by the NOMA,and then upload the remaining data during an exclusive time duration served by Orthogonal Multiple Access(OMA).The original energy minimization problem is non-convex.To efficiently solve it,we first assume that the user grouping is given,and focuses on the one group case.Then,a multilevel programming method is proposed to solve the non-convex problem by decomposing it into three subproblems,i.e.,power allocation,time slot scheduling,and offloading task assignment,which are solved optimally by carefully studying their convexity and monotonicity.The derived solution is optimal to the original problem by substituting the closed expressions obtained from those decomposed subproblems.Furthermore,we investigate the multi-user case,in which a close-to-optimal algorithm with lowcomplexity is proposed to form users into different groups with unique time slots.The simulation results verify the superior performance of the proposed scheme compared with some benchmarks,such as OMA and pure NOMA.

ANALYSIS OF SPECTRAL PHASE ENCODING OCDMA SYSTEM WITH PSEUDORANDOM CODING

Multiple access interference (MAI) is the most serious interference in spectral phase encoding optical code division multiple access (SPE OCDMA) systems. This paper focuses on the behavior of MAI in SPE OCDMA systems with pseudorandom coding. The statistical expectation of multi access interference (MAI) is derived and plotted. The results confirm that MAI can be suppressed effectively by pseudorandom coding with m sequences.

JOINT SPACE-FREQUENCY MULTIUSER SYMBOL DETECTOR FOR MC-CDMA SYSTEM WITH UNIFORM LINEAR ARRAY

The MultiCarrier Code Division Multiple Access (MC-CDMA) scheme is promising for relieving capacity limit problems of Direct Sequence (DS-) CDMA systems due to serious InterChip Interference (ICI) and MultiUser Interference (MUI) in high-data-rate wireless communication systems. In this paper, the Uniform Linear Array (ULA) is applied to the base station of macrocellular MC-CDMA systems in a frequency-selective fading channel environment. A joint space-frequency multiuser symbol sequence detector is developed for all active users within one macrocell without space-frequency channel estimation. Simultaneously, Directions-Of-Arrivals (DOAs) of all active users can also be estimated. By dividing the ULA into two identical overlapping subarrays, a specific auxiliary matrix is constructed, which includes both symbol sequence and DOA information of all active users. Then, based on the subspace method, performing the eigen decomposition on such auxiliary matrix, the closed-form solution of symbol sequences and DOAs for all active users can be obtained. In comparison with schemes based on channel estimation, our algorithm need not explicitly estimate the space-frequency channel for each active user,so it has lower computation complexity. Extensive computer simulations demonstrate the overall performance of this novel scheme.

Multi-user performance analysis of differential frequency hopping system over Rayleigh-fading channel

A novel non-coherent detection scheme for differential frequency hopping(DFH)system is proposedin asynchronous multi-user environments over Rayleigh-fading channels.The synchronous and asyn-chronous multi-user performances of DFH with the conventional detection scheme and this novel detectionscheme are analyzed,respectively.The performance results are validated with simulation.The results ofanalyses and simulations prove two conclusions.Firstly,the performance of asynchronous multi-user DFHsystem overcomes that of synchronous multi-user DFH system over Rayleigh-fading channel.Secondly,the novel detection scheme can achieve better performance than the conventional non-coherent detectionscheme in asynchronous multi-user environments.

NOVEL ADAPTIVE MULTIUSER DETECTIONALGORITHM BASED ON WAVELET TRANSFORM

The wavelet transform-based adaptive multiuser detection algorithm is presented. The novel adaptive multiuser detection algorithm uses the wavelet transform for the preprocessing, and wavelet-transformed signal uses LMS algorithm to implement the adaptive multiuser detection. The algorithm makes use of wavelet transform to divide the wavelet space, which shows that the wavelet transform has a better decorrelation ability and leads to better convergence. White noise can be wiped off under the wavelet transform according to different characteristics of signal and white noise under the wavelet transform. Theoretical analyses and simulations demonstrate that the algorithm converges faster than the conventional adaptive multiuser detection algorithm, and has the better performance. Simulation results reveal that the algorithm convergence relates to the wavelet base, and show that the algorithm convergence gets better with the increasing of regularity for the same series of the wavelet base. Finally the algorithm shows that it can be easily implemented.

Beyond 5G Networks: Integration of Communication, Computing, Caching, and Control

In recent years,the exponential proliferation of smart devices with their intelligent applications poses severe challenges on conventional cellular networks.Such challenges can be potentially overcome by integrating communication,computing,caching,and control(i4C)technologies.In this survey,we first give a snapshot of different aspects of the i4C,comprising background,motivation,leading technological enablers,potential applications,and use cases.Next,we describe different models of communication,computing,caching,and control(4C)to lay the foundation of the integration approach.We review current stateof-the-art research efforts related to the i4C,focusing on recent trends of both conventional and artificial intelligence(AI)-based integration approaches.We also highlight the need for intelligence in resources integration.Then,we discuss the integration of sensing and communication(ISAC)and classify the integration approaches into various classes.Finally,we propose open challenges and present future research directions for beyond 5G networks,such as 6G.