Deep Reinforcement Learning-Based Task Offloading and Service Migrating Policies in Service Caching-Assisted Mobile Edge Computing

Emerging mobile edge computing(MEC)is considered a feasible solution for offloading the computation-intensive request tasks generated from mobile wireless equipment(MWE)with limited computational resources and energy.Due to the homogeneity of request tasks from one MWE during a longterm time period,it is vital to predeploy the particular service cachings required by the request tasks at the MEC server.In this paper,we model a service caching-assisted MEC framework that takes into account the constraint on the number of service cachings hosted by each edge server and the migration of request tasks from the current edge server to another edge server with service caching required by tasks.Furthermore,we propose a multiagent deep reinforcement learning-based computation offloading and task migrating decision-making scheme(MBOMS)to minimize the long-term average weighted cost.The proposed MBOMS can learn the near-optimal offloading and migrating decision-making policy by centralized training and decentralized execution.Systematic and comprehensive simulation results reveal that our proposed MBOMS can converge well after training and outperforms the other five baseline algorithms.

Deep learning framework for multi‐round service bundle recommendation in iterative mashup development

Recent years have witnessed the rapid development of service‐oriented computing technologies.The boom of Web services increases software developers'selection burden in developing new service‐based systems such as mashups.Timely recommending appropriate component services for developers to build new mashups has become a fundamental problem in service‐oriented software engineering.Existing service recom-mendation approaches are mainly designed for mashup development in the single‐round scenario.It is hard for them to effectively update recommendation results according to developers'requirements and behaviours(e.g.instant service selection).To address this issue,the authors propose a service bundle recommendation framework based on deep learning,DLISR,which aims to capture the interactions among the target mashup to build,selected(component)services,and the following service to recommend.Moreover,an attention mechanism is employed in DLISR to weigh selected services when rec-ommending a candidate service.The authors also design two separate models for learning interactions from the perspectives of content and invocation history,respectively,and a hybrid model called HISR.Experiments on a real‐world dataset indicate that HISR can outperform several state‐of‐the‐art service recommendation methods to develop new mashups iteratively.

Internet of Things Enabled DDoS Attack Detection Using Pigeon Inspired Optimization Algorithm with Deep Learning Approach

Internet of Things(IoTs)provides better solutions in various fields,namely healthcare,smart transportation,home,etc.Recognizing Denial of Service(DoS)outbreaks in IoT platforms is significant in certifying the accessibility and integrity of IoT systems.Deep learning(DL)models outperform in detecting complex,non-linear relationships,allowing them to effectually severe slight deviations fromnormal IoT activities that may designate a DoS outbreak.The uninterrupted observation and real-time detection actions of DL participate in accurate and rapid detection,permitting proactive reduction events to be executed,hence securing the IoT network’s safety and functionality.Subsequently,this study presents pigeon-inspired optimization with a DL-based attack detection and classification(PIODL-ADC)approach in an IoT environment.The PIODL-ADC approach implements a hyperparameter-tuned DL method for Distributed Denial-of-Service(DDoS)attack detection in an IoT platform.Initially,the PIODL-ADC model utilizes Z-score normalization to scale input data into a uniformformat.For handling the convolutional and adaptive behaviors of IoT,the PIODL-ADCmodel employs the pigeon-inspired optimization(PIO)method for feature selection to detect the related features,considerably enhancing the recognition’s accuracy.Also,the Elman Recurrent Neural Network(ERNN)model is utilized to recognize and classify DDoS attacks.Moreover,reptile search algorithm(RSA)based hyperparameter tuning is employed to improve the precision and robustness of the ERNN method.A series of investigational validations is made to ensure the accomplishment of the PIODL-ADC method.The experimental outcome exhibited that the PIODL-ADC method shows greater accomplishment when related to existing models,with a maximum accuracy of 99.81%.

Learning-Based Joint Service Caching and Load Balancing for MEC Blockchain Networks

Integrating the blockchain technology into mobile-edge computing(MEC)networks with multiple cooperative MEC servers(MECS)providing a promising solution to improving resource utilization,and helping establish a secure reward mechanism that can facilitate load balancing among MECS.In addition,intelligent management of service caching and load balancing can improve the network utility in MEC blockchain networks with multiple types of workloads.In this paper,we investigate a learningbased joint service caching and load balancing policy for optimizing the communication and computation resources allocation,so as to improve the resource utilization of MEC blockchain networks.We formulate the problem as a challenging long-term network revenue maximization Markov decision process(MDP)problem.To address the highly dynamic and high dimension of system states,we design a joint service caching and load balancing algorithm based on the double-dueling Deep Q network(DQN)approach.The simulation results validate the feasibility and superior performance of our proposed algorithm over several baseline schemes.

A rapid, low-cost deep learning system to classify strawberry disease based on cloud service

Accurate and timely classification of diseases during strawberry planting can help growers deal with them in timely manner, thereby reducing losses. However, the classification of strawberry diseases in real planting environments is facing severe challenges, including complex planting environments, multiple disease categories with small differences, and so on. Although recent mobile vision technology based deep learning has achieved some success in overcoming the above problems, a key problem is how to construct a non-destructive, fast and convenient method to improve the efficiency of strawberry disease identification for the multi-region, multi-space and multi-time classification requirements. We develop and evaluate a rapid, low-cost system for classifying diseases in strawberry cultivation. This involves designing an easy-to-use cloudbased strawberry disease identification system, combined with our novel self-supervised multi-network fusion classification model, which consists of a Location network, a Feedback network and a Classification network to identify the categories of common strawberry diseases. With the help of a novel self-supervision mechanism, the model can effectively identify diseased regions of strawberry disease images without the need for annotations such as bounding boxes. Using accuracy, precision, recall and F1 to evaluate the classification effect, the results of the test set are 92.48, 90.68, 86.32 and 88.45%, respectively. Compared with popular Convolutional Neural Networks(CNN) and five other methods, our network achieves better disease classification effect. Currently, the client(mini program) has been released on the We Chat platform. The mini program has perfect classification effect in the actual test, which verifies the feasibility and effectiveness of the system, and can provide a reference for the intelligent research and application of strawberry disease identification.

Length matters:Scalable fast encrypted internet traffic service classification based on multiple protocol data unit length sequence with composite deep learning

As an essential function of encrypted Internet traffic analysis,encrypted traffic service classification can support both coarse-grained network service traffic management and security supervision.However,the traditional plaintext-based Deep Packet Inspection(DPI)method cannot be applied to such a classification.Moreover,machine learning-based existing methods encounter two problems during feature selection:complex feature overcost processing and Transport Layer Security(TLS)version discrepancy.In this paper,we consider differences between encryption network protocol stacks and propose a composite deep learning-based method in multiprotocol environments using a sliding multiple Protocol Data Unit(multiPDU)length sequence as features by fully utilizing the Markov property in a multiPDU length sequence and maintaining suitability with a TLS-1.3 environment.Control experiments show that both Length-Sensitive(LS)composite deep learning model using a capsule neural network and LS-long short time memory achieve satisfactory effectiveness in F1-score and performance.Owing to faster feature extraction,our method is suitable for actual network environments and superior to state-of-the-art methods.

MC-infer:DLaaS中的零知识和无真实数据模型推理攻击

深度学习即服务(DLaaS)的服务模式容易受到模型推理攻击影响。现有的推理攻击要求攻击者拥有足够的辅助信息来进行推理,这并不能完全展示出推理攻击的潜在威胁,因此,提出了MC-infer,一种零知识、无真实数据的黑盒模型推理攻击。MC-infer将从不同随机分布获得的随机噪声输入给目标模型,并根据其输出估计相应的目标分布进行模型推理。使用了蒙特卡洛对MC-infer进行了理论分析,证明了其在理论层面的可行性。实验表明MCinfer可以有效地推断目标模型。此外,研究了MC-infer的局限性和复杂性,最后讨论了几种防止攻击的策略。

Dynamic Security SFC Branching Path Selection Using Deep Reinforcement Learning

Security service function chaining(SFC)based on software-defined networking(SDN)and network function virtualization(NFV)technology allows traffic to be forwarded sequentially among different security service functions to achieve a combination of security functions.Security SFC can be deployed according to requirements,but the current SFC is not flexible enough and lacks an effective feedback mechanism.The SFC is not traffic aware and the changes of traffic may cause the previously deployed security SFC to be invalid.How to establish a closed-loop mechanism to enhance the adaptive capability of the security SFC to malicious traffic has become an important issue.Our contribution is threefold.First,we propose a secure SFC path selection framework.The framework can accept the feedback results of traffic and security service functions in SFC,and dynamically select the opti-mal path for SFC based on the feedback results.It also realizes the automatic deployment of paths,forming a complete closed loop.Second,we expand the protocol of SFC to realize the security SFC with branching path,which improve flexibility of security SFC.Third,we propose a deep reinforcement learning-based dynamic path selection method for security SFC.It infers the optimal branching path by analyzing feedback from the security SFC.We have experimented with Distributed Denial of Service(DDoS)attack detection modules as security service functions.Experimental results show that our proposed method can dynamically select the optimal branching path for a security SFC based on traffic features and the state of the SFC.And it improves the accuracy of the overall malicious traffic detection of the security SFC and significantly reduces the latency and overall load of the SFC.

Deep learning Optical Character Recognition in PCB Dark Silk Recognition

For Automatic Optical Inspection (AOI) machines that were introduced to Printed Circuit Board market more than five years ago, illumination technique and light devices are outdated. Images captured by old AOI machines are not easy to be recognized by typical optical character recognition (OCR) algorithms, especially for dark silk. How to effectively increase silk recognition accuracy is indispensable for improving overall production efficiency in SMT plant. This paper uses fine tuned Character Region Awareness for Text Detection (CRAFT) method to build model for dark silk recognition. CRAFT model consists of a structure similar to U-net, followed by VGG based convolutional neural network. Continuous two-dimensional Gaussian distribution was used for the annotation of image segmentation. CRAFT model is good at recognizing different types of printed characters with high accuracy and transferability. Results show that with the help of CRAFT model, accuracy for OK board is 95% (error rate is 5%), and accuracy for NG board is 100% (omission rate is 0%).

Privacy-preserving deep learning techniques for wearable sensor-based big data applications

Wearable technologies have the potential to become a valuable influence on human daily life where they may enable observing the world in new ways,including,for example,using augmented reality(AR)applications.Wearable technology uses electronic devices that may be carried as accessories,clothes,or even embedded in the user's body.Although the potential benefits of smart wearables are numerous,their extensive and continual usage creates several privacy concerns and tricky information security challenges.In this paper,we present a comprehensive survey of recent privacy-preserving big data analytics applications based on wearable sensors.We highlight the fundamental features of security and privacy for wearable device applications.Then,we examine the utilization of deep learning algorithms with cryptography and determine their usability for wearable sensors.We also present a case study on privacy-preserving machine learning techniques.Herein,we theoretically and empirically evaluate the privacy-preserving deep learning framework's performance.We explain the implementation details of a case study of a secure prediction service using the convolutional neural network(CNN)model and the Cheon-Kim-Kim-Song(CHKS)homomorphic encryption algorithm.Finally,we explore the obstacles and gaps in the deployment of practical real-world applications.Following a comprehensive overview,we identify the most important obstacles that must be overcome and discuss some interesting future research directions.

An effective communication and computation model based on a hybridgraph-deeplearning approach for SIoT

Social Edge Service(SES)is an emerging mechanism in the Social Internet of Things(SIoT)orchestration for effective user-centric reliable communication and computation.The services are affected by active and/or passive attacks such as replay attacks,message tampering because of sharing the same spectrum,as well as inadequate trust measurement methods among intelligent devices(roadside units,mobile edge devices,servers)during computing and content-sharing.These issues lead to computation and communication overhead of servers and computation nodes.To address this issue,we propose the HybridgrAph-Deep-learning(HAD)approach in two stages for secure communication and computation.First,the Adaptive Trust Weight(ATW)model with relation-based feedback fusion analysis to estimate the fitness-priority of every node based on directed graph theory to detect malicious nodes and reduce computation and communication overhead.Second,a Quotient User-centric Coeval-Learning(QUCL)mechanism to formulate secure channel selection,and Nash equilibrium method for optimizing the communication to share data over edge devices.The simulation results confirm that our proposed approach has achieved effective communication and computation performance,and enhanced Social Edge Services(SES)reliability than state-of-the-art approaches.

面向DAG任务的分布式智能计算卸载和服务缓存联合优化

建立了一种有向无环图(DAG,directed acyclic graph)任务卸载和资源优化问题,旨在应用最大可容忍时延等约束实现系统能耗最小化。考虑到网络中计算请求高度动态、完整的系统状态信息难以获取等因素,最后使用多智能体深度确定性策略梯度(MADDPG,multi-agent deep deterministic policy gradient)算法来探寻最优的策略。相比于现有的任务卸载算法,MADDPG算法能够降低14.2%至40.8%的系统平均能耗,并且本地缓存命中率提高3.7%至4.1%。

Toward Secure Software-Defined Networks Using Machine Learning: A Review, Research Challenges, and Future Directions

Over the past few years,rapid advancements in the internet and communication technologies have led to increasingly intricate and diverse networking systems.As a result,greater intelligence is necessary to effectively manage,optimize,and maintain these systems.Due to their distributed nature,machine learning models are challenging to deploy in traditional networks.However,Software-Defined Networking(SDN)presents an opportunity to integrate intelligence into networks by offering a programmable architecture that separates data and control planes.SDN provides a centralized network view and allows for dynamic updates of flow rules and softwarebased traffic analysis.While the programmable nature of SDN makes it easier to deploy machine learning techniques,the centralized control logic also makes it vulnerable to cyberattacks.To address these issues,recent research has focused on developing powerful machine-learning methods for detecting and mitigating attacks in SDN environments.This paper highlighted the countermeasures for cyberattacks on SDN and how current machine learningbased solutions can overcome these emerging issues.We also discuss the pros and cons of using machine learning algorithms for detecting and mitigating these attacks.Finally,we highlighted research issues,gaps,and challenges in developing machine learning-based solutions to secure the SDN controller,to help the research and network community to develop more robust and reliable solutions.

灾害场景下基于MADRL的信息收集无人机部署与节点能效优化

灾害场景下,对灾区内第一手重要信息的及时、可靠收集是灾害预警研究、灾区救援工作开展的关键。无人机是与灾区内部建立应急通信网络的高效辅助工具。通过对现有研究中应急场景下无人机的部署方法进行调查,指出了无人机部署时对节点能效考虑不充分的问题。由于地面传感器节点位于灾区内部,环境恶劣且极为被动,所以结合灾害场景,首次以提高地面节点能效为优化目标,基于深度强化学习方法,在DDQN网络模型基础上,通过自定义经验回放优先级、合理设计奖励函数和采用完全去中心化训练方式,解决该特定场景下用于信息收集无人机的自适应部署问题。仿真结果表明,所提算法的节点能源效率比DDQN基准算法提高21%,训练速度相比DDPG、A3C算法分别提升42%和34%。

Multi-Agent Deep Q-Networks for Efficient Edge Federated Learning Communications in Software-Defined IoT

Federated learning(FL)activates distributed on-device computation techniques to model a better algorithm performance with the interaction of local model updates and global model distributions in aggregation averaging processes.However,in large-scale heterogeneous Internet of Things(IoT)cellular networks,massive multi-dimensional model update iterations and resource-constrained computation are challenging aspects to be tackled significantly.This paper introduces the system model of converging softwaredefined networking(SDN)and network functions virtualization(NFV)to enable device/resource abstractions and provide NFV-enabled edge FL(eFL)aggregation servers for advancing automation and controllability.Multi-agent deep Q-networks(MADQNs)target to enforce a self-learning softwarization,optimize resource allocation policies,and advocate computation offloading decisions.With gathered network conditions and resource states,the proposed agent aims to explore various actions for estimating expected longterm rewards in a particular state observation.In exploration phase,optimal actions for joint resource allocation and offloading decisions in different possible states are obtained by maximum Q-value selections.Action-based virtual network functions(VNF)forwarding graph(VNFFG)is orchestrated to map VNFs towards eFL aggregation server with sufficient communication and computation resources in NFV infrastructure(NFVI).The proposed scheme indicates deficient allocation actions,modifies the VNF backup instances,and reallocates the virtual resource for exploitation phase.Deep neural network(DNN)is used as a value function approximator,and epsilongreedy algorithm balances exploration and exploitation.The scheme primarily considers the criticalities of FL model services and congestion states to optimize long-term policy.Simulation results presented the outperformance of the proposed scheme over reference schemes in terms of Quality of Service(QoS)performance metrics,including packet drop ratio,packet drop counts,packet delivery ratio,delay,and throughput.

HDLIDP: A Hybrid Deep Learning Intrusion Detection and Prevention Framework

Distributed denial-of-service(DDoS)attacks are designed to interrupt network services such as email servers and webpages in traditional computer networks.Furthermore,the enormous number of connected devices makes it difficult to operate such a network effectively.Software defined networks(SDN)are networks that are managed through a centralized control system,according to researchers.This controller is the brain of any SDN,composing the forwarding table of all data plane network switches.Despite the advantages of SDN controllers,DDoS attacks are easier to perpetrate than on traditional networks.Because the controller is a single point of failure,if it fails,the entire network will fail.This paper offers a Hybrid Deep Learning Intrusion Detection and Prevention(HDLIDP)framework,which blends signature-based and deep learning neural networks to detect and prevent intrusions.This framework improves detection accuracy while addressing all of the aforementioned problems.To validate the framework,experiments are done on both traditional and SDN datasets;the findings demonstrate a significant improvement in classification accuracy.

基于Faster R-CNN的服务机器人物品识别研究

传统的日用商品识别流程通常使用较为经典的图像识别和机器学习算法,如支持向量机(SVM)、随机森林或AdaBoost,然后利用目标图像的梯度、纹理或颜色的基本特征来对日用商品进行识别,可以在比较简单的背景中得到应用,但是在复杂的背景环境中很难有比较突出的表现,并且难以达到较高的准确率。目前在目标识别中表现比较优异的是卷积神经网络(CNN),并成为很多目标识别场景中的首选。考虑到服务机器人的硬件配置成本,将基于区域的卷积神经网络(R-CNN)的快速算法Faster R-CNN引入系统中,并以CPU计算的方式进行物品识别。利用CNN网络提取图像特征,在其后面接入一个区域提议层。实验结果表明,将深度学习的识别方法应用到服务机器人平台是可行的,识别效果准确,且在实验中得到较好的检测效果。

Multi-user reinforcement learning based task migration in mobile edge computing

Mobile Edge Computing(MEC)is a promising approach.Dynamic service migration is a key technology in MEC.In order to maintain the continuity of services in a dynamic environment,mobile users need to migrate tasks between multiple servers in real time.Due to the uncertainty of movement,frequent migration will increase delays and costs and non-migration will lead to service interruption.Therefore,it is very challenging to design an optimal migration strategy.In this paper,we investigate the multi-user task migration problem in a dynamic environment and minimizes the average service delay while meeting the migration cost.In order to optimize the service delay and migration cost,we propose an adaptive weight deep deterministic policy gradient(AWDDPG)algorithm.And distributed execution and centralized training are adopted to solve the high-dimensional problem.Experiments show that the proposed algorithm can greatly reduce the migration cost and service delay compared with the other related algorithms.

全矿井人工智能(AI)监管平台关键技术

以视频感知为基础,以网络、信息技术为媒介,以人工智能、大数据为技术支撑;构建了全矿井人工智能(AI)监管平台,介绍了平台的总体架构和核心场景;基于整个平台的设计,论述了低样本数据集增强技术、模型训练技术、数据推理与决策、微服务后台开发技术等关键技术。现场应用表明:全矿井人工智能(AI)监管平台的深度学习算法在矿山图像分类方面的准确率达到了90%以上,目标检测方面的准确率达到了80%以上。

基于深度学习网络的智能养殖AI算法服务平台构建方法

为将传统农业与现代科技实现有效结合,实现养殖过程的智能化,构建一种基于深度学习网络的智能养殖AI算法服务平台。该平台包括基于Yolox模型目标检测算法、基于Mask R-CNN模型的目标分割算法和虚拟化资源池,前两者分别实现牲畜点数和测长估重,后者实现完善硬件的调度支撑方法。这样的处理方式可以加快算法收敛速度、降低参数计算量、提升系统性能,适用于畜牧业中的图像处理任务,实现对牲畜的全面管理和监测。