Deep Reinforcement Learning-Based Task Offloading and Service Migrating Policies in Service Caching-Assisted Mobile Edge Computing

Emerging mobile edge computing(MEC)is considered a feasible solution for offloading the computation-intensive request tasks generated from mobile wireless equipment(MWE)with limited computational resources and energy.Due to the homogeneity of request tasks from one MWE during a longterm time period,it is vital to predeploy the particular service cachings required by the request tasks at the MEC server.In this paper,we model a service caching-assisted MEC framework that takes into account the constraint on the number of service cachings hosted by each edge server and the migration of request tasks from the current edge server to another edge server with service caching required by tasks.Furthermore,we propose a multiagent deep reinforcement learning-based computation offloading and task migrating decision-making scheme(MBOMS)to minimize the long-term average weighted cost.The proposed MBOMS can learn the near-optimal offloading and migrating decision-making policy by centralized training and decentralized execution.Systematic and comprehensive simulation results reveal that our proposed MBOMS can converge well after training and outperforms the other five baseline algorithms.

Deep learning framework for multi‐round service bundle recommendation in iterative mashup development

Recent years have witnessed the rapid development of service‐oriented computing technologies.The boom of Web services increases software developers'selection burden in developing new service‐based systems such as mashups.Timely recommending appropriate component services for developers to build new mashups has become a fundamental problem in service‐oriented software engineering.Existing service recom-mendation approaches are mainly designed for mashup development in the single‐round scenario.It is hard for them to effectively update recommendation results according to developers'requirements and behaviours(e.g.instant service selection).To address this issue,the authors propose a service bundle recommendation framework based on deep learning,DLISR,which aims to capture the interactions among the target mashup to build,selected(component)services,and the following service to recommend.Moreover,an attention mechanism is employed in DLISR to weigh selected services when rec-ommending a candidate service.The authors also design two separate models for learning interactions from the perspectives of content and invocation history,respectively,and a hybrid model called HISR.Experiments on a real‐world dataset indicate that HISR can outperform several state‐of‐the‐art service recommendation methods to develop new mashups iteratively.

Learning-Based Joint Service Caching and Load Balancing for MEC Blockchain Networks

Integrating the blockchain technology into mobile-edge computing(MEC)networks with multiple cooperative MEC servers(MECS)providing a promising solution to improving resource utilization,and helping establish a secure reward mechanism that can facilitate load balancing among MECS.In addition,intelligent management of service caching and load balancing can improve the network utility in MEC blockchain networks with multiple types of workloads.In this paper,we investigate a learningbased joint service caching and load balancing policy for optimizing the communication and computation resources allocation,so as to improve the resource utilization of MEC blockchain networks.We formulate the problem as a challenging long-term network revenue maximization Markov decision process(MDP)problem.To address the highly dynamic and high dimension of system states,we design a joint service caching and load balancing algorithm based on the double-dueling Deep Q network(DQN)approach.The simulation results validate the feasibility and superior performance of our proposed algorithm over several baseline schemes.

A rapid, low-cost deep learning system to classify strawberry disease based on cloud service

Accurate and timely classification of diseases during strawberry planting can help growers deal with them in timely manner, thereby reducing losses. However, the classification of strawberry diseases in real planting environments is facing severe challenges, including complex planting environments, multiple disease categories with small differences, and so on. Although recent mobile vision technology based deep learning has achieved some success in overcoming the above problems, a key problem is how to construct a non-destructive, fast and convenient method to improve the efficiency of strawberry disease identification for the multi-region, multi-space and multi-time classification requirements. We develop and evaluate a rapid, low-cost system for classifying diseases in strawberry cultivation. This involves designing an easy-to-use cloudbased strawberry disease identification system, combined with our novel self-supervised multi-network fusion classification model, which consists of a Location network, a Feedback network and a Classification network to identify the categories of common strawberry diseases. With the help of a novel self-supervision mechanism, the model can effectively identify diseased regions of strawberry disease images without the need for annotations such as bounding boxes. Using accuracy, precision, recall and F1 to evaluate the classification effect, the results of the test set are 92.48, 90.68, 86.32 and 88.45%, respectively. Compared with popular Convolutional Neural Networks(CNN) and five other methods, our network achieves better disease classification effect. Currently, the client(mini program) has been released on the We Chat platform. The mini program has perfect classification effect in the actual test, which verifies the feasibility and effectiveness of the system, and can provide a reference for the intelligent research and application of strawberry disease identification.

Length matters:Scalable fast encrypted internet traffic service classification based on multiple protocol data unit length sequence with composite deep learning

As an essential function of encrypted Internet traffic analysis,encrypted traffic service classification can support both coarse-grained network service traffic management and security supervision.However,the traditional plaintext-based Deep Packet Inspection(DPI)method cannot be applied to such a classification.Moreover,machine learning-based existing methods encounter two problems during feature selection:complex feature overcost processing and Transport Layer Security(TLS)version discrepancy.In this paper,we consider differences between encryption network protocol stacks and propose a composite deep learning-based method in multiprotocol environments using a sliding multiple Protocol Data Unit(multiPDU)length sequence as features by fully utilizing the Markov property in a multiPDU length sequence and maintaining suitability with a TLS-1.3 environment.Control experiments show that both Length-Sensitive(LS)composite deep learning model using a capsule neural network and LS-long short time memory achieve satisfactory effectiveness in F1-score and performance.Owing to faster feature extraction,our method is suitable for actual network environments and superior to state-of-the-art methods.

Dynamic Security SFC Branching Path Selection Using Deep Reinforcement Learning

Security service function chaining(SFC)based on software-defined networking(SDN)and network function virtualization(NFV)technology allows traffic to be forwarded sequentially among different security service functions to achieve a combination of security functions.Security SFC can be deployed according to requirements,but the current SFC is not flexible enough and lacks an effective feedback mechanism.The SFC is not traffic aware and the changes of traffic may cause the previously deployed security SFC to be invalid.How to establish a closed-loop mechanism to enhance the adaptive capability of the security SFC to malicious traffic has become an important issue.Our contribution is threefold.First,we propose a secure SFC path selection framework.The framework can accept the feedback results of traffic and security service functions in SFC,and dynamically select the opti-mal path for SFC based on the feedback results.It also realizes the automatic deployment of paths,forming a complete closed loop.Second,we expand the protocol of SFC to realize the security SFC with branching path,which improve flexibility of security SFC.Third,we propose a deep reinforcement learning-based dynamic path selection method for security SFC.It infers the optimal branching path by analyzing feedback from the security SFC.We have experimented with Distributed Denial of Service(DDoS)attack detection modules as security service functions.Experimental results show that our proposed method can dynamically select the optimal branching path for a security SFC based on traffic features and the state of the SFC.And it improves the accuracy of the overall malicious traffic detection of the security SFC and significantly reduces the latency and overall load of the SFC.

MC-infer:DLaaS中的零知识和无真实数据模型推理攻击

深度学习即服务(DLaaS)的服务模式容易受到模型推理攻击影响。现有的推理攻击要求攻击者拥有足够的辅助信息来进行推理,这并不能完全展示出推理攻击的潜在威胁,因此,提出了MC-infer,一种零知识、无真实数据的黑盒模型推理攻击。MC-infer将从不同随机分布获得的随机噪声输入给目标模型,并根据其输出估计相应的目标分布进行模型推理。使用了蒙特卡洛对MC-infer进行了理论分析,证明了其在理论层面的可行性。实验表明MCinfer可以有效地推断目标模型。此外,研究了MC-infer的局限性和复杂性,最后讨论了几种防止攻击的策略。

Deep learning Optical Character Recognition in PCB Dark Silk Recognition

For Automatic Optical Inspection (AOI) machines that were introduced to Printed Circuit Board market more than five years ago, illumination technique and light devices are outdated. Images captured by old AOI machines are not easy to be recognized by typical optical character recognition (OCR) algorithms, especially for dark silk. How to effectively increase silk recognition accuracy is indispensable for improving overall production efficiency in SMT plant. This paper uses fine tuned Character Region Awareness for Text Detection (CRAFT) method to build model for dark silk recognition. CRAFT model consists of a structure similar to U-net, followed by VGG based convolutional neural network. Continuous two-dimensional Gaussian distribution was used for the annotation of image segmentation. CRAFT model is good at recognizing different types of printed characters with high accuracy and transferability. Results show that with the help of CRAFT model, accuracy for OK board is 95% (error rate is 5%), and accuracy for NG board is 100% (omission rate is 0%).

Privacy-preserving deep learning techniques for wearable sensor-based big data applications

Wearable technologies have the potential to become a valuable influence on human daily life where they may enable observing the world in new ways,including,for example,using augmented reality(AR)applications.Wearable technology uses electronic devices that may be carried as accessories,clothes,or even embedded in the user's body.Although the potential benefits of smart wearables are numerous,their extensive and continual usage creates several privacy concerns and tricky information security challenges.In this paper,we present a comprehensive survey of recent privacy-preserving big data analytics applications based on wearable sensors.We highlight the fundamental features of security and privacy for wearable device applications.Then,we examine the utilization of deep learning algorithms with cryptography and determine their usability for wearable sensors.We also present a case study on privacy-preserving machine learning techniques.Herein,we theoretically and empirically evaluate the privacy-preserving deep learning framework's performance.We explain the implementation details of a case study of a secure prediction service using the convolutional neural network(CNN)model and the Cheon-Kim-Kim-Song(CHKS)homomorphic encryption algorithm.Finally,we explore the obstacles and gaps in the deployment of practical real-world applications.Following a comprehensive overview,we identify the most important obstacles that must be overcome and discuss some interesting future research directions.

An effective communication and computation model based on a hybridgraph-deeplearning approach for SIoT

Social Edge Service(SES)is an emerging mechanism in the Social Internet of Things(SIoT)orchestration for effective user-centric reliable communication and computation.The services are affected by active and/or passive attacks such as replay attacks,message tampering because of sharing the same spectrum,as well as inadequate trust measurement methods among intelligent devices(roadside units,mobile edge devices,servers)during computing and content-sharing.These issues lead to computation and communication overhead of servers and computation nodes.To address this issue,we propose the HybridgrAph-Deep-learning(HAD)approach in two stages for secure communication and computation.First,the Adaptive Trust Weight(ATW)model with relation-based feedback fusion analysis to estimate the fitness-priority of every node based on directed graph theory to detect malicious nodes and reduce computation and communication overhead.Second,a Quotient User-centric Coeval-Learning(QUCL)mechanism to formulate secure channel selection,and Nash equilibrium method for optimizing the communication to share data over edge devices.The simulation results confirm that our proposed approach has achieved effective communication and computation performance,and enhanced Social Edge Services(SES)reliability than state-of-the-art approaches.

Toward Secure Software-Defined Networks Using Machine Learning: A Review, Research Challenges, and Future Directions

Over the past few years,rapid advancements in the internet and communication technologies have led to increasingly intricate and diverse networking systems.As a result,greater intelligence is necessary to effectively manage,optimize,and maintain these systems.Due to their distributed nature,machine learning models are challenging to deploy in traditional networks.However,Software-Defined Networking(SDN)presents an opportunity to integrate intelligence into networks by offering a programmable architecture that separates data and control planes.SDN provides a centralized network view and allows for dynamic updates of flow rules and softwarebased traffic analysis.While the programmable nature of SDN makes it easier to deploy machine learning techniques,the centralized control logic also makes it vulnerable to cyberattacks.To address these issues,recent research has focused on developing powerful machine-learning methods for detecting and mitigating attacks in SDN environments.This paper highlighted the countermeasures for cyberattacks on SDN and how current machine learningbased solutions can overcome these emerging issues.We also discuss the pros and cons of using machine learning algorithms for detecting and mitigating these attacks.Finally,we highlighted research issues,gaps,and challenges in developing machine learning-based solutions to secure the SDN controller,to help the research and network community to develop more robust and reliable solutions.

灾害场景下基于MADRL的信息收集无人机部署与节点能效优化

灾害场景下,对灾区内第一手重要信息的及时、可靠收集是灾害预警研究、灾区救援工作开展的关键。无人机是与灾区内部建立应急通信网络的高效辅助工具。通过对现有研究中应急场景下无人机的部署方法进行调查,指出了无人机部署时对节点能效考虑不充分的问题。由于地面传感器节点位于灾区内部,环境恶劣且极为被动,所以结合灾害场景,首次以提高地面节点能效为优化目标,基于深度强化学习方法,在DDQN网络模型基础上,通过自定义经验回放优先级、合理设计奖励函数和采用完全去中心化训练方式,解决该特定场景下用于信息收集无人机的自适应部署问题。仿真结果表明,所提算法的节点能源效率比DDQN基准算法提高21%,训练速度相比DDPG、A3C算法分别提升42%和34%。

Multi-Agent Deep Q-Networks for Efficient Edge Federated Learning Communications in Software-Defined IoT

Federated learning(FL)activates distributed on-device computation techniques to model a better algorithm performance with the interaction of local model updates and global model distributions in aggregation averaging processes.However,in large-scale heterogeneous Internet of Things(IoT)cellular networks,massive multi-dimensional model update iterations and resource-constrained computation are challenging aspects to be tackled significantly.This paper introduces the system model of converging softwaredefined networking(SDN)and network functions virtualization(NFV)to enable device/resource abstractions and provide NFV-enabled edge FL(eFL)aggregation servers for advancing automation and controllability.Multi-agent deep Q-networks(MADQNs)target to enforce a self-learning softwarization,optimize resource allocation policies,and advocate computation offloading decisions.With gathered network conditions and resource states,the proposed agent aims to explore various actions for estimating expected longterm rewards in a particular state observation.In exploration phase,optimal actions for joint resource allocation and offloading decisions in different possible states are obtained by maximum Q-value selections.Action-based virtual network functions(VNF)forwarding graph(VNFFG)is orchestrated to map VNFs towards eFL aggregation server with sufficient communication and computation resources in NFV infrastructure(NFVI).The proposed scheme indicates deficient allocation actions,modifies the VNF backup instances,and reallocates the virtual resource for exploitation phase.Deep neural network(DNN)is used as a value function approximator,and epsilongreedy algorithm balances exploration and exploitation.The scheme primarily considers the criticalities of FL model services and congestion states to optimize long-term policy.Simulation results presented the outperformance of the proposed scheme over reference schemes in terms of Quality of Service(QoS)performance metrics,including packet drop ratio,packet drop counts,packet delivery ratio,delay,and throughput.

HDLIDP: A Hybrid Deep Learning Intrusion Detection and Prevention Framework

Distributed denial-of-service(DDoS)attacks are designed to interrupt network services such as email servers and webpages in traditional computer networks.Furthermore,the enormous number of connected devices makes it difficult to operate such a network effectively.Software defined networks(SDN)are networks that are managed through a centralized control system,according to researchers.This controller is the brain of any SDN,composing the forwarding table of all data plane network switches.Despite the advantages of SDN controllers,DDoS attacks are easier to perpetrate than on traditional networks.Because the controller is a single point of failure,if it fails,the entire network will fail.This paper offers a Hybrid Deep Learning Intrusion Detection and Prevention(HDLIDP)framework,which blends signature-based and deep learning neural networks to detect and prevent intrusions.This framework improves detection accuracy while addressing all of the aforementioned problems.To validate the framework,experiments are done on both traditional and SDN datasets;the findings demonstrate a significant improvement in classification accuracy.

基于Faster R-CNN的服务机器人物品识别研究

传统的日用商品识别流程通常使用较为经典的图像识别和机器学习算法,如支持向量机(SVM)、随机森林或AdaBoost,然后利用目标图像的梯度、纹理或颜色的基本特征来对日用商品进行识别,可以在比较简单的背景中得到应用,但是在复杂的背景环境中很难有比较突出的表现,并且难以达到较高的准确率。目前在目标识别中表现比较优异的是卷积神经网络(CNN),并成为很多目标识别场景中的首选。考虑到服务机器人的硬件配置成本,将基于区域的卷积神经网络(R-CNN)的快速算法Faster R-CNN引入系统中,并以CPU计算的方式进行物品识别。利用CNN网络提取图像特征,在其后面接入一个区域提议层。实验结果表明,将深度学习的识别方法应用到服务机器人平台是可行的,识别效果准确,且在实验中得到较好的检测效果。

全矿井人工智能(AI)监管平台关键技术

以视频感知为基础,以网络、信息技术为媒介,以人工智能、大数据为技术支撑;构建了全矿井人工智能(AI)监管平台,介绍了平台的总体架构和核心场景;基于整个平台的设计,论述了低样本数据集增强技术、模型训练技术、数据推理与决策、微服务后台开发技术等关键技术。现场应用表明:全矿井人工智能(AI)监管平台的深度学习算法在矿山图像分类方面的准确率达到了90%以上,目标检测方面的准确率达到了80%以上。

面向缓存的动态协作任务迁移技术研究

边缘网络中不断出现的计算密集和延迟敏感型业务推动了任务迁移技术的快速发展。然而,任务迁移过程中存在应用场景复杂多变、问题建模难度高等技术瓶颈。尤其是考虑用户移动时,如何保证用户服务的稳定性和连续性,设计合理的任务迁移策略仍是一个值得深入探讨的问题。因此,提出了一种移动感知的服务预缓存模型和任务预迁移策略,将任务迁移问题转化为最优分簇与边缘服务预缓存的组合优化问题。首先,基于用户的移动轨迹对当前执行任务状态进行预测,引入动态协作簇和迁移预测半径的概念,提出了一种面向移动和负载两种任务场景的预迁移模型,解决了何时何地迁移的问题。然后,针对需要迁移的任务,基于最大容忍时延约束分析协作簇半径和簇内目标服务器数量的极限值,提出了以用户为中心的分布式多服务器间动态协作分簇算法(Distributed Dynamic Multi-server Cooperative Clustering Algorithm,DDMC)以及面向服务缓存的深度强化学习算法(Cache Based Double Deep Q Network,C-DDQN),解决了最优分簇和服务缓存问题。最后,利用服务缓存的因果关系,设计了一种低复杂度的交替最小化服务缓存位置更新算法,求解出了最佳迁移目标服务器集合,实现了任务迁移中的服务器协作及网络负载均衡。实验结果表明,提出的迁移选择算法具有良好的鲁棒性和系统性能,相比其他迁移算法所消耗的总成本降低了至少12.06%,所消耗的总时延降低了至少31.92%。

基于深度学习网络的智能养殖AI算法服务平台构建方法

为将传统农业与现代科技实现有效结合,实现养殖过程的智能化,构建一种基于深度学习网络的智能养殖AI算法服务平台。该平台包括基于Yolox模型目标检测算法、基于Mask R-CNN模型的目标分割算法和虚拟化资源池,前两者分别实现牲畜点数和测长估重,后者实现完善硬件的调度支撑方法。这样的处理方式可以加快算法收敛速度、降低参数计算量、提升系统性能,适用于畜牧业中的图像处理任务,实现对牲畜的全面管理和监测。

结合深度强化学习的边缘计算网络服务功能链时延优化部署方法

该文针对边缘网络资源受限且对业务流端到端时延容忍度低的问题,结合深度强化学习与基于时延的Dijkstra寻路算法提出一种面向时延优化的服务功能链(SFC)部署方法。首先,设计一种基于注意力机制的序列到序列(Seq2Seq)代理网络和基于时延的Dijkstra寻路算法,用于产生虚拟网络功能(VNF)的部署以及服务SFC的链路映射,同时考虑了时延优化模型的约束问题,采用拉格朗日松弛技术将其纳入强化学习目标函数中;其次,为了辅助网络代理快速收敛,采用基线评估器网络评估部署策略的预期奖励值;最后,在测试阶段,通过贪婪搜索及抽样技术降低网络收敛到局部最优的概率,从而改进模型的部署。对比实验表明,该方法在网络资源受限的情况下,比First-Fit算法与TabuSearch算法的时延分别降低了约10%和86.3%,且较这两种算法稳定约74.2%与84.4%。该方法能较稳定地提供更低时延的端到端服务,使时延敏感类业务获得更好体验。

基于联合神经网络的投诉预测模型研究

对影响电信运营商重复投诉的关键因素进行深入探讨,旨在提高服务质量并构建风险预测模型。基于运营商客服数据,研究采用了Logistic回归、BP神经网络以及二者联合建模的方法。Logistic回归模型确定了5个主要影响因素,预测重复投诉发生的概率,精度达到80.0%。BP神经网络则选取了81个影响因素,预测精度为90.6%。在此基础上,构建了联合模型,其精度高达92.8%。实际应用于某省会电信运营商后,重复投诉率下降了3.2%,成效显著,为提高电信运营商服务质量、降低重复投诉率提供了有力支持,对我国电信行业发展具有重要意义。