Federated Feature Concatenate Method for Heterogeneous Computing in Federated Learning

Federated learning is an emerging machine learning techniquethat enables clients to collaboratively train a deep learning model withoutuploading raw data to the aggregation server. Each client may be equippedwith different computing resources for model training. The client equippedwith a lower computing capability requires more time for model training,resulting in a prolonged training time in federated learning. Moreover, it mayfail to train the entire model because of the out-of-memory issue. This studyaims to tackle these problems and propose the federated feature concatenate(FedFC) method for federated learning considering heterogeneous clients.FedFC leverages the model splitting and feature concatenate for offloadinga portion of the training loads from clients to the aggregation server. Eachclient in FedFC can collaboratively train a model with different cutting layers.Therefore, the specific features learned in the deeper layer of the serversidemodel are more identical for the data class classification. Accordingly,FedFC can reduce the computation loading for the resource-constrainedclient and accelerate the convergence time. The performance effectiveness isverified by considering different dataset scenarios, such as data and classimbalance for the participant clients in the experiments. The performanceimpacts of different cutting layers are evaluated during the model training.The experimental results show that the co-adapted features have a criticalimpact on the adequate classification of the deep learning model. Overall,FedFC not only shortens the convergence time, but also improves the bestaccuracy by up to 5.9% and 14.5% when compared to conventional federatedlearning and splitfed, respectively. In conclusion, the proposed approach isfeasible and effective for heterogeneous clients in federated learning.

A new heuristic for task scheduling in heterogeneous computing environment

Heterogeneous computing (HC) environment utilizes diverse resources with different computational capabilities to solve computing-intensive applications having diverse computational requirements and constraints. The task assignment problem in HC environment can be formally defined as for a given set of tasks and machines, assigning tasks to machines to achieve the minimum makespan. In this paper we propose a new task scheduling heuristic, high standard deviation first (HSTDF), which considers the standard deviation of the expected execution time of a task as a selection criterion. Standard deviation of the ex- pected execution time of a task represents the amount of variation in task execution time on different machines. Our conclusion is that tasks having high standard deviation must be assigned first for scheduling. A large number of experiments were carried out to check the effectiveness of the proposed heuristic in different scenarios, and the comparison with the existing heuristics (Max-min, Sufferage, Segmented Min-average, Segmented Min-min, and Segmented Max-min) clearly reveals that the proposed heuristic outperforms all existing heuristics in terms of average makespan.

Distributed Matching Theory-Based Task Re-Allocating for Heterogeneous Multi-UAV Edge Computing

Many efforts have been devoted to efficient task scheduling in Multi-Unmanned Aerial Vehicle(UAV)edge computing.However,the heterogeneity of UAV computation resource,and the task re-allocating between UAVs have not been fully considered yet.Moreover,most existing works neglect the fact that a task can only be executed on the UAV equipped with its desired service function(SF).In this backdrop,this paper formulates the task scheduling problem as a multi-objective task scheduling problem,which aims at maximizing the task execution success ratio while minimizing the average weighted sum of all tasks’completion time and energy consumption.Optimizing three coupled goals in a realtime manner with the dynamic arrival of tasks hinders us from adopting existing methods,like machine learning-based solutions that require a long training time and tremendous pre-knowledge about the task arrival process,or heuristic-based ones that usually incur a long decision-making time.To tackle this problem in a distributed manner,we establish a matching theory framework,in which three conflicting goals are treated as the preferences of tasks,SFs and UAVs.Then,a Distributed Matching Theory-based Re-allocating(DiMaToRe)algorithm is put forward.We formally proved that a stable matching can be achieved by our proposal.Extensive simulation results show that Di Ma To Re algorithm outperforms benchmark algorithms under diverse parameter settings and has good robustness.

FPGA Accelerators for Computing Interatomic Potential-Based Molecular Dynamics Simulation for Gold Nanoparticles:Exploring Different Communication Protocols

Molecular Dynamics(MD)simulation for computing Interatomic Potential(IAP)is a very important High-Performance Computing(HPC)application.MD simulation on particles of experimental relevance takes huge computation time,despite using an expensive high-end server.Heterogeneous computing,a combination of the Field Programmable Gate Array(FPGA)and a computer,is proposed as a solution to compute MD simulation efficiently.In such heterogeneous computation,communication between FPGA and Computer is necessary.One such MD simulation,explained in the paper,is the(Artificial Neural Network)ANN-based IAP computation of gold(Au_(147)&Au_(309))nanoparticles.MD simulation calculates the forces between atoms and the total energy of the chemical system.This work proposes the novel design and implementation of an ANN IAP-based MD simulation for Au_(147)&Au_(309) using communication protocols,such as Universal Asynchronous Receiver-Transmitter(UART)and Ethernet,for communication between the FPGA and the host computer.To improve the latency of MD simulation through heterogeneous computing,Universal Asynchronous Receiver-Transmitter(UART)and Ethernet communication protocols were explored to conduct MD simulation of 50,000 cycles.In this study,computation times of 17.54 and 18.70 h were achieved with UART and Ethernet,respectively,compared to the conventional server time of 29 h for Au_(147) nanoparticles.The results pave the way for the development of a Lab-on-a-chip application.

Optimization Task Scheduling Using Cooperation Search Algorithm for Heterogeneous Cloud Computing Systems

Cloud computing has taken over the high-performance distributed computing area,and it currently provides on-demand services and resource polling over the web.As a result of constantly changing user service demand,the task scheduling problem has emerged as a critical analytical topic in cloud computing.The primary goal of scheduling tasks is to distribute tasks to available processors to construct the shortest possible schedule without breaching precedence restrictions.Assignments and schedules of tasks substantially influence system operation in a heterogeneous multiprocessor system.The diverse processes inside the heuristic-based task scheduling method will result in varying makespan in the heterogeneous computing system.As a result,an intelligent scheduling algorithm should efficiently determine the priority of every subtask based on the resources necessary to lower the makespan.This research introduced a novel efficient scheduling task method in cloud computing systems based on the cooperation search algorithm to tackle an essential task and schedule a heterogeneous cloud computing problem.The basic idea of thismethod is to use the advantages of meta-heuristic algorithms to get the optimal solution.We assess our algorithm’s performance by running it through three scenarios with varying numbers of tasks.The findings demonstrate that the suggested technique beats existingmethods NewGenetic Algorithm(NGA),Genetic Algorithm(GA),Whale Optimization Algorithm(WOA),Gravitational Search Algorithm(GSA),and Hybrid Heuristic and Genetic(HHG)by 7.9%,2.1%,8.8%,7.7%,3.4%respectively according to makespan.

SCIRD: Revealing Infection of Malicious Software in Edge Computing-Enabled IoT Networks

The Internet of Things(IoT)has characteristics such as node mobility,node heterogeneity,link heterogeneity,and topology heterogeneity.In the face of the IoT characteristics and the explosive growth of IoT nodes,which brings about large-scale data processing requirements,edge computing architecture has become an emerging network architecture to support IoT applications due to its ability to provide powerful computing capabilities and good service functions.However,the defense mechanism of Edge Computing-enabled IoT Nodes(ECIoTNs)is still weak due to their limited resources,so that they are susceptible to malicious software spread,which can compromise data confidentiality and network service availability.Facing this situation,we put forward an epidemiology-based susceptible-curb-infectious-removed-dead(SCIRD)model.Then,we analyze the dynamics of ECIoTNs with different infection levels under different initial conditions to obtain the dynamic differential equations.Additionally,we establish the presence of equilibrium states in the SCIRD model.Furthermore,we conduct an analysis of the model’s stability and examine the conditions under which malicious software will either spread or disappear within Edge Computing-enabled IoT(ECIoT)networks.Lastly,we validate the efficacy and superiority of the SCIRD model through MATLAB simulations.These research findings offer a theoretical foundation for suppressing the propagation of malicious software in ECIoT networks.The experimental results indicate that the theoretical SCIRD model has instructive significance,deeply revealing the principles of malicious software propagation in ECIoT networks.This study solves a challenging security problem of ECIoT networks by determining the malicious software propagation threshold,which lays the foundation for buildingmore secure and reliable ECIoT networks.

Network simulation task partition method in heterogeneous computing environment

To reduce the running time of network simulation in heterogeneous computing environment,a network simulation task partition method,named LBPHCE,is put forward.In this method,the network simulation task is partitioned in comprehensive consideration of the load balance of both routing computing simulation and packet forwarding simulation.First,through benchmark experiments,the computation ability and routing simulation ability of each simulation machine are measured in the heterogeneous computing environment.Second,based on the computation ability of each simulation machine,the network simulation task is initially partitioned to meet the load balance of packet forwarding simulation in the heterogeneous computing environment,and then according to the routing computation ability,the scale of each partition is fine-tuned to satisfy the balance of the routing computing simulation,meanwhile the load balance of packet forwarding simulation is guaranteed.Experiments based on PDNS indicate that,compared to traditional uniform partition method,the LBPHCE method can reduce the total simulation running time by 26.3%in average,and compared to the liner partition method,it can reduce the running time by 18.3%in average.

Energy-Optimal and Delay-Bounded Computation Offloading in Mobile Edge Computing with Heterogeneous Clouds

By Mobile Edge Computing(MEC), computation-intensive tasks are offloaded from mobile devices to cloud servers, and thus the energy consumption of mobile devices can be notably reduced. In this paper, we study task offloading in multi-user MEC systems with heterogeneous clouds, including edge clouds and remote clouds. Tasks are forwarded from mobile devices to edge clouds via wireless channels, and they can be further forwarded to remote clouds via the Internet. Our objective is to minimize the total energy consumption of multiple mobile devices, subject to bounded-delay requirements of tasks. Based on dynamic programming, we propose an algorithm that minimizes the energy consumption, by jointly allocating bandwidth and computational resources to mobile devices. The algorithm is of pseudo-polynomial complexity. To further reduce the complexity, we propose an approximation algorithm with energy discretization, and its total energy consumption is proved to be within a bounded gap from the optimum. Simulation results show that, nearly 82.7% energy of mobile devices can be saved by task offloading compared with mobile device execution.

Joint Resource Allocation Using Evolutionary Algorithms in Heterogeneous Mobile Cloud Computing Networks

The problem of joint radio and cloud resources allocation is studied for heterogeneous mobile cloud computing networks. The objective of the proposed joint resource allocation schemes is to maximize the total utility of users as well as satisfy the required quality of service(QoS) such as the end-to-end response latency experienced by each user. We formulate the problem of joint resource allocation as a combinatorial optimization problem. Three evolutionary approaches are considered to solve the problem: genetic algorithm(GA), ant colony optimization with genetic algorithm(ACO-GA), and quantum genetic algorithm(QGA). To decrease the time complexity, we propose a mapping process between the resource allocation matrix and the chromosome of GA, ACO-GA, and QGA, search the available radio and cloud resource pairs based on the resource availability matrixes for ACOGA, and encode the difference value between the allocated resources and the minimum resource requirement for QGA. Extensive simulation results show that our proposed methods greatly outperform the existing algorithms in terms of running time, the accuracy of final results, the total utility, resource utilization and the end-to-end response latency guaranteeing.

System Support for Parallel Computing on Heterogeneous Networks of Workstations

Abstract In this paper, we introduce several on-going research projects to support parallel and distribut,ed computing on heterogeneous networks of workstations (NOW) in the High Performance Computing and Software Lahoratory at the University of Texas at San Antonio. The projects at aiming at addressing three technical issues. First, the factors of heterogeneity and time-sharing effects make traditional performance models/metrics for homogeneous computing performance measurement and evaluation not. suitable for bet.erogeneous computing. We develop practical models and metrics which quantify. the heterogeneity of networks and characterize the performance effects. Second, in order to perform parallel computation effectively, special system support is necessary. We are developing system schemes for heterogeneity management, process scheduling and efficient communications. Finally, to provide insight into system performance, we are developing two types of supporting tools : a graphical instrumentation monitor to aid users in investigating performance problems and in determining the most effective way of exploiting the NOW systems, and a trace-driven simulator to test and compare different system management and scheduling schemes.

Programming for scientific computing on peta-scale heterogeneous parallel systems

Peta-scale high-perfomlance computing systems are increasingly built with heterogeneous CPU and GPU nodes to achieve higher power efficiency and computation throughput. While providing unprecedented capabilities to conduct computational experiments of historic significance, these systems are presently difficult to program. The users, who are domain experts rather than computer experts, prefer to use programming models closer to their domains （e.g., physics and biology） rather than MPI and OpenME This has led the development of domain-specific programming that provides domain-specific programming interfaces but abstracts away some performance-critical architecture details. Based on experience in designing large-scale computing systems, a hybrid programming framework for scientific computing on heterogeneous architectures is proposed in this work. Its design philosophy is to provide a collaborative mechanism for domain experts and computer experts so that both domain-specific knowledge and performance-critical architecture details can be adequately exploited. Two real-world scientific applications have been evaluated on TH-IA, a peta-scale CPU-GPU heterogeneous system that is currently the 5th fastest supercomputer in the world. The experimental results show that the proposed framework is well suited for developing large-scale scientific computing applications on peta-scale heterogeneous CPU/GPU systems.

An Improved Model for Computing-Intensive Tasks on Heterogeneous Workstations

An improved algorithm, which solves cooperative concurrent computing tasks using the idle cycles of a number of high performance heterogeneous workstations interconnected through a high-speed network, was proposed. In order to get better parallel computation performance, this paper gave a model and an algorithm of task scheduling among heterogeneous workstations, in which the costs of loading data, computing, communication and collecting results are considered. Using this efficient algorithm, an optimal subset of heterogeneous workstations with the shortest parallel executing time of tasks can be selected.

Auction-based profit maximization offloading in mobile edge computing

Offloading Mobile Devices(MDs)computation tasks to Edge Nodes(ENs)is a promising solution to overcome computation and energy resources limitations of MDs.However,there exists an unreasonable profit allocation problem between MDs and ENs caused by the excessive concern on MD profit.In this paper,we propose an auction-based computation offloading algorithm,inspiring ENs to provide high-quality service by maximizing the profit of ENs.Firstly,a novel cooperation auction framework is designed to avoid overall profit damage of ENs,which is derived from the high computation delay at the overloaded ENs.Thereafter,the bidding willingness of each MD in every round of auction is determined to ensure MD rationality.Furthermore,we put forward a payment rule for the pre-selected winner to effectively guarantee auction truthfulness.Finally,the auction-based profit maximization offloading algorithm is proposed,and the MD is allowed to occupy the computation and spectrum resources of the EN for offloading if it wins the auction.Numerical results verify the performance of the proposed algorithm.Compared with the VA algorithm,the ENs profit is increased by 23.8%,and the task discard ratio is decreased by 7.5%.

eMD:基于异构计算的大规模分子动力学模拟软件

【目的】异构计算已经成为高性能计算的重要组成部分,GPU异构计算可显著提速计算密集型的分子动力学模拟应用,本文介绍自研分子动力学模拟软件eMD的系统设计及其异构计算应用。【方法】首先介绍eMD软件的目标定位,包括应用功能和计算性能两方面;然后介绍软件概要设计,包括框架、模块和接口等组成部分;重点围绕面向异构计算的软件架构设计和移植优化技术进行阐述。【结果】eMD软件系统基于GPU异构计算可实现大规模体系模拟,同时提供特色的分子动力学模拟算法和模型。【结论】eMD将充分发挥GPU异构计算算力,以提升分子动力学模拟应用效率,助力分子建模理论方法的创新应用和分子科学问题的研究。

异构边缘云架构下的多任务卸载算法

为在资源有限的终端设备上运行计算密集型与时延敏感型应用,同时降低系统时延和能耗,构建边缘云异构网络模型。本文提出了一种H-PSOGA多任务卸载优化算法,并通过无人机、路边单元、车辆等边缘设备以及边缘云服务器进行多任务计算卸载。该算法以先串行再并行的方式将粒子群和遗传算法结合在一起,通过适应度值排序、种群选择、多点交叉、反向变异等操作,利用遗传算法对粒子群进行优选,弥补粒子群算法早熟收敛、陷入局部最优的缺陷。6种标准测试函数的测试分析以及与基线方案进行仿真对比的结果表明:在用户数较多时,混合优化算法的系统平均开销可降低26%~43%,可以有效提高收敛精度。

基于云边协同的通用板级自动测试系统方案设计

针对当前数字电路自动测试领域,对通用板级自动测试系统进行了设计,完成了系统方案的设计开发和测试验证。为了解决通用板级自动测试系统的测试数据量大、本地存储容量不足、测试向量与被测试目标无法自动化匹配、无法进行批量快速测试等问题,采用云边协同架构设计了通用板级自动测试系统,避免了大量数据传输和数据集中式处理,使得云端计算资源能够集中解决关键数据处理任务。为了提高板级测试平台的通用性,使平台适配不同接口的电路板,同时具有高速网络数据处理能力,系统选用精简命令集处理器和现场可编程门阵列(Advanced RISC machines,Filed programmable gate array,ARM+FPGA)的异构计算平台作为边缘设备。为了提高系统的测试效率,采用模块化的设计思想,系统硬件测试平台设计了多总线分布式结构。系统采用扇出导向(Fan-out-oriented,FAN)算法生成测试向量,并基于浏览器/服务器(Brower/Server,B/S)架构设计了用户操作界面,用户可通过浏览器进行测试操作,完成板级自动故障测试,并自动生成故障诊断报告。实验结果表明,对两个待测板卡进行测试验证,通用板级自动测试系统可以自动识别目标板卡,自动匹配测试向量,进行自动测试,生成测试报表。基于云边协同的通用板级自动测试系统提高了测试效率,满足数字电路板卡出厂前批量测试的需求,具有实际应用价值。

基于数据-模型混合驱动的电力系统机电暂态快速仿真方法

数据驱动建模方法改变了发电机传统的建模范式,导致传统的机电暂态时域仿真方法无法直接应用于新范式下的电力系统。为此,该文提出一种基于数据-模型混合驱动的机电暂态时域仿真(data and physics driven time domain simulation,DPD-TDS)算法。算法中发电机状态变量与节点注入电流通过数据驱动模型推理计算,并通过网络方程完成节点电压计算,两者交替求解完成仿真。算法提出一种混合驱动范式下的网络代数方程组预处理方法,用以改善仿真的收敛性;算法设计一种中央处理器单元-神经网络处理器单元(central processing unit-neural network processing unit,CPU-NPU)异构计算框架以加速仿真,CPU进行机理模型的微分代数方程求解;NPU作协处理器完成数据驱动模型的前向推理。最后在IEEE-39和Polish-2383系统中将部分或全部发电机替换为数据驱动模型进行验证,仿真结果表明,所提出的仿真算法收敛性好,计算速度快,结果准确。

基于矩阵乘积态的有限纠缠量子傅里叶变换模拟

与经典计算不同,在量子计算中量子比特可以处于叠加态,多个量子比特之间还可以形成纠缠态。表示n个量子比特组成的量子态需要存储2^(n)个振幅,这种指数级的存储开销使得大规模的量子模拟难以进行。然而当量子态的纠缠程度有限时,使用矩阵乘积态表示量子态仅需要线性的空间复杂度,可以扩大模拟的规模。使用HIP-Clang语言,基于CPU+DCU的异构编程模型,使用矩阵乘积态表示量子态,对量子傅里叶变换进行模拟。结合矩阵乘积态的特点,对量子傅里叶变换线路进行分析,减少模拟实现时不必要的张量缩并运算与正交化构建。对模拟过程中的张量缩并进行分析,使用TTGT算法完成张量缩并运算,同时利用DCU的并行处理能力来提高效率。对模拟结果进行分析,分别通过振幅误差与半经典Draper量子加法器的结果验证了模拟的正确性。对模拟规模进行分析,当量子态的纠缠熵最大时,使用16 GB的内存空间最多只能模拟24位的量子态,而当量子态内部纠缠程度较低时,可以对上百位的量子态进行量子傅里叶变换模拟。

基于启发式算法的计算机异构大数据跨源调度方法

为了提高计算机异构大数据跨源调度性能,设计了一种基于启发式算法的计算机异构大数据跨源调度方法。将计算机异构大数据跨源调度任务划分为若干子任务,利用max-min思想和min-max思想,构建了跨源调度时间负载均衡模型。基于计算机异构大数据跨源调度顺序的灵活性,利用启发式算法的最优策略,选择最优调度任务。根据计算机异构大数据的状态,计算出异构大数据传输控制协议连接的吞吐量,将待调度异构大数据的质量都作为计算机数据层的子流权重,通过处理所有待调度的子流,实现计算机异构大数据跨源调度。实验结果表明,文中设计的方法可以将跨源操作的级别条件提高至10级,跨源调度计算机异构大数据的利用率超过97%,加速比大于85%,计算机异构大数据跨源调度性能明显提升。