面向大规模AI数据流的接入算法和调度机制

Access Algorithms and Scheduling Mechanisms for Large-Scale AI Data Streams

在人工智能数据流边缘接入集群中,拥有更高的接入带宽固然重要,但高效的AI数据流接入算法和调度机制更能够充分发挥接入服务器网卡等硬件性能。论文提出针对大规模AI数据流的并发接入算法和调度机制。针对AI数据单元大小动态变化的不稳定性接入,设计区域动态分组接入算法和基于接入服务器资源预测的数据流迁移调度机制。集群实验结果表明,区域动态分组接入算法可以更好地满足大规模AI数据流接入请求;在保证接入服务器集群数据流总并发量前提下,基于资源预测的流调度机制使得接入服务器资源利用均衡,大大降低系统AI数据单元丢包率。

In the edge access cluster of artificial intelligence data flow,it is important to have higher access bandwidth,but the efficient AI data flow access algorithm and scheduling mechanism can better give full play to the hardware performance such as access server network card.This paper proposes a concurrent access algorithm and scheduling mechanism for large-scale AI data flow.Aiming at the unstable access of AI data unit with dynamic change in size,a area dynamic group access algorithm and a data flow migration and scheduling mechanism based on access server resource prediction are designed.The cluster experiment results show that the area dynamic group access algorithm can better satisfy the access request of large-scale AI data stream.On the prem-ise of ensuring the total concurrency of data flow in the access server cluster,the flow scheduling mechanism based on resource pre-diction makes the utilization of access server resources balanced and greatly reduces the packet loss rate of system AI data unit.