基于强化学习的智能I/O调度算法

An Intelligent I/O Scheduling Algorithm Based on Reinforcement Learning

利用机器学习方法解决存储领域中若干技术难题是目前存储领域的研究热点之一。强化学习作为一种以环境反馈作为输入、自适应环境的特殊的机器学习方法,能通过观测环境状态的变化,评估控制决策对系统性能的影响来选择最优的控制策略,基于强化学习的智能RAID控制技术具有重要的研究价值。本文针对高性能计算应用特点,将机器学习领域中的强化学习技术引入RAID控制器中,提出了基于强化学习的智能I/O调度算法RL-scheduler,利用Q-学习策略实现了面向并行应用的自治调度策略。RL-scheduler综合考虑了调度的公平性、磁盘寻道时间和MPI应用的I/O访问效率,并提出多Q-表交叉组织方法提高Q-表的更新效率。实验结果表明,RL-scheduler缩短了并行应用的平均I/O服务时间,提高了大规模并行计算系统的I/O吞吐率。

To improve the I/O service efficiency of RAID and optimize the I/O performance of parallel applications,the paper presents an intelligent I/O scheduling algorithm,RLscheduler,in the RAID controllers based on reinforcement learning.RLscheduler utilizes the Qlearning strategy to implement a selfcontrol and selfoptimization scheduler.The algorithm leverages the scheduling equity,disk seeking time and the I/O access efficiency of the MPI applications.Furthermore,the proposed interleaving organization of multiple Qtables improves the efficiency of the Qtable updating.The experimental results show that,on a largescale parallel system with multiple parallel applications,RLscheduler shortens the average I/O waiting time of parallel applications considerably,thus increases the practical I/O throughput of largescale parallel systems.