基于Q-learning的HTTP自适应流码率控制方法研究

Research on Q-learning based rate control approach for HTTP adaptive streaming

基于HTTP的自适应流HAS已经成为自适应视频流服务的标准。在HAS客户端网络状态多变的情况下,硬编码形式的码率决策方法灵活性偏低,对用户体验考虑不足。为了优化用户体验质量(Qo E),提出一种基于Q-Learning的码率控制算法,结合HTTP自适应视频流客户端环境进行建模并定义状态转移规则;量化与用户Qo E相关的参数,构建新的回报函数;实验表明引入Q-Learning进行码率调整的自适应算法在码率切换的稳定性方面表现较好。

HTTP adaptive streaming(HAS)has become the standard for adaptive video streaming service.In changing network environments,current hardcoded-based rate adaptation algorithm was less flexible,and it is insufficient to consider the quality of experience(QoE).To optimize the QoE of users,a rate control approach based on Q-learning strategy was proposed.the client environments of HTTP adaptive video streaming was modeled and the state transition rule was defined.Three parameters related to QoE were quantified and a novel reward function was constructed.The experiments were employed by the Q-learning rate control approach in two typical HAS algorithms.The experiments show the rate control approach can enhance the stability of rate switching in HAS clients.