强化学习方法在Web服务组合中的应用比较研究

A COMPARATIVE STUDY ON THE APPLICATIONS OF REINFORCEMENT LEARNING METHODS IN WEB SERVICE COMPOSITION

为了提高服务组合适应动态环境的能力,将强化学习技术引入到Web服务组合。目前常用的强化学习方法有三种:蒙特卡罗、时序差分和Q-Learning,为了发现最适合于服务组合的强化学习方法,对这三种方法进行了对比研究。首先将Web服务组合建模为马尔科夫决策过程,然后介绍了这三种强化学习方法并分析了它们的异同,同时,提出了Web服务组合领域的奖赏值确定方法。最后,通过实验比较了这三种强化学习方法的学习效果,实验结果显示,在Web服务组合应用中,Q-Learning比另外两种方法收敛速度更快,因此更适合执行服务组合。

In order to improve the ability of service composition to be adaptive to the dynamic environment,this paper applies reinforcement learning（RL） to Web service composition（Wsc）.At present there are three commonly used RL methods： Monte Carlo,temporal difference and Q-Learning.The paper makes comparisons and studies among the three methods.Firstly Wsc is modeled with Markov Decision Process,then the above three RL methods are introduced and compared with each other.An approach to define reward in Wsc is also proposed.Finally experiments are carried out to compare effects of the three RL methods.Experiment results illustrate that the Q-Learning method is faster at convergence than the other two RL methods,so it is better fit for execution of service composition.