The end-to-end delay in a wired network is strongly dependent on congestion on intermediate nodes.Among lots of feasible approaches to avoid congestion efficiently,congestion-aware routing protocols tend to search for...The end-to-end delay in a wired network is strongly dependent on congestion on intermediate nodes.Among lots of feasible approaches to avoid congestion efficiently,congestion-aware routing protocols tend to search for an uncongested path toward the destination through rule-based approaches in reactive/incident-driven and distributed methods.However,these previous approaches have a problem accommodating the changing network environments in autonomous and self-adaptive operations dynamically.To overcome this drawback,we present a new congestion-aware routing protocol based on a Q-learning algorithm in software-defined networks where logically centralized network operation enables intelligent control and management of network resources.In a proposed routing protocol,either one of uncongested neighboring nodes are randomly selected as next hop to distribute traffic load to multiple paths or Q-learning algorithm is applied to decide the next hop by modeling the state,Q-value,and reward function to set the desired path toward the destination.A new reward function that consists of a buffer occupancy,link reliability and hop count is considered.Moreover,look ahead algorithm is employed to update the Q-value with values within two hops simultaneously.This approach leads to a decision of the optimal next hop by taking congestion status in two hops into account,accordingly.Finally,the simulation results presented approximately 20%higher packet delivery ratio and 15%shorter end-to-end delay,compared to those with the existing scheme by avoiding congestion adaptively.展开更多
基金This work was supported by Institute for Information&communications Technology Planning&Evaluation(IITP)grant funded by the Korea government(MSIT)(No.2019-0-01343,Training Key Talents in Industrial Convergence Security)and Research Cluster Project,R20143,by Zayed University Research Office.
文摘The end-to-end delay in a wired network is strongly dependent on congestion on intermediate nodes.Among lots of feasible approaches to avoid congestion efficiently,congestion-aware routing protocols tend to search for an uncongested path toward the destination through rule-based approaches in reactive/incident-driven and distributed methods.However,these previous approaches have a problem accommodating the changing network environments in autonomous and self-adaptive operations dynamically.To overcome this drawback,we present a new congestion-aware routing protocol based on a Q-learning algorithm in software-defined networks where logically centralized network operation enables intelligent control and management of network resources.In a proposed routing protocol,either one of uncongested neighboring nodes are randomly selected as next hop to distribute traffic load to multiple paths or Q-learning algorithm is applied to decide the next hop by modeling the state,Q-value,and reward function to set the desired path toward the destination.A new reward function that consists of a buffer occupancy,link reliability and hop count is considered.Moreover,look ahead algorithm is employed to update the Q-value with values within two hops simultaneously.This approach leads to a decision of the optimal next hop by taking congestion status in two hops into account,accordingly.Finally,the simulation results presented approximately 20%higher packet delivery ratio and 15%shorter end-to-end delay,compared to those with the existing scheme by avoiding congestion adaptively.
