A context-aware service in a smart environment aims to supply services according to user situational information,which changes dynamically.Most existing context-aware systems provide context-aware services based on s...A context-aware service in a smart environment aims to supply services according to user situational information,which changes dynamically.Most existing context-aware systems provide context-aware services based on supervised algorithms.Reinforcement algorithms are another type of machine-learning algorithm that have been shown to be useful in dynamic environments through trialand-error interactions.They also have the ability to build excellent self-adaptive systems.In this study,we aim to incorporate reinforcement algorithms(Q-learning)into a context-aware system to provide relevant services based on a user’s dynamic context.To accelerate the convergence of reinforcement learning(RL)algorithms and provide the correct services in real situations,we propose a combination of the Q-learning and case-based reasoning(CBR)algorithms.We then analyze how the incorporation of CBR enables Q-learning to become more effi-cient and adapt to changing environments by continuously producing suitable services.Simulation results demonstrate the effectiveness of the proposed approach compared to the traditional CBR approach.展开更多
文摘A context-aware service in a smart environment aims to supply services according to user situational information,which changes dynamically.Most existing context-aware systems provide context-aware services based on supervised algorithms.Reinforcement algorithms are another type of machine-learning algorithm that have been shown to be useful in dynamic environments through trialand-error interactions.They also have the ability to build excellent self-adaptive systems.In this study,we aim to incorporate reinforcement algorithms(Q-learning)into a context-aware system to provide relevant services based on a user’s dynamic context.To accelerate the convergence of reinforcement learning(RL)algorithms and provide the correct services in real situations,we propose a combination of the Q-learning and case-based reasoning(CBR)algorithms.We then analyze how the incorporation of CBR enables Q-learning to become more effi-cient and adapt to changing environments by continuously producing suitable services.Simulation results demonstrate the effectiveness of the proposed approach compared to the traditional CBR approach.
