Designing and evaluating assessment and learning in adaptive learning systems

The recent emergence of adaptive language learning systems calls for conceptual work to guide the design of assessment and learning in an adaptive environment.Although adaptive learning might have been touted as a universal cure for learning problems,many adaptive language learning systems fall short of educators’expectations,partly due to a lack of standards and best practices in this area.To fill this gap,this paper proposes some major considerations in designing a high-quality assessment and learning experience in adaptive learning and ways to evaluate an adaptive learning system.The architecture of adaptive learning is decomposed,with a chain of inferences supporting the overall efficacy of an adaptive learning system presented,including user property representation,user property estimation,content representation,user interaction representation,and user interaction impact.A detailed analysis of key validity issues is provided for each inference,which motivates the major considerations in designing and evaluating assessment and learning.The paper first provides an overview of different types of assessment used in adaptive learning and an analysis of the assessment approach,priorities,and design considerations of each to optimize its use in adaptive learning.Then it proposes a framework for evaluating different aspects of an adaptive learning system.Some special connections are made to models,techniques,designs,and technologies specific to language learning and assessment,bringing more relevance to adaptive language learning solutions.Through establishing some guidelines on key aspects to evaluate and how to evaluate them,the work intends to bring more rigor to the field of adaptive language learning systems.