The Evaluation of Adaptive Technology-Enhanced Learning Systems

Abstract

Adaptive technology enhanced learning has attracted significant interest with the promise of supporting individual learning tailored to the unique circumstances, preferences, and prior knowledge of a learner. Evaluation of the overall performance of such adaptive TEL systems is a major challenge; as such systems react differently for each individual user and context of use. Evaluation of such systems has become a significant but very complex area of research in itself since depending on the aspect of adaptivity and personalisation that needs to be evaluated (quality of the user modelling, performance of different adaptation approaches, knowledge gain from using the personalised system or overall end user experience), several evaluation techniques need to be combined and executed differently. This paper proposes a hybrid recommendation service for recommending appropriate evaluation techniques (approach, methods, metrics and criteria). It also discusses evaluation challenges and presents analysed results of a survey on evaluations of adaptive systems.