Evaluation Framework for Statistical User Models.

This paper analyzes the main barriers that user model developers have to face when evaluating a statistical user model. Main techniques used to evaluate statistical user models, mostly borrowed from the areas of Machine Learning and Information Retrieval, are examined. Then an evaluation methodology for statistical user models is proposed together with a set of metrics to specifically evaluate statistical user models. Finally, a benchmark for statistical user models is proposed, thus making possible to compare and replicate the evaluations. Thus, main contribution of this paper is to enable that several user model evaluations were comparable.