Expert-centered vs user-centered evaluation of Virtual Learning Environments

Abstract

The paper aims to analyse different expert-centered and user-centered methods to evaluate virtual learning environments (VLEs). Advantages and disadvantages of applying different evaluation methods are analysed. Special attention is paid to evaluate VLE personalisation level. VLE personalisation level means the level of VLE optimisation to particular students according to their personal needs, e.g. learning styles. Several original methods are analysed in the paper, e.g. expertcentered methods based on Fuzzy logic and AHP (Analytic Hierarchy Process) as well as usercentered method to evaluate suitability, acceptance and use of VLEs. In the paper, an original method to establish and evaluate suitability of VLEs to particular students’ needs. A number of MCDM (multiple criteria decision making) principles are applied in these methods to create comprehensive quality models (criteria systems) for evaluating quality of VLEs. Several optimisation methods are explored and applied in these methods to optimise VLEs according to particular students’ needs. The research results have shown that all analysed methods are quite objective, exact and simple to use for selecting qualitative personalised VLEs alternatives for particular learners. The differences are based on using different evaluation methods for different purposes and in different pedagogical situations. The proposed user-centered method to evaluate suitability, acceptance and use of VLEs is technological and pedagogical criteria-based, on the one hand, and ETAS-M (Educational Technology Acceptance & Satisfaction Model) - based, on the other. It’s more convenient in comparison with purely technological and pedagogical criteria-based models used by Fuzzy and AHP because it is based only on suitability, acceptance and use evaluation made by the users, and fully reflects their needs and points of view. Additionally, this kind of models does not require specific high-level technological and pedagogical expertise from experts-evaluators like in Fuzzy and AHP methods. On the other hand, this model is better than pure ETAS-M-based model because it’s more flexible since it takes into consideration corresponding average probabilistic indexes of VLE suitability to particular learners’ needs. These methods and their proper application in different pedagogical situations are helpful to enhance learning quality and effectiveness.