Logo: Deutsches Institut für Internationale Pädagogische Forschung



show results

Hahnel, Carolin; Kroehne, Ulf; Goldhammer, Frank; Schoor, Cornelia; Mahlow, Nina; Artelt, Cordula:

Validating process variables of sourcing in an assessment of multiple document comprehension

In: British Journal of Educational Psychology, 89 (2019) 3 , 524-537

URL des Volltextes:



3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft

Leistungsmessung, Technologiebasiertes Testen, Dokument, Verstehen, Quelle, Information, Strategie, Logdatei, Indikator, Validität, Student, Universität, Deutschland

Background: With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be validated with regard to their intended meaning. Aims: This study seeks to demonstrate how process data from an assessment of multiple document comprehension can be used to represent sourcing, which summarizes activities for the consideration of the origin and intention of documents. The investigated process variables were created according to theoretical assumptions about sourcing, and systematically tested for differences between persons, units (i.e., documents and items), and properties of the test administration. Sample: The sample included 310 German university students (79.4% female), enrolled in several bachelor's or master's programmes of the social sciences and humanities. Methods: Regarding the hierarchical data structure, the hypotheses were analysed with generalized linear mixed models (GLMM). Results: The results mostly revealed expected differences between individuals and units. However, unexpected effects of the administered order of units and documents were detected. Conclusions: The study demonstrates the theoryinformed construction of process variables from logfiles and an approach for empirical validation of their interpretation. The results suggest that students apply sourcing for different reasons, but also stress the need of further validation studies and refinements in the operationalization of the indicators investigated. (DIPF/Orig.)

Bildungsqualität und Evaluation


last modified Nov 11, 2016