-
-
Author(s): Kröhne, Ulf; Gnambs, Timo; Goldhammer, Frank
Title: Disentangling setting and mode effects for online competence assessment
In: Blossfeld, Hans-Peter; Roßbach, Hans-Günther (Hrsg.): Education as a lifelong process: The German National Education Panel Study (NEPS), Wiesbaden: Springer VS, 2019 (Edition ZfE, 3), S. 171-193
DOI: 10.1007/978-3-658-23162-0_10
URL: https://link.springer.com/chapter/10.1007%2F978-3-658-23162-0_10
Publication Type: 4. Beiträge in Sammelbänden; Sammelband (keine besondere Kategorie)
Language: Englisch
Abstract: Many large-scale competence assessments such as the National Educational Panel Study (NEPS) have introduced novel test designs to improve response rates and measurement precision. In particular, unstandardized online assessments (UOA) offer an economic approach to reach heterogeneous populations that otherwise would not participate in face-to-face assessments. Acknowledging the difference between delivery, mode, and test setting, this chapter extends the theoretical background for dealing with mode effects in NEPS competence assessments (Kroehne and Martens in Zeitschrift für Erziehungswissenschaft 14:169-186, 2011 2011) and discusses two specific facets of UOA: (a) the confounding of selection and setting effects and (b) the role of test-taking behavior as mediator variable. We present a strategy that allows the integration of results from UOA into the results from proctored computerized assessments and generalizes the idea of motivational filtering, known for the treatment of rapid guessing behavior in low-stakes assessment. We particularly emphasize the relationship between paradata and the investigation of test-taking behavior, and illustrate how a reference sample formed by competence assessments under standardized and supervised conditions can be used to increase the comparability of UOA in mixed-mode designs. The closing discussion reflects on the trade-off between data quality and the benefits of UOA. (DIPF/Orig.)
DIPF-Departments: Bildungsqualität und Evaluation