Logo: Deutsches Institut für Internationale Pädagogische Forschung

Publications

Publikationendatenbank

show results

Autor:
Zehner, Fabian; Kroehne, Ulf; Hahnel, Carolin; Goldhammer, Frank:

Titel:
PISA reading
Mode effects unveiled in short text responses

Quelle:
In: Psychological Test and Assessment Modeling, 62 (2020) 1 , 85-105

URL des Volltextes:
https://www.psychologie-aktuell.com/fileadmin/Redaktion/Journale/ptam-2020-1/05_Zehner.pdf

Sprache:
Englisch

Dokumenttyp:
3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft

Schlagwörter:
PISA (Programme for International Student Assessment), Deutschland, Schülerleistung, Leistungstest, Computerunterstütztes Verfahren, Papier, Bleistift, Antwort, Text, Inhalt, Information, Quantität, Methodenwechsel, Effekt, Wirkungsforschung, Datenanalyse, Sekundäranalyse


Abstract(englisch):
Educational largescale assessments risk their temporal comparability when shifting from paperto computerbased assessment. A recent study showed how text responses have altered alongside PISA's mode change, indicating mode effects. Uncertainty remained, however, because it compared students from 2012 and 2015. We aimed at reproducing the findings in an experimental setting, in which n = 836 students answered PISA reading questions on computer, paper, or both. Text response features for information quantity and relevance were extracted automatically. Results show a comprehensive recovery of findings. Students incorporated more information into their text responses on computer than on paper, with some items being more affected than others. Regarding information relevance, we found less mode effect variance across items than the original study. Hints for a relationship between mode effect and gender across items could be reproduced. The study demonstrates the stability of linguistic feature extraction from text responses. (DIPF/Orig.)


DIPF-Abteilung:
Bildungsqualität und Evaluation

Notizen:

last modified Nov 11, 2016