DIPF database of publications

Detailansicht Treffer

DIPF database of publications

Show results

Author
Zehner, Fabian; Kroehne, Ulf; Hahnel, Carolin; Goldhammer, Frank:

Title:
PISA reading
Mode effects unveiled in short text responses

Source:
In: Psychological Test and Assessment Modeling, 62 (2020) 1 , 85-105

URL of full text:
https://www.psychologie-aktuell.com/fileadmin/Redaktion/Journale/ptam-2020-1/05_Zehner.pdf

URN:
urn:nbn:de:0111-pedocs-203542

Language:
Englisch

Document type
3a. Beiträge in begutachteten Zeitschriften; Beitrag in Sonderheft

Schlagwörter:
PISA (Programme for International Student Assessment), Deutschland, Schülerleistung, Leistungstest, Computerunterstütztes Verfahren, Papier, Bleistift, Antwort, Text, Inhalt, Information, Quantität, Methodenwechsel, Effekt, Wirkungsforschung, Datenanalyse, Sekundäranalyse


Abstract(englisch):
Educational largescale assessments risk their temporal comparability when shifting from paperto computerbased assessment. A recent study showed how text responses have altered alongside PISA's mode change, indicating mode effects. Uncertainty remained, however, because it compared students from 2012 and 2015. We aimed at reproducing the findings in an experimental setting, in which n = 836 students answered PISA reading questions on computer, paper, or both. Text response features for information quantity and relevance were extracted automatically. Results show a comprehensive recovery of findings. Students incorporated more information into their text responses on computer than on paper, with some items being more affected than others. Regarding information relevance, we found less mode effect variance across items than the original study. Hints for a relationship between mode effect and gender across items could be reproduced. The study demonstrates the stability of linguistic feature extraction from text responses. (DIPF/Orig.)


DIPF-Departments:
Educational Quality and Evaluation

Notes: