-
-
Autor*innen: Zehner, Fabian; Goldhammer, Frank; Sälzer, Christine
Titel: Using and improving coding guides for and by automatic coding of PISA short text responses
Aus: IEEE ICDM Workshop (Hrsg.): Proceedings of the IEEE ICDM Workshop on Data Mining for Educational Assessment and Feedback (ASSESS 2015), Atlantic City; NJ: Institute of Electrical and Electronics Engineers, 2015 , S. 1-8
URL: http://www.aspiringminds.com/assess/2015/papers/PID3896317.pdf
Dokumenttyp: 4. Beiträge in Sammelwerken; Tagungsband/Konferenzbeitrag/Proceedings
Sprache: Englisch
Schlagwörter: Antwort; Codierung; PISA <Programme for International Student Assessment>; Regel <Richtlinie>; Text
Abstract (english): We propose and empirically evaluate a theoretical framework of how to use coding guides for automatic coding (scoring) and how, in turn, automatic coding can enhance the use of coding guides. We adopted a recently described baseline approach to automatically classify responses. Well-established coding guides from PISA, comprising reference responses, and its German sample from 2012 were used for evaluation. Ten items with 41,990 responses at total were analyzed. Results showed that (1) responses close to the cluster centroid constitute prototypes, (2) automatic coding can improve coding guides, while (3) the proposed procedure leads to unreliable accuracy for small numbers of clusters but promising agreement to human coding for higher numbers. Further analyses are still to be done to find the optimal balance of the implied coding effort and model accuracy. (DIPF/Orig.)
DIPF-Abteilung: Bildungsqualität und Evaluation