DIPF database of publications

Detailansicht Treffer

DIPF database of publications

Show results

Zehner, Fabian; Goldhammer, Frank; Sälzer, Christine:

Using and improving coding guides for and by automatic coding of PISA short text responses

In: IEEE ICDM Workshop (Hrsg.): Proceedings of the IEEE ICDM Workshop on Data Mining for Educational Assessment and Feedback (ASSESS 2015) Atlantic City, NJ : Institute of Electrical and Electronics Engineers (2015) , 1-8

URL of full text:


Document type
4. Beiträge in Sammelwerken; Tagungsband/Konferenzbeitrag/Proceedings

Antwort, Codierung, PISA (Programme for International Student Assessment), Regel (Richtlinie), Text

We propose and empirically evaluate a theoretical framework of how to use coding guides for automatic coding (scoring) and how, in turn, automatic coding can enhance the use of coding guides. We adopted a recently described baseline approach to automatically classify responses. Well-established coding guides from PISA, comprising reference responses, and its German sample from 2012 were used for evaluation. Ten items with 41,990 responses at total were analyzed. Results showed that (1) responses close to the cluster centroid constitute prototypes, (2) automatic coding can improve coding guides, while (3) the proposed procedure leads to unreliable accuracy for small numbers of clusters but promising agreement to human coding for higher numbers. Further analyses are still to be done to find the optimal balance of the implied coding effort and model accuracy. (DIPF/Orig.)

Educational Quality and Evaluation