-
-
Autor*innen: Meyer, Christian M.; Mieskes, Margot; Stab, Christian; Gurevych, Iryna
Titel: DKPro Agreement. An open-source Java library for measuring inter-rater agreement
Aus: Tsujii, Junichi; Hajic, Jan (Hrsg.): Proceedings of COLING 2014: System demonstrations, Stroudsburg; PA: Association for Computational Linguistics, 2014 , S. 105-109
URL: http://www.aclweb.org/anthology/C/C14/C14-2023.pdf
Dokumenttyp: 4. Beiträge in Sammelbänden; Tagungsband/Konferenzbeitrag/Proceedings
Sprache: Englisch
Schlagwörter: Computerlinguistik; Daten; Messung; Messverfahren; Modell; Reliabilität; Software
Abstract (english): In this paper, we introduce a novel Java implementation of multiple inter-rater agreement measures, which we make available as open-source software. Besides assessing the reliability of coding tasks using S, π, κ, α, etc., we particularly support unitizing tasks by measuring αU as the agreement of the boundaries of the identified annotation units. We provide a unified interface and data model for both tasks as well as multiple diagnostic devices for analyzing the results. (DIPF/Org.)
DIPF-Abteilung: Informationszentrum Bildung