Quarterly (March, June, September, December)
160 pp. per issue
6 3/4 x 10
ISSN
0891-2017
E-ISSN
1530-9312
2014 Impact factor:
1.23

Computational Linguistics

Paola Merlo, Editor
March 2004, Vol. 30, No. 1, Pages 95-101
(doi: 10.1162/089120104773633402)
© 2004 Association for Computational Linguistics
The Kappa Statistic: A Second Look
Article PDF (77.11 KB)
Abstract

In recent years, the kappa coefficient of agreement has become the de facto standard for evaluating intercoder agreement for tagging tasks. In this squib, we highlight issues that affect κ and that the community has largely neglected. First, we discuss the assumptions underlying different computations of the expected agreement component of κ. Second, we discuss how prevalence and bias affect the κ measure.