The kappa statistic measures the extent to which different raters or examiners differ when looking at the same data and assigning categories. The statistic is a ratio:
The numerator is P(A) – P(E), where P(A) is the proportion of times the k raters agree, and P(E) is the proportion of times the k raters are expected to agree by chance alone. The denominator is 1 – P(E).