1 Section of Biostatistics, Department of Public Health, Faculty of Health and Medical Sciences, Københavns Universitet2 Department of Public Health, Faculty of Health and Medical Sciences, Københavns Universitet3 unknown4 Section of Biostatistics, Department of Public Health, Faculty of Health and Medical Sciences, Københavns Universitet
We present a new model-based approach to the analysis of agreement between raters in a situation where all raters have supplied dichotomous ratings of the same cases in a sample. The model is a logistic regression model with random effects - a Rasch model. In the rater setting, the Rasch model includes parameters that allow raters to have different propensities to score a given set of individuals positively or negatively - the rater bias. An exact score test of the hypothesis of no rater bias is proposed and is shown to be an exact generalised McNemar's test. Based on the model, we suggest quantifying the rater variation as a suitable measure of the variation of the rater odds ratios. An important example that will serve to motivate and illustrate the proposed model, is the study of Umbilical artery Doppler velocimetry used by obstetricians to assess the status of a foetus. The purpose of the assessment is to improve the foetus' chance of survival by choosing the optimal time of elective delivery. In the study, data related to 139 perinatal deaths were sent to 32 experts who were asked whether the use of Doppler velocimetry might have prevented each death.