Analyzing Longitudinal Multirater Data with Changing and Stable Raters

  • Stephen West (Contributor)
  • Michael Eid (Contributor)
  • Tobias Koch (Contributor)
  • Jana Holtmann (Contributor)



One issue in analyzing longitudinal multirater data arises if raters drop-in or drop-out throughout a longitudinal study. We term this issue random rater movement (RRM), assuming that the selection of raters into the study approximates a random process and is strongly ignorable. We explain how RRM can be modeled in case of longitudinal multirater designs with (a) interchangeable raters or (b) structurally different raters. To analyze measurement designs with stable and changing interchangeable raters, we recommend using a longitudinal multilevel confirmatory factor model. To analyze measurement designs with stable and changing structurally different raters, we propose a longitudinal multigroup confirmatory factor model. The proposed model is illustrated using real data. Additionally, the performance of the models with regard to a small number of raters and a relatively small overall sample size is examined in Monte Carlo simulation studies. Future directions for analyzing rater movement over time are provided.
Date made availableJan 2 2020
Publisherfigshare Academic Research System

Cite this