Scale development with small samples: a new application of longitudinal item response theory

Carrie R. Houts, Robert Morlock, Steven I. Blum, Michael Edwards, R. J. Wirth

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


Purpose: Measurement development in hard-to-reach populations can pose methodological challenges. Item response theory (IRT) is a useful statistical tool, but often requires large samples. We describe the use of longitudinal IRT models as a pragmatic approach to instrument development when large samples are not feasible. Methods: The statistical foundations and practical benefits of longitudinal IRT models are briefly described. Results from a simulation study are reported to demonstrate the model’s ability to recover the generating measurement structure and parameters using a range of sample sizes, number of items, and number of time points. An example using early-phase clinical trial data in a rare condition demonstrates these methods in practice. Results: Simulation study results demonstrate that the longitudinal IRT model’s ability to recover the generating parameters rests largely on the interaction between sample size and the number of time points. Overall, the model performs well even in small samples provided a sufficient number of time points are available. The clinical trial data example demonstrates that by using conditional, longitudinal IRT models researchers can obtain stable estimates of psychometric characteristics from samples typically considered too small for rigorous psychometric modeling. Conclusion: Capitalizing on repeated measurements, it is possible to estimate psychometric characteristics for an assessment even when sample size is small. This allows researchers to optimize study designs and have increased confidence in subsequent comparisons using scores obtained from such models. While there are limitations and caveats to consider when using these models, longitudinal IRT modeling may be especially beneficial when developing measures for rare conditions and diseases in difficult-to-reach populations.

Original languageEnglish (US)
Pages (from-to)1721-1734
Number of pages14
JournalQuality of Life Research
Issue number7
StatePublished - Jul 1 2018


  • Item response theory
  • Longitudinal data
  • Rare diseases
  • Scale development
  • Small sample statistics

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health


Dive into the research topics of 'Scale development with small samples: a new application of longitudinal item response theory'. Together they form a unique fingerprint.

Cite this