Generalizability of Dynamic Fit Index, Equivalence Testing, and Hu & Bentler Cutoffs for Evaluating Fit in Factor Analysis

Factor analysis is often used to model scales created to measure latent constructs, and internal structure validity evidence is commonly assessed with indices like RMSEA, and CFI. These indices are essentially effect size measures and definitive benchmarks regarding which values connote reasonable fit have been elusive. Simulations from the 1990s suggesting possible benchmark values are among the most highly cited methodological papers across any discipline. However, simulations have suggested that fixed benchmarks do not generalize well–fit indices are systematically impacted by characteristics like the number of items and the magnitude of the loadings, so fixed benchmarks can confound misfit with model characteristics. Alternative frameworks for creating customized, model-specific benchmarks have recently been proposed to circumvent these issues but they have not been systematically evaluated. Motivated by two empirical applications where different methods yield inconsistent conclusions, two simulation studies are performed to assess the ability of three different approaches to correctly classify models that are correct or misspecified across different conditions. Results show that dynamic fit indices and equivalence testing both improved upon the traditional Hu & Bentler benchmarks and dynamic fit indices appeared to be least confounded with model characteristics in the conditions studied.