Analyzing Complex Treatment Effects in Nonrandomized Observational Studies: The Case of Retention of Students in Grade

Should low-achieving students be promoted to the next grade or be retained (held back) in the prior grade? This special section presents a discussion of the application of marginal structural models to the challenging problem of estimating the effect of promotion versus retention in grade on math scores in elementary school. Vandecandelaere, De Fraine, Van Damme, and Vansteelandt provide a didactic presentation of the marginal structural modeling approach, noting retention is a time-varying treatment because promoted low-achieving students may be retained in a subsequent grade. Steiner, Park, and Kim's commentary presents a detailed analysis of the treatment effects being estimated in same-age versus same-grade comparisons from the perspective of the potential outcomes model. Reshetnyak, Cham, and Kim's commentary clarifies the conditions under which same-age versus same-grade comparisons might be preferred; they also identify methods of further improving the estimation of retention effects. In their rejoinder, Vandecandelaere and Vansteelandt discuss tradeoffs in comparing the promoted and retained groups and highlight sensitivity analysis as a method of probing the robustness of treatment effect estimates. Our hope is that this combined didactic presentation and critical evaluation will encourage researchers to add marginal structural models to their methodological toolkits.