Meta-analyses in the social sciences and education often provide policy makers, practitioners, and researchers with critical new information that summarizes the central quantitative findings from a particular research literature. However, meta-analytic articles and reports are also rather technical pieces of work that can leave many readers without a clear sense of the results and their implications. In this respect, both visual and narrative interpretations of research syntheses are important considerations for presenting and describing complex results in more informative and accessible ways. Typically, readers of meta-analyses in the social sciences and education are offered two types of summary tables to help them make sense of the results. First, a table presents the overall summary effect size for the entire meta-analysis. This table may also include mean effect sizes for subgroups of studies defined by important substantive or methodological characteristics of the included studies (for instance, whether the study employed an experimental or nonexperimental design). Reports and articles often present a second table that includes a lengthy catalog of all studies included in the meta-analysis, along with such information as their publication dates, the authors' names, sample sizes, effect sizes, and, perhaps, differences among the studies along key substantive or methodological dimensions such as the ages of participants or the nature of the control group employed. Usually the material in these sorts of tables is not arrayed in a particularly informative order but instead simply sorted chronologically by publication year or alphabetically by the authors' last names. As we show in this chapter, however, the situation is improving. Such tables can be useful to describe the individual data elements - the outcomes gleaned from each of the studies included in the meta-analysis - and can help present information about the overall effects observed and how those effects might vary by interesting methodological and substantive dimensions. However, we assert that data from meta-analyses can be presented in ways that facilitate understanding of the outcomes and that communicate more information than have been conventionally employed in the literature. Rather than limiting presentation and interpretation of meta-analytic data to simple tables and descriptive text, we argue for greater consideration of figures and displays. This chapter also examines how the careful integration of quantitative and narrative approaches to research synthesis can enhance the interpretation of meta-analyses. We outline three key circumstances under which one should consider applying narrative techniques to quantitative reviews. First, some of the earliest critics of metaanalysis suggested that the methodology applied a mechanistic approach to research synthesis that sacrificed most of the information contributed by the included studies and glossed over salient methodological and substantive differences (Eysenck 1978; Slavin 1984). We believe that many contemporary applications of meta-analysis have overcome these past criticisms through more rigorous analyses of effect-size variability and modeling of the substantive and methodological moderators that might explain variability in the observed outcomes. However, we argue that integration of narrative forms of synthesis can help further clarify and describe a variegated research literature. Second, there may be evidence that some literature, though of general interest and importance, is not amenable to meta-analysis. The typical meta-analysis would simply exclude this literature and focus only on the results that can be summarized using quantitative techniques. We believe, though, that such a situation may call for blending meta-analytic and narrative synthesis so that a more comprehensive body of research can be considered. Finally, beyond descriptive data and significance tests of the effect sizes, we contend that researchers should also use narrative techniques for interpreting effect sizes, primarily in terms of understanding the practical or policy importance of their magnitudes. In these ways and others, research syntheses that offer the best of both meta-analysis and narrative review offer great promise.
|Original language||English (US)|
|Title of host publication||The Hand. of Res. Synthesis and Meta-Analysis, 2nd Ed.|
|Publisher||Russell Sage Foundation|
|Number of pages||23|
|State||Published - 2009|
ASJC Scopus subject areas
- Social Sciences(all)