Meta-Analysis (and Systematic Review) Resources

Prof. Johnson’s Online Guide to State-of-the-Science (and -Art) Systematic Reviews

  •  Overviews of the Systematic Review Process
  1. Johnson, B. T. (2021). Toward a more transparent, rigorous, and generative psychology. Psychological Bulletin, 147(1), 1-15. DOI: 10.1037/bul0000317
  2. Johnson, B. T., & Hennessy, E. A. (2019). Systematic reviews and meta-analyses in the health sciences: Best practice methods for research syntheses. Social Science & Medicine, 233, 237-251. DOI: 10.1016/j.socscimed.2019.05.035
  3. Johnson, B. T., & Eagly, A. H. (2014). Meta-analysis of social-personality psychological research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd Ed., pp. 675-707). London:  Cambridge University Press. [Download Link]
  4. Overview of meta-analytic reviews (updated version of a talk Blair T. Johnson and Emily A. Hennessy originally presented at the CDC in April 2021). [Link 71:11]
  • The Meta-Review Phase of Research: Do You Have a Good Topic?
  1. Helpful flow chart (here) for deciding whether a review concept should be pursued.
  2. Hennessy, E. A., Johnson, B. T., & Keenan, C. (2019). Best practice guidelines and essential methodological steps to conduct rigorous and systematic meta‐reviews. Applied Psychology: Health and Well‐Being, 11(3), 353-381. [article]
  3. Hennessy, E. A., & Johnson, B. T. (2020). Examining overlap of included studies in meta‐reviews: Guidance for using the corrected covered area index. Research Synthesis Methods, 11(1), 134-145. [article]
  4. Pescatello, L. S., Hennessy, E. A., Page, W., Craft, L., Katzmarzyk, P., Fish, A., & Johnson, B. T. (2021). Best practices for meta-reviews in physical activity and health research. Journal of Physical Activity & Health. DOI: 10.1123/jpah.2021-0243
  5. Overview of meta-reviews (updated version of a talk Emily A. Hennessy and Blair T. Johnson presented at the CDC in May 2021). [Link 31:42]
  6. Why focus purely and only on randomized controlled trials (RCTs?) Why uncontrolled trials might prove truly important. [Link 20:27]
  • Literature Search
  1. How do you define a topic? Use TOPICS+M (vs. PICO, PICOT, PICOTS, etc.), brief video [Link; 1:42]
  2. ISSG Search Filters Resource
  3. litsearchr is an R package to facilitate quasi-automatic searches for systematic reviews. There’s a video tutorial here on YouTube (of the first version). The article introducing listsearchr is here. This software can automatically write Boolean search strings in over 50 languages, with stemming support for English. It’s a partial solution to the problem of mono-language bias in literature searches.
  4. How many studies will I find? [Link 20:32]
  • Coding Studies
  1. Coding traditional study dimensions. [Email Prof. Johnson if you want access to this video.]
  2. Incorporating spatiotemporal dimensions (e.g., community- or nation-level dimensions).
    • Estimating Effect Sizes
    1. Guide to Effect Sizes and Confidence Intervals (Jané et al., 2024)
    2. Campbell Collaboration (David B. Wilson) Effect Size Calculator
    3. Calculating effect sizes, David B. Wilson
    4. Complexities in Effect Size Calculation: Using p-values and deducing pooled standard deviations
    1. HubMeta’s related links
    2. Harvard’s Systematic Reviews and Meta Analysis page

      Notes. Prof. Johnson’s YouTube channel is here. These resources compiled as of Monday, 04 February 2024. Please email suggestions to Prof. Johnson.

      Synonyms and related terms: Meta-Analysis. Research Synthesis. Quantitative Review. Meta-Synthesis. Meta-Review. “Umbrella Review”. “Overview”. Meta-Regression. Big Data. Generalizable Knowledge.