A Review of Meta-Analyses in Education: Methodological Strengths and Weaknesses

Soyeon Ahn, Allison J. Ames, Nicholas Myers

Research output: Contribution to journalArticle

57 Citations (Scopus)

Abstract

The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and weaknesses, and provide recommendations for improvements in order to generate a more valid and credible knowledge base of what works in practice. It was found that 56 meta-analyses followed general recommendations fairly well in problem formulation and data collection, but much improvement is needed in data evaluation and analysis. Particularly, lack of information reported as well as little transparency in the use of statistical methods are concerns for generating credible and generalizable meta-analytic findings that can be transformed to educational practices. Recommendations for yielding more reliable and valid inferences from meta-analyses are provided.

Original languageEnglish
Pages (from-to)436-476
Number of pages41
JournalReview of Educational Research
Volume82
Issue number4
DOIs
StatePublished - Dec 1 2012

Fingerprint

data analysis
education
educational practice
statistical method
credibility
transparency
lack

Keywords

  • credibility
  • generalizability
  • meta-analysis
  • validity

ASJC Scopus subject areas

  • Education

Cite this

A Review of Meta-Analyses in Education : Methodological Strengths and Weaknesses. / Ahn, Soyeon; Ames, Allison J.; Myers, Nicholas.

In: Review of Educational Research, Vol. 82, No. 4, 01.12.2012, p. 436-476.

Research output: Contribution to journalArticle

Ahn, Soyeon ; Ames, Allison J. ; Myers, Nicholas. / A Review of Meta-Analyses in Education : Methodological Strengths and Weaknesses. In: Review of Educational Research. 2012 ; Vol. 82, No. 4. pp. 436-476.
@article{05283e073f1a47119d20bc82d409c10c,
title = "A Review of Meta-Analyses in Education: Methodological Strengths and Weaknesses",
abstract = "The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and weaknesses, and provide recommendations for improvements in order to generate a more valid and credible knowledge base of what works in practice. It was found that 56 meta-analyses followed general recommendations fairly well in problem formulation and data collection, but much improvement is needed in data evaluation and analysis. Particularly, lack of information reported as well as little transparency in the use of statistical methods are concerns for generating credible and generalizable meta-analytic findings that can be transformed to educational practices. Recommendations for yielding more reliable and valid inferences from meta-analyses are provided.",
keywords = "credibility, generalizability, meta-analysis, validity",
author = "Soyeon Ahn and Ames, {Allison J.} and Nicholas Myers",
year = "2012",
month = "12",
day = "1",
doi = "10.3102/0034654312458162",
language = "English",
volume = "82",
pages = "436--476",
journal = "Review of Educational Research",
issn = "0034-6543",
publisher = "SAGE Publications Inc.",
number = "4",

}

TY - JOUR

T1 - A Review of Meta-Analyses in Education

T2 - Methodological Strengths and Weaknesses

AU - Ahn, Soyeon

AU - Ames, Allison J.

AU - Myers, Nicholas

PY - 2012/12/1

Y1 - 2012/12/1

N2 - The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and weaknesses, and provide recommendations for improvements in order to generate a more valid and credible knowledge base of what works in practice. It was found that 56 meta-analyses followed general recommendations fairly well in problem formulation and data collection, but much improvement is needed in data evaluation and analysis. Particularly, lack of information reported as well as little transparency in the use of statistical methods are concerns for generating credible and generalizable meta-analytic findings that can be transformed to educational practices. Recommendations for yielding more reliable and valid inferences from meta-analyses are provided.

AB - The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and weaknesses, and provide recommendations for improvements in order to generate a more valid and credible knowledge base of what works in practice. It was found that 56 meta-analyses followed general recommendations fairly well in problem formulation and data collection, but much improvement is needed in data evaluation and analysis. Particularly, lack of information reported as well as little transparency in the use of statistical methods are concerns for generating credible and generalizable meta-analytic findings that can be transformed to educational practices. Recommendations for yielding more reliable and valid inferences from meta-analyses are provided.

KW - credibility

KW - generalizability

KW - meta-analysis

KW - validity

UR - http://www.scopus.com/inward/record.url?scp=84869815605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84869815605&partnerID=8YFLogxK

U2 - 10.3102/0034654312458162

DO - 10.3102/0034654312458162

M3 - Article

AN - SCOPUS:84869815605

VL - 82

SP - 436

EP - 476

JO - Review of Educational Research

JF - Review of Educational Research

SN - 0034-6543

IS - 4

ER -