Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?

David H. Greenberg, Charles Michalopoulos, Philip Robins

Research output: Contribution to journalReview article

19 Citations (Scopus)

Abstract

This paper uses meta-analysis to investigate whether random assignment (or experimental) evaluations of voluntary government-funded training programs for the disadvantaged have produced different conclusions than nonexperimental evaluations. Information includes several hundred estimates from 31 evaluations of 15 programs that operated between 1964 and 1998. The results suggest that experimental and nonexperimental evaluations yield similar conclusions about the effectiveness of training programs, but that estimates of average effects for youth and possibly men might have been larger in experimental studies. The results also suggest that variation among nonexprimental estimates of program effects is similar to variation among experimental estimates for men and youth, but not for women (for whom it seems to be larger), although small sample sizes make the estimated differences somewhat imprecise for all three groups. The policy implications of the findings are discussed.

Original languageEnglish (US)
Pages (from-to)523-552
Number of pages30
JournalJournal of Policy Analysis and Management
Volume25
Issue number3
DOIs
StatePublished - Jun 1 2006

Fingerprint

government program
training program
evaluation
Government
Training program
Evaluation
Group

ASJC Scopus subject areas

  • Business, Management and Accounting(all)
  • Sociology and Political Science
  • Public Administration

Cite this

Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs? / Greenberg, David H.; Michalopoulos, Charles; Robins, Philip.

In: Journal of Policy Analysis and Management, Vol. 25, No. 3, 01.06.2006, p. 523-552.

Research output: Contribution to journalReview article

@article{cab98cc47d7d4046956b0acfdaa6e1e4,
title = "Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?",
abstract = "This paper uses meta-analysis to investigate whether random assignment (or experimental) evaluations of voluntary government-funded training programs for the disadvantaged have produced different conclusions than nonexperimental evaluations. Information includes several hundred estimates from 31 evaluations of 15 programs that operated between 1964 and 1998. The results suggest that experimental and nonexperimental evaluations yield similar conclusions about the effectiveness of training programs, but that estimates of average effects for youth and possibly men might have been larger in experimental studies. The results also suggest that variation among nonexprimental estimates of program effects is similar to variation among experimental estimates for men and youth, but not for women (for whom it seems to be larger), although small sample sizes make the estimated differences somewhat imprecise for all three groups. The policy implications of the findings are discussed.",
author = "Greenberg, {David H.} and Charles Michalopoulos and Philip Robins",
year = "2006",
month = "6",
day = "1",
doi = "10.1002/pam.20190",
language = "English (US)",
volume = "25",
pages = "523--552",
journal = "Journal of Policy Analysis and Management",
issn = "0276-8739",
publisher = "Wiley-Liss Inc.",
number = "3",

}

TY - JOUR

T1 - Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?

AU - Greenberg, David H.

AU - Michalopoulos, Charles

AU - Robins, Philip

PY - 2006/6/1

Y1 - 2006/6/1

N2 - This paper uses meta-analysis to investigate whether random assignment (or experimental) evaluations of voluntary government-funded training programs for the disadvantaged have produced different conclusions than nonexperimental evaluations. Information includes several hundred estimates from 31 evaluations of 15 programs that operated between 1964 and 1998. The results suggest that experimental and nonexperimental evaluations yield similar conclusions about the effectiveness of training programs, but that estimates of average effects for youth and possibly men might have been larger in experimental studies. The results also suggest that variation among nonexprimental estimates of program effects is similar to variation among experimental estimates for men and youth, but not for women (for whom it seems to be larger), although small sample sizes make the estimated differences somewhat imprecise for all three groups. The policy implications of the findings are discussed.

AB - This paper uses meta-analysis to investigate whether random assignment (or experimental) evaluations of voluntary government-funded training programs for the disadvantaged have produced different conclusions than nonexperimental evaluations. Information includes several hundred estimates from 31 evaluations of 15 programs that operated between 1964 and 1998. The results suggest that experimental and nonexperimental evaluations yield similar conclusions about the effectiveness of training programs, but that estimates of average effects for youth and possibly men might have been larger in experimental studies. The results also suggest that variation among nonexprimental estimates of program effects is similar to variation among experimental estimates for men and youth, but not for women (for whom it seems to be larger), although small sample sizes make the estimated differences somewhat imprecise for all three groups. The policy implications of the findings are discussed.

UR - http://www.scopus.com/inward/record.url?scp=33745781500&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745781500&partnerID=8YFLogxK

U2 - 10.1002/pam.20190

DO - 10.1002/pam.20190

M3 - Review article

AN - SCOPUS:33745781500

VL - 25

SP - 523

EP - 552

JO - Journal of Policy Analysis and Management

JF - Journal of Policy Analysis and Management

SN - 0276-8739

IS - 3

ER -