Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature

Karoline Mortensen, Taylor L. Hughes

Research output: Contribution to journalReview article

22 Citations (Scopus)

Abstract

Background: The goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com’s Mechanical Turk (MTurk) as a research tool in a health services research and medical context. Methods: Searches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria. Results: The vast majority of the studies supported the use of MTurk for a variety of academic purposes. Discussion: The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.

Original languageEnglish (US)
Pages (from-to)533-538
Number of pages6
JournalJournal of General Internal Medicine
Volume33
Issue number4
DOIs
StatePublished - Apr 1 2018

Fingerprint

Manuscripts
Health Services Research
Expert Testimony
PubMed
Biomedical Research
Databases
Costs and Cost Analysis
Health
Research
Population
Surveys and Questionnaires

Keywords

  • Alternate data sources
  • Amazon Mechanical Turk
  • Health and medical research
  • MTurk

ASJC Scopus subject areas

  • Internal Medicine

Cite this

Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature. / Mortensen, Karoline; Hughes, Taylor L.

In: Journal of General Internal Medicine, Vol. 33, No. 4, 01.04.2018, p. 533-538.

Research output: Contribution to journalReview article

@article{010872ba5cbf47fbbbf4cae4966b30e7,
title = "Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature",
abstract = "Background: The goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com’s Mechanical Turk (MTurk) as a research tool in a health services research and medical context. Methods: Searches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria. Results: The vast majority of the studies supported the use of MTurk for a variety of academic purposes. Discussion: The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.",
keywords = "Alternate data sources, Amazon Mechanical Turk, Health and medical research, MTurk",
author = "Karoline Mortensen and Hughes, {Taylor L.}",
year = "2018",
month = "4",
day = "1",
doi = "10.1007/s11606-017-4246-0",
language = "English (US)",
volume = "33",
pages = "533--538",
journal = "Journal of General Internal Medicine",
issn = "0884-8734",
publisher = "Springer New York",
number = "4",

}

TY - JOUR

T1 - Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature

AU - Mortensen, Karoline

AU - Hughes, Taylor L.

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Background: The goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com’s Mechanical Turk (MTurk) as a research tool in a health services research and medical context. Methods: Searches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria. Results: The vast majority of the studies supported the use of MTurk for a variety of academic purposes. Discussion: The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.

AB - Background: The goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com’s Mechanical Turk (MTurk) as a research tool in a health services research and medical context. Methods: Searches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria. Results: The vast majority of the studies supported the use of MTurk for a variety of academic purposes. Discussion: The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.

KW - Alternate data sources

KW - Amazon Mechanical Turk

KW - Health and medical research

KW - MTurk

UR - http://www.scopus.com/inward/record.url?scp=85040068819&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85040068819&partnerID=8YFLogxK

U2 - 10.1007/s11606-017-4246-0

DO - 10.1007/s11606-017-4246-0

M3 - Review article

C2 - 29302882

AN - SCOPUS:85040068819

VL - 33

SP - 533

EP - 538

JO - Journal of General Internal Medicine

JF - Journal of General Internal Medicine

SN - 0884-8734

IS - 4

ER -