The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers

Yaakov Ophir*, Itay Sisso, Christa S.C. Asterhan, Refael Tikochinski, Roi Reichart

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

87 Scopus citations

Abstract

Data collection from online platforms, such as Amazon’s Mechanical Turk (MTurk), has become popular in clinical research. However, there are also concerns about the representativeness and the quality of these data for clinical studies. The present work explores these issues in the specific case of major depression. Analyses of two large data sets gathered from MTurk (Sample 1: N = 2,692; Sample 2: N = 2,354) revealed two major findings: First, failing to screen for inattentive and fake respondents inflates the rates of major depression artificially and significantly (by 18.5%–27.5%). Second, after cleaning the data sets, depression in MTurk is still 1.6 to 3.6 times higher than general population estimates. Approximately half of this difference can be attributed to differences in the composition of MTurk samples and the general population (i.e., sociodemographics, health, and physical activity lifestyle). Several explanations for the other half are proposed, and practical data-quality tools are provided.

Original languageAmerican English
Pages (from-to)65-83
Number of pages19
JournalClinical Psychological Science
Volume8
Issue number1
DOIs
StatePublished - 1 Jan 2020

Bibliographical note

Publisher Copyright:
© The Author(s) 2019.

Keywords

  • Mechanical Turk
  • crowdsourcing
  • data quality measures
  • depression
  • open data
  • open materials
  • prevalence of depression

Fingerprint

Dive into the research topics of 'The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers'. Together they form a unique fingerprint.

Cite this