How Big Is "Big"? Interpreting Effect Sizes in L2 Research

Luke D Plonsky, Frederick L. Oswald

Research output: Contribution to journalArticle

227 Citations (Scopus)

Abstract

The calculation and use of effect sizes-such as d for mean differences and r for correlations-has increased dramatically in second language (L2) research in the last decade. Interpretations of these effects, however, have been rare and, when present, have largely defaulted to Cohen's levels of small (d = .2, r = .1), medium (.5, .3), and large (.8, .5), which were never intended as prescriptions but rather as a general guide. As Cohen himself and many others have argued, effect sizes are best understood when interpreted within a particular discipline or domain. This article seeks to promote more informed and field-specific interpretations of d and r by presenting a description of L2 effects from 346 primary studies and 91 meta-analyses (N > 604,000). Results reveal that Cohen's benchmarks generally underestimate the effects obtained in L2 research. Based on our analysis, we propose a field-specific scale for interpreting effect sizes, and we outline eight key considerations for gauging relative magnitude and practical significance in primary and secondary studies, such as theoretical maturity in the domain, the degree of experimental manipulation, and the presence of publication bias.

Original languageEnglish (US)
Pages (from-to)878-912
Number of pages35
JournalLanguage Learning
Volume64
Issue number4
DOIs
StatePublished - Dec 1 2014

Fingerprint

interpretation
maturity
manipulation
Effect Size
medication
present
trend
language
Maturity
Language
Benchmark
Prescription
Manipulation

Keywords

  • Effect sizes
  • Meta-analysis
  • Practical significance
  • Quantitative research methods

ASJC Scopus subject areas

  • Linguistics and Language
  • Education
  • Language and Linguistics

Cite this

How Big Is "Big"? Interpreting Effect Sizes in L2 Research. / Plonsky, Luke D; Oswald, Frederick L.

In: Language Learning, Vol. 64, No. 4, 01.12.2014, p. 878-912.

Research output: Contribution to journalArticle

Plonsky, Luke D ; Oswald, Frederick L. / How Big Is "Big"? Interpreting Effect Sizes in L2 Research. In: Language Learning. 2014 ; Vol. 64, No. 4. pp. 878-912.
@article{82a39f37423f4d32bcbc42f1c70c0cbd,
title = "How Big Is {"}Big{"}? Interpreting Effect Sizes in L2 Research",
abstract = "The calculation and use of effect sizes-such as d for mean differences and r for correlations-has increased dramatically in second language (L2) research in the last decade. Interpretations of these effects, however, have been rare and, when present, have largely defaulted to Cohen's levels of small (d = .2, r = .1), medium (.5, .3), and large (.8, .5), which were never intended as prescriptions but rather as a general guide. As Cohen himself and many others have argued, effect sizes are best understood when interpreted within a particular discipline or domain. This article seeks to promote more informed and field-specific interpretations of d and r by presenting a description of L2 effects from 346 primary studies and 91 meta-analyses (N > 604,000). Results reveal that Cohen's benchmarks generally underestimate the effects obtained in L2 research. Based on our analysis, we propose a field-specific scale for interpreting effect sizes, and we outline eight key considerations for gauging relative magnitude and practical significance in primary and secondary studies, such as theoretical maturity in the domain, the degree of experimental manipulation, and the presence of publication bias.",
keywords = "Effect sizes, Meta-analysis, Practical significance, Quantitative research methods",
author = "Plonsky, {Luke D} and Oswald, {Frederick L.}",
year = "2014",
month = "12",
day = "1",
doi = "10.1111/lang.12079",
language = "English (US)",
volume = "64",
pages = "878--912",
journal = "Language Learning",
issn = "0023-8333",
publisher = "Wiley-Blackwell",
number = "4",

}

TY - JOUR

T1 - How Big Is "Big"? Interpreting Effect Sizes in L2 Research

AU - Plonsky, Luke D

AU - Oswald, Frederick L.

PY - 2014/12/1

Y1 - 2014/12/1

N2 - The calculation and use of effect sizes-such as d for mean differences and r for correlations-has increased dramatically in second language (L2) research in the last decade. Interpretations of these effects, however, have been rare and, when present, have largely defaulted to Cohen's levels of small (d = .2, r = .1), medium (.5, .3), and large (.8, .5), which were never intended as prescriptions but rather as a general guide. As Cohen himself and many others have argued, effect sizes are best understood when interpreted within a particular discipline or domain. This article seeks to promote more informed and field-specific interpretations of d and r by presenting a description of L2 effects from 346 primary studies and 91 meta-analyses (N > 604,000). Results reveal that Cohen's benchmarks generally underestimate the effects obtained in L2 research. Based on our analysis, we propose a field-specific scale for interpreting effect sizes, and we outline eight key considerations for gauging relative magnitude and practical significance in primary and secondary studies, such as theoretical maturity in the domain, the degree of experimental manipulation, and the presence of publication bias.

AB - The calculation and use of effect sizes-such as d for mean differences and r for correlations-has increased dramatically in second language (L2) research in the last decade. Interpretations of these effects, however, have been rare and, when present, have largely defaulted to Cohen's levels of small (d = .2, r = .1), medium (.5, .3), and large (.8, .5), which were never intended as prescriptions but rather as a general guide. As Cohen himself and many others have argued, effect sizes are best understood when interpreted within a particular discipline or domain. This article seeks to promote more informed and field-specific interpretations of d and r by presenting a description of L2 effects from 346 primary studies and 91 meta-analyses (N > 604,000). Results reveal that Cohen's benchmarks generally underestimate the effects obtained in L2 research. Based on our analysis, we propose a field-specific scale for interpreting effect sizes, and we outline eight key considerations for gauging relative magnitude and practical significance in primary and secondary studies, such as theoretical maturity in the domain, the degree of experimental manipulation, and the presence of publication bias.

KW - Effect sizes

KW - Meta-analysis

KW - Practical significance

KW - Quantitative research methods

UR - http://www.scopus.com/inward/record.url?scp=84908627995&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84908627995&partnerID=8YFLogxK

U2 - 10.1111/lang.12079

DO - 10.1111/lang.12079

M3 - Article

AN - SCOPUS:84908627995

VL - 64

SP - 878

EP - 912

JO - Language Learning

JF - Language Learning

SN - 0023-8333

IS - 4

ER -