Bootstrapping in Applied Linguistics

Assessing its Potential Using Shared Data

Luke D Plonsky, Jesse Egbert, Geoffrey T. Laflair

Research output: Contribution to journalArticle

20 Citations (Scopus)

Abstract

Parametric analyses such as t tests and ANOVAs are the norm - if not the default - statistical tests found in quantitative applied linguistics research (Gass 2009). Applied statisticians and one applied linguist (Larson-Hall 2010, 2012; Larson-Hall and Herrington 2010), however, have argued that this approach may not be appropriate for small samples and/or nonnormally distributed data (e.g. Wilcox 2003), both common in second language (L2) research. They recommend instead 'robust statistics' such as bootstrapping, a nonparametric procedure that randomly resamples from an observed data set to produce a simulated but more stable and statistically accurate outcome. The present study tests the usefulness of bootstrapping by reanalyzing raw data from 26 studies of applied linguistics research. Our results found no evidence of Type II error (false negative). However, 4 out of 16 statistically significant results were not replicated (i.e. a Type I error 'misfit' five times higher than an alpha of. 05). We discuss empirically justified suggestions for the use of bootstrapping in the context of broader methodological issues and reforms in applied linguistics (see Plonsky 2013, 2014).

Original languageEnglish (US)
Pages (from-to)591-610
Number of pages20
JournalApplied Linguistics
Volume36
Issue number5
DOIs
StatePublished - Dec 1 2015

Fingerprint

Linguistics
linguistics
statistician
Statistical tests
statistical test
Analysis of variance (ANOVA)
statistics
Statistics
reform
language
evidence
Applied Linguistics
Bootstrapping
Usefulness
Small Sample
Analysis of Variance
Language
Statistical Tests

ASJC Scopus subject areas

  • Linguistics and Language
  • Communication
  • Language and Linguistics

Cite this

Bootstrapping in Applied Linguistics : Assessing its Potential Using Shared Data. / Plonsky, Luke D; Egbert, Jesse; Laflair, Geoffrey T.

In: Applied Linguistics, Vol. 36, No. 5, 01.12.2015, p. 591-610.

Research output: Contribution to journalArticle

Plonsky, Luke D ; Egbert, Jesse ; Laflair, Geoffrey T. / Bootstrapping in Applied Linguistics : Assessing its Potential Using Shared Data. In: Applied Linguistics. 2015 ; Vol. 36, No. 5. pp. 591-610.
@article{313f7f8812f046f58523f18741796ff8,
title = "Bootstrapping in Applied Linguistics: Assessing its Potential Using Shared Data",
abstract = "Parametric analyses such as t tests and ANOVAs are the norm - if not the default - statistical tests found in quantitative applied linguistics research (Gass 2009). Applied statisticians and one applied linguist (Larson-Hall 2010, 2012; Larson-Hall and Herrington 2010), however, have argued that this approach may not be appropriate for small samples and/or nonnormally distributed data (e.g. Wilcox 2003), both common in second language (L2) research. They recommend instead 'robust statistics' such as bootstrapping, a nonparametric procedure that randomly resamples from an observed data set to produce a simulated but more stable and statistically accurate outcome. The present study tests the usefulness of bootstrapping by reanalyzing raw data from 26 studies of applied linguistics research. Our results found no evidence of Type II error (false negative). However, 4 out of 16 statistically significant results were not replicated (i.e. a Type I error 'misfit' five times higher than an alpha of. 05). We discuss empirically justified suggestions for the use of bootstrapping in the context of broader methodological issues and reforms in applied linguistics (see Plonsky 2013, 2014).",
author = "Plonsky, {Luke D} and Jesse Egbert and Laflair, {Geoffrey T.}",
year = "2015",
month = "12",
day = "1",
doi = "10.1093/applin/amu001",
language = "English (US)",
volume = "36",
pages = "591--610",
journal = "Applied Linguistics",
issn = "0142-6001",
publisher = "Oxford University Press",
number = "5",

}

TY - JOUR

T1 - Bootstrapping in Applied Linguistics

T2 - Assessing its Potential Using Shared Data

AU - Plonsky, Luke D

AU - Egbert, Jesse

AU - Laflair, Geoffrey T.

PY - 2015/12/1

Y1 - 2015/12/1

N2 - Parametric analyses such as t tests and ANOVAs are the norm - if not the default - statistical tests found in quantitative applied linguistics research (Gass 2009). Applied statisticians and one applied linguist (Larson-Hall 2010, 2012; Larson-Hall and Herrington 2010), however, have argued that this approach may not be appropriate for small samples and/or nonnormally distributed data (e.g. Wilcox 2003), both common in second language (L2) research. They recommend instead 'robust statistics' such as bootstrapping, a nonparametric procedure that randomly resamples from an observed data set to produce a simulated but more stable and statistically accurate outcome. The present study tests the usefulness of bootstrapping by reanalyzing raw data from 26 studies of applied linguistics research. Our results found no evidence of Type II error (false negative). However, 4 out of 16 statistically significant results were not replicated (i.e. a Type I error 'misfit' five times higher than an alpha of. 05). We discuss empirically justified suggestions for the use of bootstrapping in the context of broader methodological issues and reforms in applied linguistics (see Plonsky 2013, 2014).

AB - Parametric analyses such as t tests and ANOVAs are the norm - if not the default - statistical tests found in quantitative applied linguistics research (Gass 2009). Applied statisticians and one applied linguist (Larson-Hall 2010, 2012; Larson-Hall and Herrington 2010), however, have argued that this approach may not be appropriate for small samples and/or nonnormally distributed data (e.g. Wilcox 2003), both common in second language (L2) research. They recommend instead 'robust statistics' such as bootstrapping, a nonparametric procedure that randomly resamples from an observed data set to produce a simulated but more stable and statistically accurate outcome. The present study tests the usefulness of bootstrapping by reanalyzing raw data from 26 studies of applied linguistics research. Our results found no evidence of Type II error (false negative). However, 4 out of 16 statistically significant results were not replicated (i.e. a Type I error 'misfit' five times higher than an alpha of. 05). We discuss empirically justified suggestions for the use of bootstrapping in the context of broader methodological issues and reforms in applied linguistics (see Plonsky 2013, 2014).

UR - http://www.scopus.com/inward/record.url?scp=84894096224&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84894096224&partnerID=8YFLogxK

U2 - 10.1093/applin/amu001

DO - 10.1093/applin/amu001

M3 - Article

VL - 36

SP - 591

EP - 610

JO - Applied Linguistics

JF - Applied Linguistics

SN - 0142-6001

IS - 5

ER -