Statistical assumptions in L2 research: A systematic review

Yuhang Hu, Luke Plonsky

Research output: Contribution to journalReview article

1 Scopus citations

Abstract

Statistical tests carry with them a number of assumptions that must be checked. Failing to do so and to report the results of such preliminary analyses introduce a potential threat to the internal validity of a study and to our ability as consumers to put faith in study findings. This article systematically examines the reporting of checks on assumptions in two major journals of second-language (L2) research (Language Learning and Second Language Research) over a span of six years (2012–2017 inclusive). Using synthetic techniques, each study in the sample (K = 107) was coded for whether or not the assumptions associated with five statistical tests were reported: independent samples t-test, one-way ANOVA, Pearson correlation, multiple regression, and chi-square. In doing so, two coding standards (‘stringent’ and ‘lenient’) were adopted to arrive at a more comprehensive picture of researchers’ practices and to reflect the different types of checks required by quantitative analyses. The lenient standard was coded as being met if one or more of the assumptions for a given analytical technique was reported; the stringent standard, by contrast, was recorded as met if all assumptions were reported and checked. Overall, 17% and 24% of the sample were found to have met the stringent and lenient standards, respectively. The study also addresses differences in the checking and reporting of assumptions across the five statistical procedures we examined. Despite overall weak reporting, we note several improvements taking place over time. Suggestions for improving data analytic and reporting practices in the field are discussed as well as implications for researcher training (see Loewen et al., 2014; Plonsky, 2014).

Original languageEnglish (US)
JournalSecond Language Research
DOIs
StateAccepted/In press - Jan 1 2019

Keywords

  • methodological synthesis
  • quantitative research methods
  • second-language research
  • statistical assumptions
  • study quality

ASJC Scopus subject areas

  • Education
  • Linguistics and Language

Fingerprint Dive into the research topics of 'Statistical assumptions in L2 research: A systematic review'. Together they form a unique fingerprint.

  • Cite this