Predicting Patterns of Grammatical Complexity Across Language Exam Task Types and Proficiency Levels

Douglas E Biber, Bethany Gray, Shelley Staples

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

In the present article, we explore the extent to which previous research on register variation can be used to predict spoken/written task-type variation as well as differences across score levels in the context of a major standardized language exam (TOEFL iBT). Specifically, we carry out two sets of linguistic analyses based on a large corpus of TOEFL iBT responses: one investigating the use of 23 grammatical complexity features, and the second based on co-occurrence patterns among linguistic features, using Multi-Dimensional (MD) analysis. The first set of analyses confirms the predictions from research on register variation: there are systematic linguistic differences among spoken versus written and independent versus integrated task types. However, hypothesized developmental progressions in the use of these grammatical complexity features were generally not confirmed by score-level differences. In contrast, the MD analysis yielded more robust predictors of both task types and score-level differences, indicating that linguistic descriptions are more reliable and informative when they are based on dimensions of co-occurring lexico-grammatical features. In conclusion, we discuss the application of such dimensions as holistic complexity measures in language development/testing research.

Original languageEnglish (US)
Pages (from-to)639-668
Number of pages30
JournalApplied Linguistics
Volume37
Issue number5
DOIs
StatePublished - Oct 1 2016

Fingerprint

Linguistics
linguistics
dimensional analysis
language
Proficiency
Task Type
Language
Testing
Multidimensional Analysis
TOEFL
Register Variation
Language Development
Co-occurrence
Progression
Prediction
Linguistic Features
Predictors
Linguistic Description

ASJC Scopus subject areas

  • Language and Linguistics
  • Communication
  • Linguistics and Language

Cite this

Predicting Patterns of Grammatical Complexity Across Language Exam Task Types and Proficiency Levels. / Biber, Douglas E; Gray, Bethany; Staples, Shelley.

In: Applied Linguistics, Vol. 37, No. 5, 01.10.2016, p. 639-668.

Research output: Contribution to journalArticle

@article{d927ad423eec457da50821df58611f26,
title = "Predicting Patterns of Grammatical Complexity Across Language Exam Task Types and Proficiency Levels",
abstract = "In the present article, we explore the extent to which previous research on register variation can be used to predict spoken/written task-type variation as well as differences across score levels in the context of a major standardized language exam (TOEFL iBT). Specifically, we carry out two sets of linguistic analyses based on a large corpus of TOEFL iBT responses: one investigating the use of 23 grammatical complexity features, and the second based on co-occurrence patterns among linguistic features, using Multi-Dimensional (MD) analysis. The first set of analyses confirms the predictions from research on register variation: there are systematic linguistic differences among spoken versus written and independent versus integrated task types. However, hypothesized developmental progressions in the use of these grammatical complexity features were generally not confirmed by score-level differences. In contrast, the MD analysis yielded more robust predictors of both task types and score-level differences, indicating that linguistic descriptions are more reliable and informative when they are based on dimensions of co-occurring lexico-grammatical features. In conclusion, we discuss the application of such dimensions as holistic complexity measures in language development/testing research.",
author = "Biber, {Douglas E} and Bethany Gray and Shelley Staples",
year = "2016",
month = "10",
day = "1",
doi = "10.1093/applin/amu059",
language = "English (US)",
volume = "37",
pages = "639--668",
journal = "Applied Linguistics",
issn = "0142-6001",
publisher = "Oxford University Press",
number = "5",

}

TY - JOUR

T1 - Predicting Patterns of Grammatical Complexity Across Language Exam Task Types and Proficiency Levels

AU - Biber, Douglas E

AU - Gray, Bethany

AU - Staples, Shelley

PY - 2016/10/1

Y1 - 2016/10/1

N2 - In the present article, we explore the extent to which previous research on register variation can be used to predict spoken/written task-type variation as well as differences across score levels in the context of a major standardized language exam (TOEFL iBT). Specifically, we carry out two sets of linguistic analyses based on a large corpus of TOEFL iBT responses: one investigating the use of 23 grammatical complexity features, and the second based on co-occurrence patterns among linguistic features, using Multi-Dimensional (MD) analysis. The first set of analyses confirms the predictions from research on register variation: there are systematic linguistic differences among spoken versus written and independent versus integrated task types. However, hypothesized developmental progressions in the use of these grammatical complexity features were generally not confirmed by score-level differences. In contrast, the MD analysis yielded more robust predictors of both task types and score-level differences, indicating that linguistic descriptions are more reliable and informative when they are based on dimensions of co-occurring lexico-grammatical features. In conclusion, we discuss the application of such dimensions as holistic complexity measures in language development/testing research.

AB - In the present article, we explore the extent to which previous research on register variation can be used to predict spoken/written task-type variation as well as differences across score levels in the context of a major standardized language exam (TOEFL iBT). Specifically, we carry out two sets of linguistic analyses based on a large corpus of TOEFL iBT responses: one investigating the use of 23 grammatical complexity features, and the second based on co-occurrence patterns among linguistic features, using Multi-Dimensional (MD) analysis. The first set of analyses confirms the predictions from research on register variation: there are systematic linguistic differences among spoken versus written and independent versus integrated task types. However, hypothesized developmental progressions in the use of these grammatical complexity features were generally not confirmed by score-level differences. In contrast, the MD analysis yielded more robust predictors of both task types and score-level differences, indicating that linguistic descriptions are more reliable and informative when they are based on dimensions of co-occurring lexico-grammatical features. In conclusion, we discuss the application of such dimensions as holistic complexity measures in language development/testing research.

UR - http://www.scopus.com/inward/record.url?scp=84995902320&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84995902320&partnerID=8YFLogxK

U2 - 10.1093/applin/amu059

DO - 10.1093/applin/amu059

M3 - Article

AN - SCOPUS:84995902320

VL - 37

SP - 639

EP - 668

JO - Applied Linguistics

JF - Applied Linguistics

SN - 0142-6001

IS - 5

ER -