Impact of rater characteristics and prosodic features of speaker accentedness on ratings of international teaching assistants' oral performance

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

Few prior studies have examined degree of fidelity between raters' assessments of oral performances and objectively observable prosodic indices of accentedness. Prosodic indices of accentedness quantify trait-relevant variance, whereas rater background variables represent trait-irrelevant variance. The present study, therefore, investigated the extent to which raters' background characteristics and instrumentally measured prosodic indices of speakers' accentedness jointly influenced the rating of oral performances. Seventy U.S. undergraduate students rated the speaking and teaching proficiency of 11 international teaching assistants (ITAs). Using the PRAAT computer program, 5 min of continuous speech from each of the ITAs were instrumentally analyzed for a number of indices of speech rate, pausing, stress, and intonation. Dependent variables were undergraduates' ratings of ITA oral proficiency and instuctional competence. Rater background variables such as the listener's native speaker status and experience as a language tutor explained 7-9% of the variance in oral performance ratings, whereas 18-19% was attributable to the prosody variables. These findings suggest that U.S. undergraduates are sensitive to trait-relevant indicators of ITA oral proficiency. At the same time, their speech evaluations are subject to substantial bias based on their own backgrounds.

Original languageEnglish (US)
Pages (from-to)249-269
Number of pages21
JournalLanguage Assessment Quarterly
Volume9
Issue number3
DOIs
StatePublished - Jul 1 2012

Fingerprint

assistant
rating
Teaching
performance
data processing program
tutor
listener
speaking
Raters
International Teaching Assistants
Rating
trend
language
evaluation
Undergraduate
Proficiency
experience
student

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

@article{a2c2f42f79ba4106b2c8155e5550e0ce,
title = "Impact of rater characteristics and prosodic features of speaker accentedness on ratings of international teaching assistants' oral performance",
abstract = "Few prior studies have examined degree of fidelity between raters' assessments of oral performances and objectively observable prosodic indices of accentedness. Prosodic indices of accentedness quantify trait-relevant variance, whereas rater background variables represent trait-irrelevant variance. The present study, therefore, investigated the extent to which raters' background characteristics and instrumentally measured prosodic indices of speakers' accentedness jointly influenced the rating of oral performances. Seventy U.S. undergraduate students rated the speaking and teaching proficiency of 11 international teaching assistants (ITAs). Using the PRAAT computer program, 5 min of continuous speech from each of the ITAs were instrumentally analyzed for a number of indices of speech rate, pausing, stress, and intonation. Dependent variables were undergraduates' ratings of ITA oral proficiency and instuctional competence. Rater background variables such as the listener's native speaker status and experience as a language tutor explained 7-9{\%} of the variance in oral performance ratings, whereas 18-19{\%} was attributable to the prosody variables. These findings suggest that U.S. undergraduates are sensitive to trait-relevant indicators of ITA oral proficiency. At the same time, their speech evaluations are subject to substantial bias based on their own backgrounds.",
author = "Okim Kang",
year = "2012",
month = "7",
day = "1",
doi = "10.1080/15434303.2011.642631",
language = "English (US)",
volume = "9",
pages = "249--269",
journal = "Language Assessment Quarterly",
issn = "1543-4303",
publisher = "Routledge",
number = "3",

}

TY - JOUR

T1 - Impact of rater characteristics and prosodic features of speaker accentedness on ratings of international teaching assistants' oral performance

AU - Kang, Okim

PY - 2012/7/1

Y1 - 2012/7/1

N2 - Few prior studies have examined degree of fidelity between raters' assessments of oral performances and objectively observable prosodic indices of accentedness. Prosodic indices of accentedness quantify trait-relevant variance, whereas rater background variables represent trait-irrelevant variance. The present study, therefore, investigated the extent to which raters' background characteristics and instrumentally measured prosodic indices of speakers' accentedness jointly influenced the rating of oral performances. Seventy U.S. undergraduate students rated the speaking and teaching proficiency of 11 international teaching assistants (ITAs). Using the PRAAT computer program, 5 min of continuous speech from each of the ITAs were instrumentally analyzed for a number of indices of speech rate, pausing, stress, and intonation. Dependent variables were undergraduates' ratings of ITA oral proficiency and instuctional competence. Rater background variables such as the listener's native speaker status and experience as a language tutor explained 7-9% of the variance in oral performance ratings, whereas 18-19% was attributable to the prosody variables. These findings suggest that U.S. undergraduates are sensitive to trait-relevant indicators of ITA oral proficiency. At the same time, their speech evaluations are subject to substantial bias based on their own backgrounds.

AB - Few prior studies have examined degree of fidelity between raters' assessments of oral performances and objectively observable prosodic indices of accentedness. Prosodic indices of accentedness quantify trait-relevant variance, whereas rater background variables represent trait-irrelevant variance. The present study, therefore, investigated the extent to which raters' background characteristics and instrumentally measured prosodic indices of speakers' accentedness jointly influenced the rating of oral performances. Seventy U.S. undergraduate students rated the speaking and teaching proficiency of 11 international teaching assistants (ITAs). Using the PRAAT computer program, 5 min of continuous speech from each of the ITAs were instrumentally analyzed for a number of indices of speech rate, pausing, stress, and intonation. Dependent variables were undergraduates' ratings of ITA oral proficiency and instuctional competence. Rater background variables such as the listener's native speaker status and experience as a language tutor explained 7-9% of the variance in oral performance ratings, whereas 18-19% was attributable to the prosody variables. These findings suggest that U.S. undergraduates are sensitive to trait-relevant indicators of ITA oral proficiency. At the same time, their speech evaluations are subject to substantial bias based on their own backgrounds.

UR - http://www.scopus.com/inward/record.url?scp=84865700995&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865700995&partnerID=8YFLogxK

U2 - 10.1080/15434303.2011.642631

DO - 10.1080/15434303.2011.642631

M3 - Article

AN - SCOPUS:84865700995

VL - 9

SP - 249

EP - 269

JO - Language Assessment Quarterly

JF - Language Assessment Quarterly

SN - 1543-4303

IS - 3

ER -