Automated English proficiency scoring of unconstrained speech using prosodic features

Okim Kang, David O. Johnson

Research output: Contribution to journalConference article

Abstract

This paper evaluates the performance of 17 machinelearning classifiers in automatically scoring the English proficiency of unconstrained speech. Each classifier was tested with different groups of features drawn from a master set of prosodic measures founded in Brazil’s model [3]. The prosodic measures were calculated from the output of an ASR that recognizes phones instead of words and other software designed to detect the elements of Brazil’s prosody model. The performance of the best classifier was 0.68 (p < 0.01) in terms of the correlation between the computer’s calculated proficiency ratings and those scored by humans. Using only prosodic features, this correlation is in the range of other similar computer programs for automatically scoring the proficiency of unconstrained speech.

Original languageEnglish (US)
Pages (from-to)617-620
Number of pages4
JournalProceedings of the International Conference on Speech Prosody
Volume2018-June
DOIs
StatePublished - Jan 1 2018
Event9th International Conference on Speech Prosody, SP 2018 - Poznan, Poland
Duration: Jun 13 2018Jun 16 2018

Fingerprint

data processing program
performance
rating
Group
English Proficiency
Scoring
Classifier
Proficiency
software
Prosody
Software
Phone
Rating
Machine Learning

Keywords

  • Automatic speech recognition (ASR)
  • Brazil’s prosody model
  • Large vocabulary spontaneous speech recognition (LVCSR)
  • World Englishes

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Automated English proficiency scoring of unconstrained speech using prosodic features. / Kang, Okim; Johnson, David O.

In: Proceedings of the International Conference on Speech Prosody, Vol. 2018-June, 01.01.2018, p. 617-620.

Research output: Contribution to journalConference article

@article{19399bcee0b24f69999cce34b100bb8b,
title = "Automated English proficiency scoring of unconstrained speech using prosodic features",
abstract = "This paper evaluates the performance of 17 machinelearning classifiers in automatically scoring the English proficiency of unconstrained speech. Each classifier was tested with different groups of features drawn from a master set of prosodic measures founded in Brazil’s model [3]. The prosodic measures were calculated from the output of an ASR that recognizes phones instead of words and other software designed to detect the elements of Brazil’s prosody model. The performance of the best classifier was 0.68 (p < 0.01) in terms of the correlation between the computer’s calculated proficiency ratings and those scored by humans. Using only prosodic features, this correlation is in the range of other similar computer programs for automatically scoring the proficiency of unconstrained speech.",
keywords = "Automatic speech recognition (ASR), Brazil’s prosody model, Large vocabulary spontaneous speech recognition (LVCSR), World Englishes",
author = "Okim Kang and Johnson, {David O.}",
year = "2018",
month = "1",
day = "1",
doi = "10.21437/SpeechProsody.2018-125",
language = "English (US)",
volume = "2018-June",
pages = "617--620",
journal = "Proceedings of the International Conference on Speech Prosody",
issn = "2333-2042",

}

TY - JOUR

T1 - Automated English proficiency scoring of unconstrained speech using prosodic features

AU - Kang, Okim

AU - Johnson, David O.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper evaluates the performance of 17 machinelearning classifiers in automatically scoring the English proficiency of unconstrained speech. Each classifier was tested with different groups of features drawn from a master set of prosodic measures founded in Brazil’s model [3]. The prosodic measures were calculated from the output of an ASR that recognizes phones instead of words and other software designed to detect the elements of Brazil’s prosody model. The performance of the best classifier was 0.68 (p < 0.01) in terms of the correlation between the computer’s calculated proficiency ratings and those scored by humans. Using only prosodic features, this correlation is in the range of other similar computer programs for automatically scoring the proficiency of unconstrained speech.

AB - This paper evaluates the performance of 17 machinelearning classifiers in automatically scoring the English proficiency of unconstrained speech. Each classifier was tested with different groups of features drawn from a master set of prosodic measures founded in Brazil’s model [3]. The prosodic measures were calculated from the output of an ASR that recognizes phones instead of words and other software designed to detect the elements of Brazil’s prosody model. The performance of the best classifier was 0.68 (p < 0.01) in terms of the correlation between the computer’s calculated proficiency ratings and those scored by humans. Using only prosodic features, this correlation is in the range of other similar computer programs for automatically scoring the proficiency of unconstrained speech.

KW - Automatic speech recognition (ASR)

KW - Brazil’s prosody model

KW - Large vocabulary spontaneous speech recognition (LVCSR)

KW - World Englishes

UR - http://www.scopus.com/inward/record.url?scp=85050190339&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050190339&partnerID=8YFLogxK

U2 - 10.21437/SpeechProsody.2018-125

DO - 10.21437/SpeechProsody.2018-125

M3 - Conference article

AN - SCOPUS:85050190339

VL - 2018-June

SP - 617

EP - 620

JO - Proceedings of the International Conference on Speech Prosody

JF - Proceedings of the International Conference on Speech Prosody

SN - 2333-2042

ER -