In-house or commercial speaking tests

Evaluating strengths for EAP placement

Joan M Jamieson, Linxiao Wang, Jacqueline Church

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests.

Original languageEnglish (US)
Pages (from-to)288-298
Number of pages11
JournalJournal of English for Academic Purposes
Volume12
Issue number4
DOIs
StatePublished - Dec 2013

Fingerprint

speaking
student
staff
Placement
ability
costs
language
deliberation
coverage
curriculum

Keywords

  • Assessment
  • Computers
  • ESL
  • Speaking
  • Versant English

ASJC Scopus subject areas

  • Education
  • Linguistics and Language
  • Language and Linguistics

Cite this

In-house or commercial speaking tests : Evaluating strengths for EAP placement. / Jamieson, Joan M; Wang, Linxiao; Church, Jacqueline.

In: Journal of English for Academic Purposes, Vol. 12, No. 4, 12.2013, p. 288-298.

Research output: Contribution to journalArticle

@article{680a5c672ef747628a36319345a69d2a,
title = "In-house or commercial speaking tests: Evaluating strengths for EAP placement",
abstract = "When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests.",
keywords = "Assessment, Computers, ESL, Speaking, Versant English",
author = "Jamieson, {Joan M} and Linxiao Wang and Jacqueline Church",
year = "2013",
month = "12",
doi = "10.1016/j.jeap.2013.09.003",
language = "English (US)",
volume = "12",
pages = "288--298",
journal = "Journal of English for Academic Purposes",
issn = "1475-1585",
publisher = "Elsevier BV",
number = "4",

}

TY - JOUR

T1 - In-house or commercial speaking tests

T2 - Evaluating strengths for EAP placement

AU - Jamieson, Joan M

AU - Wang, Linxiao

AU - Church, Jacqueline

PY - 2013/12

Y1 - 2013/12

N2 - When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests.

AB - When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests.

KW - Assessment

KW - Computers

KW - ESL

KW - Speaking

KW - Versant English

UR - http://www.scopus.com/inward/record.url?scp=84886782959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84886782959&partnerID=8YFLogxK

U2 - 10.1016/j.jeap.2013.09.003

DO - 10.1016/j.jeap.2013.09.003

M3 - Article

VL - 12

SP - 288

EP - 298

JO - Journal of English for Academic Purposes

JF - Journal of English for Academic Purposes

SN - 1475-1585

IS - 4

ER -