Classification Performance of Answer-Copying Indices Under Different Types of IRT Models

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Test fraud has recently received increased attention in the field of educational testing, and the use of comprehensive integrity analysis after test administration is recommended for investigating different types of potential test frauds. One type of test fraud involves answer copying between two examinees, and numerous statistical methods have been proposed in the literature to screen and identify unusual response similarity or irregular response patterns on multiple-choice tests. The current study examined the classification performance of answer-copying indices measured by the area under the receiver operating characteristic (ROC) curve under different item response theory (IRT) models (one- [1PL], two- [2PL], three-parameter [3PL] models, nominal response model [NRM]) using both simulated and real response vectors. The results indicated that although there is a slight increase in the performance for low amount of copying conditions (20%), when nominal response outcomes were used, these indices performed in a similar manner for 40% and 60% copying conditions when dichotomous response outcomes were utilized. The results also indicated that the performance with simulated response vectors was almost identically reproducible with real response vectors.

Original languageEnglish (US)
Pages (from-to)592-607
Number of pages16
JournalApplied Psychological Measurement
Volume40
Issue number8
DOIs
StatePublished - Nov 1 2016

Fingerprint

Fraud
model theory
ROC Curve
performance
fraud
statistical method
integrity
recipient

Keywords

  • answer copying
  • item response theory
  • person fit
  • response similarity
  • test score integrity
  • test security

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Cite this

Classification Performance of Answer-Copying Indices Under Different Types of IRT Models. / Zopluoglu, Cengiz.

In: Applied Psychological Measurement, Vol. 40, No. 8, 01.11.2016, p. 592-607.

Research output: Contribution to journalArticle

@article{e2f76645856c4956949b8fd4bc3efe05,
title = "Classification Performance of Answer-Copying Indices Under Different Types of IRT Models",
abstract = "Test fraud has recently received increased attention in the field of educational testing, and the use of comprehensive integrity analysis after test administration is recommended for investigating different types of potential test frauds. One type of test fraud involves answer copying between two examinees, and numerous statistical methods have been proposed in the literature to screen and identify unusual response similarity or irregular response patterns on multiple-choice tests. The current study examined the classification performance of answer-copying indices measured by the area under the receiver operating characteristic (ROC) curve under different item response theory (IRT) models (one- [1PL], two- [2PL], three-parameter [3PL] models, nominal response model [NRM]) using both simulated and real response vectors. The results indicated that although there is a slight increase in the performance for low amount of copying conditions (20{\%}), when nominal response outcomes were used, these indices performed in a similar manner for 40{\%} and 60{\%} copying conditions when dichotomous response outcomes were utilized. The results also indicated that the performance with simulated response vectors was almost identically reproducible with real response vectors.",
keywords = "answer copying, item response theory, person fit, response similarity, test score integrity, test security",
author = "Cengiz Zopluoglu",
year = "2016",
month = "11",
day = "1",
doi = "10.1177/0146621616664724",
language = "English (US)",
volume = "40",
pages = "592--607",
journal = "Applied Psychological Measurement",
issn = "0146-6216",
publisher = "SAGE Publications Inc.",
number = "8",

}

TY - JOUR

T1 - Classification Performance of Answer-Copying Indices Under Different Types of IRT Models

AU - Zopluoglu, Cengiz

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Test fraud has recently received increased attention in the field of educational testing, and the use of comprehensive integrity analysis after test administration is recommended for investigating different types of potential test frauds. One type of test fraud involves answer copying between two examinees, and numerous statistical methods have been proposed in the literature to screen and identify unusual response similarity or irregular response patterns on multiple-choice tests. The current study examined the classification performance of answer-copying indices measured by the area under the receiver operating characteristic (ROC) curve under different item response theory (IRT) models (one- [1PL], two- [2PL], three-parameter [3PL] models, nominal response model [NRM]) using both simulated and real response vectors. The results indicated that although there is a slight increase in the performance for low amount of copying conditions (20%), when nominal response outcomes were used, these indices performed in a similar manner for 40% and 60% copying conditions when dichotomous response outcomes were utilized. The results also indicated that the performance with simulated response vectors was almost identically reproducible with real response vectors.

AB - Test fraud has recently received increased attention in the field of educational testing, and the use of comprehensive integrity analysis after test administration is recommended for investigating different types of potential test frauds. One type of test fraud involves answer copying between two examinees, and numerous statistical methods have been proposed in the literature to screen and identify unusual response similarity or irregular response patterns on multiple-choice tests. The current study examined the classification performance of answer-copying indices measured by the area under the receiver operating characteristic (ROC) curve under different item response theory (IRT) models (one- [1PL], two- [2PL], three-parameter [3PL] models, nominal response model [NRM]) using both simulated and real response vectors. The results indicated that although there is a slight increase in the performance for low amount of copying conditions (20%), when nominal response outcomes were used, these indices performed in a similar manner for 40% and 60% copying conditions when dichotomous response outcomes were utilized. The results also indicated that the performance with simulated response vectors was almost identically reproducible with real response vectors.

KW - answer copying

KW - item response theory

KW - person fit

KW - response similarity

KW - test score integrity

KW - test security

UR - http://www.scopus.com/inward/record.url?scp=84990248032&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990248032&partnerID=8YFLogxK

U2 - 10.1177/0146621616664724

DO - 10.1177/0146621616664724

M3 - Article

AN - SCOPUS:84990248032

VL - 40

SP - 592

EP - 607

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

SN - 0146-6216

IS - 8

ER -