Evaluating the sampling performance of exploratory and cross-validated DETECT procedure with imperfect models

Research output: Contribution to journalArticle

Abstract

Among the methods proposed for identifying the number of latent traits in multidimensional IRT models, DETECT has attracted the attention of both methodologists and applied researchers as a nonparametric counterpart to other procedures. The current study investigated the overall performance of the DETECT procedure and its outcomes using a real-data sampling design recommended byMacCallum (2003) and compared the results from a purely simulated data set that was generated with a well-specified “perfect” model. The comparison revealed that the sampling behavior of the maximized DETECT value and R-ratio statistics was quite robust to minor factors and other model misspecifications that potentially exist in the real data set, as there were negligible differences between the results of the real and simulated data sets. Item classification accuracy was also nearly identical for the real and simulated data sets. The accuracy of the identified number of dimensions reported by DETECT was the only outcome with an obvious difference between the purely simulated data set and the real data set. While the difference was small for smaller sample sizes, the identified number of dimensions was more accurate for larger sample sizes when the population data set was purely simulated. In many instances, exploratory DETECT analysis outperformed the cross-validated DETECT analysis in terms of overall accuracy.

Original languageEnglish (US)
Pages (from-to)632-644
Number of pages13
JournalMultivariate Behavioral Research
Volume50
Issue number6
DOIs
StatePublished - Jan 1 2015

Keywords

  • DETECT
  • Dimensionality
  • Dimensionality assessment
  • IRT
  • Item response theory
  • Number of factors

ASJC Scopus subject areas

  • Statistics and Probability
  • Experimental and Cognitive Psychology
  • Arts and Humanities (miscellaneous)

Fingerprint Dive into the research topics of 'Evaluating the sampling performance of exploratory and cross-validated DETECT procedure with imperfect models'. Together they form a unique fingerprint.

  • Cite this