Abstract
The James, Demaree, and Wolf within-group agreement index (rwG) may be used to assess interrater agreement when two or more individuals (“judges”) provide ratings on one or more items (“targets”). Although the James et al. index is increasingly being used to assess whether the aggregation of data is justified for higher levels of analysis, the sampling distribution of rWG has not been delineated, and therefore it has not been possible to conduct tests of the statistical significance of obtained results. As a consequence of this current deficiency in knowledge about rWG, a Monte Carlo simulation program (AGREESIM) was developed to estimate selected quantiles for the sampling distribution of rWG under any user-specified null distribution of judges’ scores. The output is in the form of quantiles and their 95% confidence intervals (showing the precision of the quantile estimates).
| Original language | English (US) |
|---|---|
| Pages (from-to) | 435-437 |
| Number of pages | 3 |
| Journal | Educational and Psychological Measurement |
| Volume | 55 |
| Issue number | 3 |
| DOIs | |
| State | Published - 1995 |
Fingerprint
ASJC Scopus subject areas
- Developmental and Educational Psychology
- Applied Psychology
- Applied Mathematics
- Education
Cite this
Estimation of Quantiles for the Sampling Distribution of the rWG within Group Agreement Index. / Charnes, John M.; Schriesheim, Chester.
In: Educational and Psychological Measurement, Vol. 55, No. 3, 1995, p. 435-437.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Estimation of Quantiles for the Sampling Distribution of the rWG within Group Agreement Index
AU - Charnes, John M.
AU - Schriesheim, Chester
PY - 1995
Y1 - 1995
N2 - The James, Demaree, and Wolf within-group agreement index (rwG) may be used to assess interrater agreement when two or more individuals (“judges”) provide ratings on one or more items (“targets”). Although the James et al. index is increasingly being used to assess whether the aggregation of data is justified for higher levels of analysis, the sampling distribution of rWG has not been delineated, and therefore it has not been possible to conduct tests of the statistical significance of obtained results. As a consequence of this current deficiency in knowledge about rWG, a Monte Carlo simulation program (AGREESIM) was developed to estimate selected quantiles for the sampling distribution of rWG under any user-specified null distribution of judges’ scores. The output is in the form of quantiles and their 95% confidence intervals (showing the precision of the quantile estimates).
AB - The James, Demaree, and Wolf within-group agreement index (rwG) may be used to assess interrater agreement when two or more individuals (“judges”) provide ratings on one or more items (“targets”). Although the James et al. index is increasingly being used to assess whether the aggregation of data is justified for higher levels of analysis, the sampling distribution of rWG has not been delineated, and therefore it has not been possible to conduct tests of the statistical significance of obtained results. As a consequence of this current deficiency in knowledge about rWG, a Monte Carlo simulation program (AGREESIM) was developed to estimate selected quantiles for the sampling distribution of rWG under any user-specified null distribution of judges’ scores. The output is in the form of quantiles and their 95% confidence intervals (showing the precision of the quantile estimates).
UR - http://www.scopus.com/inward/record.url?scp=84973799195&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84973799195&partnerID=8YFLogxK
U2 - 10.1177/0013164495055003007
DO - 10.1177/0013164495055003007
M3 - Article
AN - SCOPUS:84973799195
VL - 55
SP - 435
EP - 437
JO - Educational and Psychological Measurement
JF - Educational and Psychological Measurement
SN - 0013-1644
IS - 3
ER -