Publicación:
Improving reliability estimation in cognitive diagnosis modeling

Vista sencilla de ítem
dc.contributor.authorTorre, Jimmy de la
dc.contributor.authorSorrel, Miguel A.
dc.contributor.authorNájera, Pablo
dc.contributor.authorAbad, Francisco
dc.contributor.authorSchames Kreitchmann, Rodrigo
dc.date.accessioned2024-05-20T11:48:04Z
dc.date.available2024-05-20T11:48:04Z
dc.date.issued2023-10-01
dc.description.abstractCognitive diagnosis models (CDMs) are used in educational, clinical, or personnel selection settings to classify respondents with respect to discrete attributes, identifying strengths and needs, and thus allowing to provide tailored training/treatment. As in any assessment, an accurate reliability estimation is crucial for valid score interpretations. In this sense, most CDM reliability indices are based on the posterior probabilities of the estimated attribute profiles. These posteriors are traditionally computed using point estimates for the model parameters as approximations to their populational values. If the uncertainty around these parameters is unaccounted for, the posteriors may be overly peaked, deriving into overestimated reliabilities. This article presents a multiple imputation (MI) procedure to integrate out the model parameters in the estimation of the posterior distributions, thus correcting the reliability estimation. A simulation study was conducted to compare the MI procedure with the traditional reliability estimation. Five factors were manipulated: the attribute structure, the CDM model (DINA and G-DINA), test length, sample size, and item quality. Additionally, an illustration using the Examination for the Certificate of Proficiency in English data was analyzed. The effect of sample size was studied by sampling subsets of subjects from the complete data. In both studies, the traditional reliability estimation systematically provided overestimated reliabilities, whereas the MI procedure offered more accurate results. Accordingly, practitioners in small educational or clinical settings should be aware that the reliability estimation using model parameter point estimates may be positively biased. R codes for the MI procedure are made availableen
dc.description.versionversión publicada
dc.identifier.doihttps://doi.org/10.3758/s13428-022-01967-5
dc.identifier.issn1554-351X; eISSN: 1554-3528
dc.identifier.urihttps://hdl.handle.net/20.500.14468/12570
dc.journal.titleBehavior Research Methods
dc.journal.volume55
dc.language.isoen
dc.publisherSpringer
dc.relation.centerFacultad de Psicología
dc.relation.departmentMetodología de las Ciencias del Comportamiento
dc.rightsAtribución 4.0 Internacional
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rights.urihttp://creativecommons.org/licenses/by/4.0
dc.subject.keywordsCognitive diagnosis
dc.subject.keywordsDiagnostic classification
dc.subject.keywordsReliability
dc.subject.keywordsClassification accuracy
dc.subject.keywordsMultiple imputation
dc.titleImproving reliability estimation in cognitive diagnosis modelinges
dc.typeartículoes
dc.typejournal articleen
dspace.entity.typePublication
relation.isAuthorOfPublicationf5f17927-c033-42e0-867c-3a658141ef26
relation.isAuthorOfPublication.latestForDiscoveryf5f17927-c033-42e0-867c-3a658141ef26
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
SCHAMES_KREITCHMANN_Rodrigo_Improving_reliability.pdf
Tamaño:
2.03 MB
Formato:
Adobe Portable Document Format
Descargar
Colecciones
Artículos y papers