On bank assembly and block selection in multidimensional forced-choice adaptive assessments

Schames Kreitchmann, Rodrigo, Sorrel, Miguel A. y Abad, Francisco J. . (2023) On bank assembly and block selection in multidimensional forced-choice adaptive assessments. Educational and Psychological Measurement, 83(2), 294-321

Ficheros (Some files may be inaccessible until you login with your e-spacio credentials)
Nombre Descripción Tipo MIME Size
SCHAMES_KREITCHMANN_Rodrigo_On_bank_assembly.pdf SCHAMES_KREITCHMANN_Rodrigo_On_bank_assembly.pdf application/pdf 933.88KB

Título On bank assembly and block selection in multidimensional forced-choice adaptive assessments
Autor(es) Schames Kreitchmann, Rodrigo
Sorrel, Miguel A.
Abad, Francisco J.
Materia(s) Psicología
Abstract Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in non-cognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, IRT models enable the estimation of non-ipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of 1) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled on-the-fly considering every possible pair of items), and 2) block selection rules (i.e., T, and Bayesian D and A-rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a non-adaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A-rule with questionnaires assembled on-the-fly, the T-rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT.
Palabras clave forced-choice format
ipsative data
multidimensional IRT
adaptive testing
item selection
Editor(es) SAGE
Fecha 2023-04-01
Formato application/pdf
Identificador bibliuned:DptoMCC-FPSI-Articulos-Rschames-0003
http://e-spacio.uned.es/fez/view/bibliuned:DptoMCC-FPSI-Articulos-Rschames-0003
DOI - identifier https://doi.org/10.1177/00131644221087986
ISSN - identifier 0013-1644; eISSN: 1552-3888
Nombre de la revista Educational and Psychological Measurement
Número de Volumen 83
Número de Issue 2
Página inicial 294
Página final 321
Publicado en la Revista Educational and Psychological Measurement, 83(2), 294-321
Idioma eng
Versión de la publicación acceptedVersion
Tipo de recurso Article
Derechos de acceso y licencia http://creativecommons.org/licenses/by-nc-nd/4.0
info:eu-repo/semantics/openAccess
Tipo de acceso Acceso abierto
Notas adicionales This is an Accepted Manuscript of an article published by SAGE in "Educational and Psychological Measurement, 83(2), 294-321" on Apr 2023, available at: https://doi.org/10.1177/00131644221087986
Notas adicionales Este es el manuscrito aceptado del artículo publicado por SAGE en "Educational and Psychological Measurement, 83(2), 294-321" en Abril de 2023, disponible en línea: https://doi.org/10.1177/00131644221087986

 
Versiones
Versión Tipo de filtro
Contador de citas: Google Scholar Search Google Scholar
Estadísticas de acceso: 28 Visitas, 14 Descargas  -  Estadísticas en detalle
Creado: Tue, 06 Feb 2024, 19:44:10 CET