Combining evaluation metrics via the unanimous improvement ratio and its application in weps clustering task

Amigó, Enrique, Gonzalo, Julio, Artiles, Javier y Verdejo, Felisa . (2011) Combining evaluation metrics via the unanimous improvement ratio and its application in weps clustering task. Journal of Artificial Intelligence Research 42 (2011) 689-718. DOI:10.1613/jair.3401

Ficheros (Some files may be inaccessible until you login with your e-spacio credentials)
Nombre Descripción Tipo MIME Size
Documento.pdf Pdf del documento application/pdf

Título Combining evaluation metrics via the unanimous improvement ratio and its application in weps clustering task
Autor(es) Amigó, Enrique
Gonzalo, Julio
Artiles, Javier
Verdejo, Felisa
Materia(s) Informática
Resumen Many Artificial Intelligence tasks cannot be evaluated with a single quality criterion and some sort of weighted combination is needed to provide system rankings. A problem of weighted combination measures is that slight changes in the relative weights may produce substantial changes in the system rankings. This paper introduces the Unanimous Improvement Ratio (UIR), a measure that complements standard metric combination criteria (such as van Rijsbergen's F-measure) and indicates how robust the measured differences are to changes in the relative weights of the individual metrics. UIR is meant to elucidate whether a perceived difference between two systems is an artifact of how individual metrics are weighted. Besides discussing the theoretical foundations of UIR, this paper presents empirical results that confirm the validity and usefulness of the metric for the Text Clustering problem, where there is a tradeoff between precision and recall based metrics and results are particularly sensitive to the weighting scheme used to combine them. Remarkably, our experiments show that UIR can be used as a predictor of how well differences between systems measured on a given test bed will also hold in a different test bed.
Editor(es) Association for the Advancement of Artificial Intelligence
Fecha 2011-12-01
Formato application/pdf
Identificador http://e-spacio.uned.es/fez/view/bibliuned:DptoLSI-ETSI-MA2VICMR-1025
bibliuned:DptoLSI-ETSI-MA2VICMR-1025
DOI - identifier doi:10.1613/jair.3401
Publicado en la Revista Journal of Artificial Intelligence Research 42 (2011) 689-718. DOI:10.1613/jair.3401
Idioma eng
Versión de la publicación publishedVersion
Relacionado con el proyecto: info:eu-repo/grantAgreement/S2009/TIC-1542
Tipo de recurso Article
Derechos de acceso y licencia http://creativecommons.org/licenses/by-nc-nd/4.0
info:eu-repo/semantics/openAccess
Tipo de acceso Acceso abierto

 
Versiones
Versión Tipo de filtro
Contador de citas: Google Scholar Search Google Scholar
Estadísticas de acceso: 542 Visitas, 290 Descargas  -  Estadísticas en detalle
Creado: Wed, 19 Nov 2014, 15:19:27 CET