Publicación: Comparación de modelos pre-entrenados basados en Transformers aplicados a la Búsqueda de Respuestas sobre COVID-19
Cargando...
Fecha
2021-10-01
Autores
Editor/a
Director/a
Tutor/a
Coordinador/a
Prologuista
Revisor/a
Ilustrador/a
Derechos de acceso
Atribución-NoComercial-SinDerivadas 4.0 Internacional
info:eu-repo/semantics/openAccess
info:eu-repo/semantics/openAccess
Título de la revista
ISSN de la revista
Título del volumen
Editor
Universidad Nacional de Educación a Distancia (España). Escuela Técnica Superior de Ingeniería Informática. Departamento de Lenguajes y Sistemas Informáticos
Resumen
Debido a la pandemia mundial provocada por el virus COVID-19, en el año 2020 surgen diversos desafíos en el ámbito de la investigación del procesamiento del lenguaje natural para intentar proporcionar sistemas automáticos que respondan de forma eficiente a preguntas relacionadas con la enfermedad. En este proyecto, se presenta una adaptación de un sistema existente de búsqueda de respuestas realizada para dar solución a uno de estos desafíos, el EPIC-QA. Dentro de este ámbito, el desafío propone dos tareas: dar respuesta a un conjunto de preguntas realizadas por usuarios generalistas y a un conjunto de preguntas formuladas por perfiles científicos o médicos. Para ello, el dataset utilizado incluye dos baterías de documentos distintas, una de documentos científicos y otra de textos más generalistas, que se adaptan mejor a cada una de las tareas. El objetivo del proyecto era analizar el impacto del uso de modelos pre-entrenados para el módulo de extracción de respuestas, realizando una comparativa entre tres modelos distintos. El sistema para la experimentación está preparado para responder a cualquier tipo de pregunta y consta de un módulo de recuperación de información a partir de una indexación de los documentos del dataset, seguido del módulo de extracción de respuestas para el que se implementaron las tres configuraciones distintas, con modelos basados en BERT utilizando Transformes y entrenados con baterías de datos adaptadas al ámbito científico y médico: SciBERT, ELECTRA y RoBERTa. Para analizar los resultados, se utilizó el método propuesto en EPIQ-QA basado en nuggets anotados manualmente por los evaluadores. Los resultados obtenidos en la experimentación muestran cómo una de las configuraciones propuestas se adapta mejor a los distintos escenarios planteados, obteniendo en todos los casos las mejores puntuaciones.
Due to the global pandemic caused by the COVID-19 virus, different Natural Language Processing research challenges were proposed for providing systems capable to efficiently answer virus linked questions. In this project, an adaptation of an existing answer searching system created to solve one of the mentioned challenges (EPIC-QA) is presented. Within this scope, the challenge proposes two tasks: answering a set of questions asked by general users and a set of questions asked by scientific or medical profiles. To do this, the dataset used includes two different document batteries, one for scientific documents and the other for more generalists texts, which are better adapted to each of the tasks. The main goal of the project was to analyze the impact of the use of pre-trained models for the question answering module, making a comparison between three different models. The system for experimentation is prepared to answer any type of question and consists of an information retrieval module from an indexing of the dataset documents, followed by the question answering module for which the three different configurations were implemented with models based on BERT using Transforms and trained with datasets adapted to the scientific and medical field: SciBERT, ELECTRA and RoBERTa. To analyze the results, the method proposed in EPIQ-QA based on nuggets manually annotated by the evaluators was used. The obtained results show one of the configurations as the best for the proposed scenarios in terms of solution metrics.
Due to the global pandemic caused by the COVID-19 virus, different Natural Language Processing research challenges were proposed for providing systems capable to efficiently answer virus linked questions. In this project, an adaptation of an existing answer searching system created to solve one of the mentioned challenges (EPIC-QA) is presented. Within this scope, the challenge proposes two tasks: answering a set of questions asked by general users and a set of questions asked by scientific or medical profiles. To do this, the dataset used includes two different document batteries, one for scientific documents and the other for more generalists texts, which are better adapted to each of the tasks. The main goal of the project was to analyze the impact of the use of pre-trained models for the question answering module, making a comparison between three different models. The system for experimentation is prepared to answer any type of question and consists of an information retrieval module from an indexing of the dataset documents, followed by the question answering module for which the three different configurations were implemented with models based on BERT using Transforms and trained with datasets adapted to the scientific and medical field: SciBERT, ELECTRA and RoBERTa. To analyze the results, the method proposed in EPIQ-QA based on nuggets manually annotated by the evaluators was used. The obtained results show one of the configurations as the best for the proposed scenarios in terms of solution metrics.
Descripción
Categorías UNESCO
Palabras clave
Citación
Centro
Facultades y escuelas::E.T.S. de Ingeniería Informática
Departamento
Lenguajes y Sistemas Informáticos