Publicación: Agrupación automática de mensajes de foros
Cargando...
Fecha
2024
Autores
Editor/a
Director/a
Tutor/a
Coordinador/a
Prologuista
Revisor/a
Ilustrador/a
Derechos de acceso
info:eu-repo/semantics/openAccess
Título de la revista
ISSN de la revista
Título del volumen
Editor
Universidad Nacional de Educación a Distancia (España). Escuela Técnica Superior de Ingeniería Informática.
Resumen
Los foros de discusión permiten formular preguntas y obtener respuestas aprovechando la denominada sabiduría de las masas, y se han convertido en herramientas esenciales de cursos en línea, como los de la UNED. Los foros suelen estar divididos en subforos dedicados a temas específicos, pero a menudo los usuarios escriben mensajes en el subforo equivocado, lo que dificulta su visibilidad y puede hacer necesaria una reubicación manual. Para ayudar a prevenir estos errores y aliviar las tareas de mantenimiento, en este trabajo se desarrolla un sistema que agrupa automáticamente foros como los de la UNED y permite medir la similitud semántica entre mensajes. Asimismo, dada una estructura de subforos llena de mensajes y un mensaje nuevo, el sistema es capaz de generar recomendaciones de inserción basadas en similitud. El trabajo incluye una parte investigativa fundamental en la que se lleva a cabo un análisis exploratorio de 7 foros de la UNED y se experimenta con diversas técnicas de procesamiento de lenguaje natural y de aprendizaje no supervisado. Por ejemplo, se ensaya con representaciones vectoriales de documentos de tipo bolsa de palabras así como con otras más modernas, como los embeddings de palabras e incluso de frases. Los mejores resultados se obtienen con versiones ponderadas de la bolsa de palabras y con modelos multilingües de codificación de frases pre-entrenados. En cuanto a la similitud entre mensajes, las métricas coseno y angular producen resultados parecidos, mas la segunda tiene la posible ventaja de ser propiamente una distancia. Por ´ultimo, se prueban los algoritmos de clustering k-medias, aglomerativo y HDBSCAN, que también es jerárquico pero basado en densidad. Los agrupamientos se evalúan usando medidas externas, como la información mutua ajustada, y también internas, como la silueta y el índice de validación basado en densidad. El algoritmo k-medias consigue el mejor alineamiento medio con la estructura de subforos original, pero los otros dos tienen asimismo ventajas, en cuanto a tiempo de ejecución y la información adicional que proporcionan sus jerarquías. El método HDBSCAN destaca por su flexibilidad, robustez y el carácter intuitivo de sus parámetros. El sistema de agrupación desarrollado es capaz de identificar por sí solo grupos que tienen pleno sentido. En ocasiones, dichos grupos son subconjuntos de un subforo original, e incluso pueden ser parientes cercanos de otros subconjuntos del mismo subforo en un agrupamiento jerárquico. Otras veces, los grupos generados son transversales a la estructura original, debido a la presencia de mensajes semejantes, por ejemplo agradecimientos, a través de los subforos. Aun cuando la estructura original resulte difícil de reproducir automáticamente, el ranking de similitud creado por el sistema debería de facilitar la colocación correcta de mensajes nuevos.
Discussion forums enable asking questions and obtaining answers through the wisdom of the crowd, and have become an essential tool in online courses, such as those run by UNED. Forums are typically divided in subforums devoted to specific topics, but users frequently write messages in the wrong subforum, which can hinder visibility and require manual relocation. To help prevent these errors and ease maintenance tasks, this work presents development of a system that automatically clusters messages from forums like those from UNED and enables measuring semantic similarity between messages. In addition, given a structure of subforums prefilled with messages and a new message, the system can generate recommendations for its assignment based on similarity. This work includes a fundamental investigative part consisting of an exploratory analysis of 7 UNED forums and experiments with various natural language processing and unsupervised learning techniques. For example, bag-of-words models are tried out along with more modern vector representations, such as word and sentence embeddings. The best results are obtained with weighted versions of the bag of words and with pretrained multilingual sentence encoders. With regard to message similarity, the cosine and angular metrics yield similar results, but the latter has the advantage of being a genuine distance. Lastly, the clustering algorithm trials cover k-means, agglomerative and HDBSCAN, which is also hierarchal but based on density. Clusters are evaluated using external measures, such as adjusted mutual information, and also intrinsic measures, such as the silhouette score and the density-based cluster validation index. The k-means algorithm achieves the best average alignment with the original structure of subforums, but the other two algorithms also have their own advantages, in terms of execution time and the additional information provided by their hierarchies. The HDBSCAN method stands out because of its flexibility, robustness, and the intuitive nature of its parameters. The developed clustering system is capable of autonomously identifying some meaningful clusters. Sometimes, those clusters are subsets of an original subforum, and may even be close relatives of other subsets of the same subforum in a hierarchal clustering. Other times, the generated clusters are transverse to the original structure, because of the presence of similar messages, for instance thank-yous, across subforums. Even in those cases where the original structure is difficult to reproduce automatically, the similarity raking created by the system should facilitate the correct placement of new messages.
Discussion forums enable asking questions and obtaining answers through the wisdom of the crowd, and have become an essential tool in online courses, such as those run by UNED. Forums are typically divided in subforums devoted to specific topics, but users frequently write messages in the wrong subforum, which can hinder visibility and require manual relocation. To help prevent these errors and ease maintenance tasks, this work presents development of a system that automatically clusters messages from forums like those from UNED and enables measuring semantic similarity between messages. In addition, given a structure of subforums prefilled with messages and a new message, the system can generate recommendations for its assignment based on similarity. This work includes a fundamental investigative part consisting of an exploratory analysis of 7 UNED forums and experiments with various natural language processing and unsupervised learning techniques. For example, bag-of-words models are tried out along with more modern vector representations, such as word and sentence embeddings. The best results are obtained with weighted versions of the bag of words and with pretrained multilingual sentence encoders. With regard to message similarity, the cosine and angular metrics yield similar results, but the latter has the advantage of being a genuine distance. Lastly, the clustering algorithm trials cover k-means, agglomerative and HDBSCAN, which is also hierarchal but based on density. Clusters are evaluated using external measures, such as adjusted mutual information, and also intrinsic measures, such as the silhouette score and the density-based cluster validation index. The k-means algorithm achieves the best average alignment with the original structure of subforums, but the other two algorithms also have their own advantages, in terms of execution time and the additional information provided by their hierarchies. The HDBSCAN method stands out because of its flexibility, robustness, and the intuitive nature of its parameters. The developed clustering system is capable of autonomously identifying some meaningful clusters. Sometimes, those clusters are subsets of an original subforum, and may even be close relatives of other subsets of the same subforum in a hierarchal clustering. Other times, the generated clusters are transverse to the original structure, because of the presence of similar messages, for instance thank-yous, across subforums. Even in those cases where the original structure is difficult to reproduce automatically, the similarity raking created by the system should facilitate the correct placement of new messages.
Descripción
Categorías UNESCO
Palabras clave
foro, agrupamiento, clustering, similitud, bolsa de palabras, embedding, procesamiento de lenguaje natural, forum, clustering, similarity, bag of words, embedding, natural language processing
Citación
Centro
Facultades y escuelas::E.T.S. de Ingeniería Informática
Departamento
Inteligencia Artificial