On the Significance of Graph Neural Networks With Pretrained Transformers in Content-Based Recommender Systems for Academic Article Classification

Liu, JiayunCastillo-Cara, ManuelGarcía Castro, Raúl2025-10-142025-10-142025-05-27Liu, J., Castillo-Cara, M. and García-Castro, R. (2025), On the Significance of Graph Neural Networks With Pretrained Transformers in Content-Based Recommender Systems for Academic Article Classification. Expert Systems, 42: e70073. https://doi.org/10.1111/exsy.700731468-0394https://doi.org/10.1111/exsy.70073https://hdl.handle.net/20.500.14468/30410The registered version of this article, first published in Expert Systems: The Journal of Knowledge Engineering, is available online at the publisher's website: Wiley, https://doi.org/10.1111/exsy.70073La versión registrada de este artículo, publicado por primera vez en Expert Systems: The Journal of Knowledge Engineering, está disponible en línea en el sitio web del editor: Wiley, https://doi.org/10.1111/exsy.70073This work was supported by CYTED Ciencia y Tecnología para el Desarrollo and Comunidad de Madrid.Recommender systems are tools for interacting with large and complex information spaces by providing a personalised view of such spaces, prioritising items that are likely to be of interest to the user. In addition, they serve as a significant tool in academic research, helping authors select the most appropriate journals for their academic articles. This paper presents a comprehensive study of various journal recommender systems, focusing on the synergy of graph neural networks (GNNs) with pretrained transformers for enhanced text classification. Furthermore, we propose a content-based journal recommender system that combines a pretrained Transformer with a Graph Attention Network (GAT) using title, abstract and keywords as input data. The proposed architecture enhances text representation by forming graphs from the Transformers' hidden states and attention matrices, excluding padding tokens. Our findings highlight that this integration improves the accuracy of the journal recommendations and reduces the transformer oversmoothing problem, with RoBERTa outperforming BERT models. Furthermore, excluding padding tokens from graph construction reduces training time by 8%–15%. Furthermore, we offer a publicly available dataset comprising 830,978 articles.eninfo:eu-repo/semantics/openAccess1203.04 Inteligencia artificialOn the Significance of Graph Neural Networks With Pretrained Transformers in Content-Based Recommender Systems for Academic Article ClassificationartículoBERTgraph attention networkgraph neural networkjournal recommendationrecommender systemsRoBERTatransformers