No hay miniatura disponible
Fecha
2022-01-01
Editor/a
Director/a
Tutor/a
Coordinador/a
Prologuista
Revisor/a
Ilustrador/a
Derechos de acceso
info:eu-repo/semantics/openAccess
Título de la revista
ISSN de la revista
Título del volumen
Editorial
Association for Computational Linguistics
Citas
0 citas en 
3 citas en 
Resumen
In this paper we conduct a set of experiments aimed to improve our understanding of the lack of semantic isometry in BERT, i.e. the lack of correspondence between the embedding and meaning spaces of its contextualized word representations. Our empirical results show that, contrary to popular belief, the anisotropy is not the root cause of the poor performance of these contextual models’ embeddings in semantic tasks. What does affect both the anisotropy and semantic isometry is a set of known biases: frequency, subword, punctuation, and case. For each one of them, we measure its magnitude and the effect of its removal, showing that these biases contribute but do not completely explain the phenomenon of anisotropy and lack of semantic isometry of these contextual language models.
Descripción
The registered version of this conference paper, first published in "Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4271–4281, Abu Dhabi, United Arab Emirates", is available online at the publisher's website: Association for Computational Linguistics, https://doi.org/10.18653/v1/2022.findings-emnlp.314
La versión registrada de esta comunicación, publicada por primera vez en "Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4271–4281, Abu Dhabi, United Arab Emirates", está disponible en línea en el sitio web del editor: Association for Computational Linguistics, https://doi.org/10.18653/v1/2022.findings-emnlp.314
La versión registrada de esta comunicación, publicada por primera vez en "Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4271–4281, Abu Dhabi, United Arab Emirates", está disponible en línea en el sitio web del editor: Association for Computational Linguistics, https://doi.org/10.18653/v1/2022.findings-emnlp.314
Categorías UNESCO
Palabras clave
Citación
Alejandro Fuster Baggetto and Victor Fresno. 2022. Is anisotropy really the cause of BERT embeddings not being semantic?. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4271–4281, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Centro
E.T.S. de Ingeniería Informática
Departamento
Lenguajes y Sistemas Informáticos