Publicación:
A multimodal approach using fundus images and text meta-data in a machine learning classifier with embeddings to predict years with self-reported diabetes – an exploratory analysis

Vista sencilla de ítem
dc.contributor.authorCarrillo Larco, Rodrigo M.
dc.contributor.authorBravo Rocca, Gusseppe
dc.contributor.authorCastillo Cara, José Manuel
dc.contributor.authorXu, Xiaolin
dc.contributor.authorBernabé Ortiz, Antonio
dc.contributor.orcidhttps://orcid.org/0000-0002-2090-1856
dc.contributor.orcidhttps://orcid.org/0000-0001-6824-1124
dc.contributor.orcidhttps://orcid.org/0000-0002-8203-9878
dc.contributor.orcidhttps://orcid.org/0000-0002-6834-1376
dc.date.accessioned2024-05-28T16:55:48Z
dc.date.available2024-05-28T16:55:48Z
dc.date.issued2024-05-22
dc.descriptionThe registered version of this article, first published in Primary Care Diabetes, is available online at the publisher's website: Elsevier, https://doi.org/10.1016/j.pcd.2024.04.002
dc.descriptionLa versión registrada de este artículo, publicado por primera vez en Primary Care Diabetes, está disponible en línea en el sitio web del editor: Elsevier, https://doi.org/10.1016/j.pcd.2024.04.002
dc.description.abstractAims Machine learning models can use image and text data to predict the number of years since diabetes diagnosis; such model can be applied to new patients to predict, approximately, how long the new patient may have lived with diabetes unknowingly. We aimed to develop a model to predict self-reported diabetes duration. Methods We used the Brazilian Multilabel Ophthalmological Dataset. Unit of analysis was the fundus image and its meta-data, regardless of the patient. We included people 40 + years and fundus images without diabetic retinopathy. Fundus images and meta-data (sex, age, comorbidities and taking insulin) were passed to the MedCLIP model to extract the embedding representation. The embedding representation was passed to an Extra Tree Classifier to predict: 0–4, 5–9, 10–14 and 15 + years with self-reported diabetes. Results There were 988 images from 563 people (mean age = 67 years; 64 % were women). Overall, the F1 score was 57 %. The group 15 + years of self-reported diabetes had the highest precision (64 %) and F1 score (63 %), while the highest recall (69 %) was observed in the group 0–4 years. The proportion of correctly classified observations was 55 % for the group 0–4 years, 51 % for 5–9 years, 58 % for 10–14 years, and 64 % for 15 + years with self-reported diabetes. Conclusions The machine learning model had acceptable accuracy and F1 score, and correctly classified more than half of the patients according to diabetes duration. Using large foundational models to extract image and text embeddings seems a feasible and efficient approach to predict years living with self-reported diabetes.es
dc.description.versionversión final
dc.identifier.citationCarrillo-Larco, R. M., Bravo-Rocca, G., Castillo-Cara, M., Xu, X., & Bernabe-Ortiz, A. (2024). A multimodal approach using fundus images and text meta-data in a machine learning classifier with embeddings to predict years with self-reported diabetes–An exploratory analysis. Primary Care Diabetes.
dc.identifier.doihttps://doi.org/10.1016/j.pcd.2024.04.002
dc.identifier.issn1751-9918
dc.identifier.urihttps://hdl.handle.net/20.500.14468/22199
dc.journal.issue3
dc.journal.titlePrimary Care Diabetes
dc.journal.volume18
dc.language.isoen
dc.page.final332
dc.page.initial327
dc.publisherElsevier
dc.relation.centerE.T.S. de Ingeniería Informática
dc.relation.departmentInteligencia Artificial
dc.relation.researchgroupSistemas inteligentes de ayuda a la decisión
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/deed.es
dc.subject33 Ciencias Tecnológicas
dc.subject32 Ciencias Médicas
dc.subject.keywordsdeep learningen
dc.subject.keywordsartificial intelligenceen
dc.subject.keywordsnon-communicable diseasesen
dc.titleA multimodal approach using fundus images and text meta-data in a machine learning classifier with embeddings to predict years with self-reported diabetes – an exploratory analysisen
dc.typejournal articleen
dc.typeartículoes
dspace.entity.typePublication
relation.isAuthorOfPublicationc0e39bd2-c0d8-4743-953d-488baf6b977e
relation.isAuthorOfPublication.latestForDiscoveryc0e39bd2-c0d8-4743-953d-488baf6b977e
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
2024_PCD_Multimodaldiabetes.pdf
Tamaño:
499.4 KB
Formato:
Adobe Portable Document Format
Descargar
Bloque de licencias
Mostrando 1 - 1 de 1
No hay miniatura disponible
Nombre:
license.txt
Tamaño:
3.62 KB
Formato:
Item-specific license agreed to upon submission
Descripción:
Descargar
Colecciones
Artículos y papers