Persona:
Moreno Álvarez, Sergio

Foto de perfil
Dirección de correo electrónico
ORCID
0000-0002-1858-9920
Fecha de nacimiento
Proyectos de investigación
Unidades organizativas
Puesto de trabajo
Apellidos
Moreno Álvarez
Nombre de pila
Sergio
Nombre

Filtros

2018 - 201942020 - 202423
Restablecer filtros

Ajustes

Tiene archivos: true × search.filters.applied.f.itemDepartamento: Lenguajes y Sistemas Informáticos ×

Resultados de la búsqueda

Mostrando 1 - 10 de 27
  • Miniatura
    Publicación
    Evaluación de Rendimiento del Entrenamiento Distribuido de Redes Neuronales Profundas en Plataformas Heterogéneas
    (Universidad de Extremadura, 2019) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Haut, Juan Mario; Rico Gallego, Juan Antonio; Plaza, Javier; Díaz Martín, Juan Carlos; Vega Rodriguez, Miguel ángel; Plaza Miguel, Antonio J.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-8908-1606; https://orcid.org/0000-0002-8435-3844
    Asynchronous stochastic gradient descent es una tecnica de optimizacion comunmente utilizada en el entrenamiento distribuido de redes neuronales profundas. En distribuciones basadas en particionamiento de datos, se entrena una replica del modelo en cada unidad de procesamiento de la plataforma, utilizando conjuntos de muestras denominados mini-batches. Este es un proceso iterativo en el que al nal de cada mini-batch, las replicas combinan los gradientes calculados para actualizar su copia local de los parametros. Sin embargo, al utilizar asincronismo, las diferencias en el tiempo de entrenamiento por iteracion entre replicas provocan la aparicion del staleness, esto es, las replicas progresan a diferente velocidad y en el entrenamiento de cada replica se utiliza una vers on no actualizada de los parametros. Un alto gradde staleness tiene un impacto negativo en la precision del modelo resultante. Ademas, las plataformas de computacion de alto rendimiento suelen ser heterogeneas, compuestas por CPUs y GPUs de diferentes capacidades, lo que agrava el problema de staleness. En este trabajo, se propone aplicar t ecnicas de equilibrio de carga computacional, bien conocidas en el campo de la Computaci on de Altas Prestaciones, al entrenamiento distribuido de modelos profundos. A cada r eplica se asignar a un n umero de mini-batches en proporci on a su velocidad relativa. Los resultados experimentales obtenidos en una plataforma hete-rog enea muestran que, si bien la precisi on se mantiene constante, el rendimiento del entrenamiento aumenta considerablemente, o desde otro punto de vista, en el mismo tiempo de entrenamiento, se alcanza una mayor precisi on en las estimaciones del modelo. Discutimos las causas de tal incremento en el rendimiento y proponemos los pr oximos pasos para futuras investigaciones.
  • Miniatura
    Publicación
    Correlation-Aware Averaging for Federated Learning in Remote Sensing Data Classification
    (IEEE, 2024) Moreno Álvarez, Sergio; han, lirong; Paoletti, Mercedes Eugenia; Haut, Juan Mario; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X
    The increasing volume of remote sensing (RS) data offers substantial benefits for the extraction and interpretation of features from these scenes. Indeed, the detection of distinguishing features among captured materials and objects is crucial for classification purposes, such as in environmental monitoring applications. In these algorithms, the classes characterized by lower correlation often exhibit more distinct and discernible features, facilitating their differentiation in a straightforward manner. Nevertheless, the rise of Big Data provides a wide range of data acquired through multiple decentralized devices, where its susceptibility to be shared among various users or clients presents challenges in safeguarding privacy. Meanwhile, global features for similar classes are required to be learned for generalization purposes in the classification process. To address this, federated learning (FL) emerges as a privacy efficient decentralized solution. Firstly, in such scenarios, proprietary data is held by individual clients participating in the training of a global model. Secondly, clients may encounter challenges in identifying features that are more distinguishable within the data distributions of other clients. In this study, in order to handle these challenges, a novel methodology is proposed that considers the least correlated classes (LCCs) included in each client data distribution. This strategy exploits the distinctive features between classes, thereby enhancing performance and generalization ability in a secure and private environment.
  • Miniatura
    Publicación
    Optimizing Distributed Deep Learning in Heterogeneous Computing Platforms for Remote Sensing Data Classification
    (IEEE, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Cavallaro, Gabriele; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-3239-9904; https://orcid.org/0000-0001-6701-961X
    Applications from Remote Sensing (RS) unveiled unique challenges to Deep Learning (DL) due to the high volume and complexity of their data. On the one hand, deep neural network architectures have the capability to automatically ex-tract informative features from RS data. On the other hand, these models have massive amounts of tunable parameters, re-quiring high computational capabilities. Distributed DL with data parallelism on High-Performance Computing (HPC) sys-tems have proved necessary in dealing with the demands of DL models. Nevertheless, a single HPC system can be al-ready highly heterogeneous and include different computing resources with uneven processing power. In this context, a standard data parallelism strategy does not partition the data efficiently according to the available computing resources. This paper proposes an alternative approach to compute the gradient, which guarantees that the contribution to the gradi-ent calculation is proportional to the processing speed of each DL model's replica. The experimental results are obtained in a heterogeneous HPC system with RS data and demon-strate that the proposed approach provides a significant training speed up and gain in the global accuracy compared to one of the state-of-the-art distributed DL framework.
  • Miniatura
    Publicación
    Deep Attention-Driven HSI Scene Classification Based on Inverted Dot-Product
    (Institute of Electrical and Electronics Engineers Inc., 2022) Paoletti, Mercedes Eugenia; Tao, Xuanwen; han, lirong; Wu, Zhaoyue; Moreno Álvarez, Sergio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-1093-0079; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0002-6797-2440; https://orcid.org/0000-0001-6701-961X
    Capsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attention
  • Miniatura
    Publicación
    Deep Robust Hashing Using Self-Distillation for Remote Sensing Image Retrieval
    (IEEE, 2024) han,lirong; Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; Haut, Juan Mario; Plaza, Antonio; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-9613-1659
    This paper presents a novel self-distillation based deep robust hash for fast remote sensing (RS) image retrieval. Specifically, there are two primary processes in our proposed model: teacher learning (TL) and student learning (SL). Two transformed samples are produced from one sample image through nuanced and signalized transformations, respectively. Transformed samples are fed into both the TL and the SL flows. To reduce discrepancies in the processed samples and guarantee a consistent hash code, the parameters are shared by the two modules during the training stage. Then, a resilient module is employed to enhance the image features in order to ensure more dependable hash code production. Lastly, a three-component loss function is developed to train the entire model. Comprehensive experiments are conducted on two common RS datasets: UCMerced and AID. The experimental results validate that the proposed method has competitive performance against other RS image hashing methods.
  • Miniatura
    Publicación
    Estimación Automática del Coste de Comunicación de Aplicaciones Paralelas en Plataformas Heterogéneas
    (Universidad Extremadura, 2018) Moreno Álvarez, Sergio; Rico Gallego, Juan A.; Díaz Martín, Juan Carlos; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-8435-3844
    Optimizar el tiempo de ejecución de aplicaciones paralelas en plataformas heterogéneas de altas prestaciones es un problema complejo. Estas aplicaciones cient´ıficas normalmente se componen de kernels que implementan algoritmos como la multiplicación de matrices, ecuaciones en derivadas parciales o Transformadas de Fourier. Los kernels son ejecutados por los procesos desplegados en los diferentes recursos de cómputo de una plataforma, por ejemplo, en procesadores multi-core o aceleradores (GPUs, Xeon PHIs, etc.). El volumen de datos del kernel se distribuye entre los procesos de forma proporcional a su capacidad de cómputo, de forma que se equilibra la carga computacional global. Este equilibrado de carga no homogéneo tiene un impacto importante en el coste de las comunicaciones. La optimización del coste de las comunicaciones de éstas aplicaciones se aborda habitualmente mediante pruebas exhaustivas en la plataforma destino. Sin embargo, estas pruebas consumen recursos y tiempo, y a menudo se basan en la extrapolación de los resultados obtenidos con la ejecución de una versión reducida de la aplicación en la plataforma. Los Modelos Anal´ıticos de Rendimiento de Comunicaciones ofrecen una alternativa factible y prometedora en este sentido. Estos modelos representan el coste de las comunicaciones de un kernel en una plataforma heterogénea, ofreciendo una estimación precisa de su tiempo de comunicación de forma no invasiva, esto es, sin utilizar recursos de cómputo HPC en la estimación. Este trabajo contribuye ofreciendo una herramienta de estimación que permite representar y evaluar expresiones de coste de comunicaciones que siguen el modelo t- Lop. Adem´as, permite incluir el c´alculo de coste de las comunicaciones de forma autom´atica en algoritmos de particionamiento y optimización de comunicaciones. En este documento se proporcionan ejemplos tanto de uso b´asico como avanzado. Se incluyen tres casos de ejemplo de modelado de comunicaciones en kernels representativos utilizando la herramienta: la solución de una ecuación diferencial utilizando la técnica de elementos finitos, un algoritmo paralelo de multiplicación de matrices densas, y una simulación N-Body. Estos kernels utilizan diferentes patrones de comunicación y particionamiento del espacio de datos.
  • Miniatura
    Publicación
    Heterogeneous model parallelism for deep neural networks
    (ELSEVIER, 2021-06-21) Moreno Álvarez, Sergio; Haut, Juan M.; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473
    Deep neural networks (DNNs) have transformed computer vision, establishing themselves as the current state-of-the-art for image processing. Nevertheless, the training of current large DNN models is one of the main challenges to be solved. In this sense, data-parallelism has been the most widespread distributed training strategy since it is easy to program and can be applied to almost all cases. However, this solution suffers from several limitations, such as its high communication requirements and the memory constraints when training very large models. To overcome these limitations model-parallelism has been proposed, solving the most substantial problems of the former strategy. However, describing and implementing the parallelization of the training of a DNN model across a set of processes deployed on several devices is a challenging task. Current proposed solutions assume a homogeneous distribution, being impractical when working with devices of different computational capabilities, which is quite common on high performance computing platforms. To address previous shortcomings, this work proposes a novel model-parallelism technique considering heterogeneous platforms, where a load balancing mechanism between uneven devices of an HPC platform has been implemented. Our proposal takes advantage of the Google Brain’s Mesh-TensorFlow for convolutional networks, splitting computing tensors across filter dimension in order to balance the computational load of the available devices. Conducted experiments show an improvement in the exploitation of heterogeneous computational resources, enhancing the training performance. The code is available on: https://github.com/mhaut/HeterogeneusModelDNN.
  • Miniatura
    Publicación
    Self-Supervised Learning on Small In-Domain Datasets Can Overcome Supervised Learning in Remote Sensing
    (IEEE, 2024) Sanchez-Fernandez, Andres J.; Moreno Álvarez, Sergio; Rico Gallego, Juan Antonio; Tabik, Siham; https://orcid.org/0000-0001-6743-3570; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0003-4093-5356
    The availability of high-resolution satellite images has accelerated the creation of new datasets designed to tackle broader remote sensing (RS) problems. Although popular tasks, such as scene classification, have received significant attention, the recent release of the Land-1.0 RS dataset marks the initiation of endeavors to estimate land-use and land-cover (LULC) fraction values per RGB satellite image. This challenging problem involves estimating LULC composition, i.e., the proportion of different LULC classes from satellite imagery, with major applications in environmental monitoring, agricultural/urban planning, and climate change studies. Currently, supervised deep learning models—the state-of-the-art in image classification—require large volumes of labeled training data to provide good generalization. To face the challenges posed by the scarcity of labeled RS data, self-supervised learning (SSL) models have recently emerged, learning directly from unlabeled data by leveraging the underlying structure. This is the first article to investigate the performance of SSL in LULC fraction estimation on RGB satellite patches using in-domain knowledge. We also performed a complementary analysis on LULC scene classification. Specifically, we pretrained Barlow Twins, MoCov2, SimCLR, and SimSiam SSL models with ResNet-18 using the Sentinel2GlobalLULC small RS dataset and then performed transfer learning to downstream tasks on Land-1.0. Our experiments demonstrate that SSL achieves competitive or slightly better results when trained on a smaller high-quality in-domain dataset of 194 877 samples compared to the supervised model trained on ImageNet-1k with 1 281 167 samples. This outcome highlights the effectiveness of SSL using in-distribution datasets, demonstrating efficient learning with fewer but more relevant data.
  • No hay miniatura disponible
    Publicación
    Parameter-Free Attention Network for Spectral–Spatial Hyperspectral Image Classification
    (IEEE, 2023) Paoletti, Mercedes Eugenia; Tao, Xuanwen; han, lirong; Wu, Zhaoyue; Moreno Álvarez, Sergio; Kumar Roy, Swalpa; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-1093-0079; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0002-6797-2440; https://orcid.org/0000-0002-6580-3977
    Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in the HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this article, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches) while providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted—using different benchmark HSIs—to illustrate these advantages. The code is available on https://github.com/mhaut/Free2Resnet
  • Miniatura
    Publicación
    A tool to assess the communication cost of parallel kernels on heterogeneous platforms
    (Springer, 2020) Rico Gallego, Juan Antonio; Moreno Álvarez, Sergio; Díaz Martín, Juan Carlos; Lastovetsky, Alexey L.; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-8435-3844; https://orcid.org/0000-0001-9460-3897
    Ensuring applications to achieve an efficient usage of resources and fast execution time in the complex current heterogeneous high-performance computing platforms is a paramount problem. Essential efforts to reach the goal are the optimal partitioning of the data space between the processes composing a typical task/data-parallel application, and their right mapping and deployment on the platform. The computational and communication performance modeling describing the platform and the application behaviors is an increasingly recognized approach. This paper discusses the utility of the τ–Lop analytic communication performance model in facing these issues and contributes with a practical symbolic computation tool that represents, manipulates and accurately evaluates the formal communication cost expression derived from a hybrid kernel. We identify a set of scenarios where the tool could be applied, provide with both basic and advanced use examples and evaluate the tool on real-life kernels.