Persona: Moreno Álvarez, Sergio
Cargando...
Dirección de correo electrónico
ORCID
0000-0002-1858-9920
Fecha de nacimiento
Proyectos de investigación
Unidades organizativas
Puesto de trabajo
Apellidos
Moreno Álvarez
Nombre de pila
Sergio
Nombre
15 resultados
Resultados de la búsqueda
Mostrando 1 - 10 de 15
Publicación Optimizing Distributed Deep Learning in Heterogeneous Computing Platforms for Remote Sensing Data Classification(IEEE, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Cavallaro, Gabriele; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-3239-9904; https://orcid.org/0000-0001-6701-961XApplications from Remote Sensing (RS) unveiled unique challenges to Deep Learning (DL) due to the high volume and complexity of their data. On the one hand, deep neural network architectures have the capability to automatically ex-tract informative features from RS data. On the other hand, these models have massive amounts of tunable parameters, re-quiring high computational capabilities. Distributed DL with data parallelism on High-Performance Computing (HPC) sys-tems have proved necessary in dealing with the demands of DL models. Nevertheless, a single HPC system can be al-ready highly heterogeneous and include different computing resources with uneven processing power. In this context, a standard data parallelism strategy does not partition the data efficiently according to the available computing resources. This paper proposes an alternative approach to compute the gradient, which guarantees that the contribution to the gradi-ent calculation is proportional to the processing speed of each DL model's replica. The experimental results are obtained in a heterogeneous HPC system with RS data and demon-strate that the proposed approach provides a significant training speed up and gain in the global accuracy compared to one of the state-of-the-art distributed DL framework.Publicación Heterogeneous gradient computing optimization for scalable deep neural networks(Springer, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0001-6701-961XNowadays, data processing applications based on neural networks cope with the growth in the amount of data to be processed and with the increase in both the depth and complexity of the neural networks architectures, and hence in the number of parameters to be learned. High-performance computing platforms are provided with fast computing resources, including multi-core processors and graphical processing units, to manage such computational burden of deep neural network applications. A common optimization technique is to distribute the workload between the processes deployed on the resources of the platform. This approach is known as data-parallelism. Each process, known as replica, trains its own copy of the model on a disjoint data partition. Nevertheless, the heterogeneity of the computational resources composing the platform requires to unevenly distribute the workload between the replicas according to its computational capabilities, to optimize the overall execution performance. Since the amount of data to be processed is different in each replica, the influence of the gradients computed by the replicas in the global parameter updating should be different. This work proposes a modification of the gradient computation method that considers the different speeds of the replicas, and hence, its amount of data assigned. The experimental results have been conducted on heterogeneous high-performance computing platforms for a wide range of models and datasets, showing an improvement in the final accuracy with respect to current techniques, with a comparable performance.Publicación AAtt-CNN: Automatic Attention-Based Convolutional Neural Networks for Hyperspectral Image Classification(IEEE, 2023) Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; xue, yu; Haut, Juan M.; Plaza, Antonio; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-9069-7547; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-9613-1659Convolutional models have provided outstanding performance in the analysis of hyperspectral images (HSIs). These architectures are carefully designed to extract intricate information from nonlinear features for classification tasks. Notwithstanding their results, model architectures are manually engineered and further optimized for generalized feature extraction. In general terms, deep architectures are time-consuming for complex scenarios, since they require fine-tuning. Neural architecture search (NAS) has emerged as a suitable approach to tackle this shortcoming. In parallel, modern attention-based methods have boosted the recognition of sophisticated features. The search for optimal neural architectures combined with attention procedures motivates the development of this work. This article develops a new method to automatically design and optimize convolutional neural networks (CNNs) for HSI classification using channel-based attention mechanisms. Specifically, 1-D and spectral–spatial (3-D) classifiers are considered to handle the large amount of information contained in HSIs from different perspectives. Furthermore, the proposed automatic attention-based CNN ( AAtt-CNN ) method meets the requirement to lower the large computational overheads associated with architectural search. It is compared with current state-of-the-art (SOTA) classifiers. Our experiments, conducted using a wide range of HSI images, demonstrate that AAtt-CNN succeeds in finding optimal architectures for classification, leading to SOTA results.Publicación Training deep neural networks: a static load balancing approach(Springer, 2020-03-02) Moreno Álvarez, Sergio; Haut, Juan Mario; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Díaz Martín, Juan Carlos; Plaza, Javier; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-8435-3844; https://orcid.org/0000-0002-8908-1606Deep neural networks are currently trained under data-parallel setups on high-performance computing (HPC) platforms, so that a replica of the full model is charged to each computational resource using non-overlapped subsets known as batches. Replicas combine the computed gradients to update their local copies at the end of each batch. However, differences in performance of resources assigned to replicas in current heterogeneous platforms induce waiting times when synchronously combining gradients, leading to an overall performance degradation. Albeit asynchronous communication of gradients has been proposed as an alternative, it suffers from the so-called staleness problem. This is due to the fact that the training in each replica is computed using a stale version of the parameters, which negatively impacts the accuracy of the resulting model. In this work, we study the application of well-known HPC static load balancing techniques to the distributed training of deep models. Our approach is assigning a different batch size to each replica, proportional to its relative computing capacity, hence minimizing the staleness problem. Our experimental results (obtained in the context of a remotely sensed hyperspectral image processing application) show that, while the classification accuracy is kept constant, the training time substantially decreases with respect to unbalanced training. This is illustrated using heterogeneous computing platforms, made up of CPUs and GPUs with different performance.Publicación Deep mixed precision for hyperspectral image classification(Springer, 2021-02-03) Paoletti, Mercedes Eugenia; X. Tao; Haut, Juan Mario; Moreno Álvarez, Sergio; Plaza, Antonio; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-9613-1659Hyperspectral images (HSIs) record scenes at different wavelength channels, providing detailed spatial and spectral information. How to storage and process this highdimensional data plays a vital role in many practical applications, where classification technologies have emerged as excellent processing tools. However, their high computational complexity and energy requirements bring some challenges. Adopting low-power consumption architectures and deep learning (DL) approaches has to provide acceptable computing capabilities without reducing accuracy demand. However, most DL architectures employ single-precision (FP32) to train models, and some big DL architectures will have a limitation on memory and computation resources. This can negatively affect the network learning process. This letter leads these challenges by using mixed precision into DL architectures for HSI classification to speed up the training process and reduce the memory consumption/access. Proposed models are evaluated on four widely used data sets. Also, low and highpower consumption devices are compared, considering NVIDIA Jetson Xavier and Titan RTX GPUs, to evaluate the proposal viability in on-board processing devices. Obtained results demonstrate the efficiency and effectiveness of these models within HSI classification task for both devices. Source codes: https ://githu b.com/mhaut / CNN-MP-HSI.Publicación Parameter-Free Attention Network for Spectral–Spatial Hyperspectral Image Classification(IEEE, 2023) Paoletti, Mercedes Eugenia; Tao, Xuanwen; han, lirong; Wu, Zhaoyue; Moreno Álvarez, Sergio; Kumar Roy, Swalpa; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-1093-0079; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0002-6797-2440; https://orcid.org/0000-0002-6580-3977Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in the HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this article, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches) while providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted—using different benchmark HSIs—to illustrate these advantages. The code is available on https://github.com/mhaut/Free2ResnetPublicación Distributed Deep Learning for Remote Sensing Data Interpretation(IEEE, 2021-03-15) Haut, Juan Mario; Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; Plaza, Javier; Rico Gallego, Juan Antonio; Plaza, Antonio; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-2384-9141; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-9613-1659As a newly emerging technology, deep learning (DL) is a very promising field in big data applications. Remote sensing often involves huge data volumes obtained daily by numerous in-orbit satellites. This makes it a perfect target area for data-driven applications. Nowadays, technological advances in terms of software and hardware have a noticeable impact on Earth observation applications, more specifically in remote sensing techniques and procedures, allowing for the acquisition of data sets with greater quality at higher acquisition ratios. This results in the collection of huge amounts of remotely sensed data, characterized by their large spatial resolution (in terms of the number of pixels per scene), and very high spectral dimensionality, with hundreds or even thousands of spectral bands. As a result, remote sensing instruments on spaceborne and airborne platforms are now generating data cubes with extremely high dimensionality, imposing several restrictions in terms of both processing runtimes and storage capacity. In this article, we provide a comprehensive review of the state of the art in DL for remote sensing data interpretation, analyzing the strengths and weaknesses of the most widely used techniques in the literature, as well as an exhaustive description of their parallel and distributed implementations (with a particular focus on those conducted using cloud computing systems). We also provide quantitative results, offering an assessment of a DL technique in a specific case study (source code available: https://github.com/mhaut/cloud-dnn-HSI). This article concludes with some remarks and hints about future challenges in the application of DL techniques to distributed remote sensing data interpretation problems. We emphasize the role of the cloud in providing a powerful architecture that is now able to manage vast amounts of remotely sensed data due to its implementation simplicity, low cost, and high efficiency compared to other parallel and distributed architectures, such as grid computing or dedicated clusters.Publicación A Comprehensive Survey of Imbalance Correction Techniques for Hyperspectral Data Classification(IEEE, 2023) Paoletti, Mercedes Eugenia; Mogollón Gutiérrez, Óscar; Moreno Álvarez, Sergio; Sancho, José Carlos; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-2980-9236; https://orcid.org/0000-0002-4584-6945; https://orcid.org/0000-0001-6701-961XLand-cover classification is an important topic for remotely sensed hyperspectral (HS) data exploitation. In this regard, HS classifiers have to face important challenges, such as the high spectral redundancy, as well as noise, present in the data, and the fact that obtaining accurate labeled training data for supervised classification is expensive and time-consuming. As a result, the availability of large amounts of training samples, needed to alleviate the so-called Hughes phenomenon, is often unfeasible in practice. The class-imbalance problem, which results from the uneven distribution of labeled samples per class, is also a very challenging factor for HS classifiers. In this article, a comprehensive review of oversampling techniques is provided, which mitigate the aforementioned issues by generating new samples for the minority classes. More specifically, this article pursues a twofold objective. First, it reviews the most relevant oversampling methods that can be adopted according to the nature of HS data. Second, it provides a comprehensive experimental study and comparison, which are useful to derive practical conclusions about the performance of oversampling techniques in different HS image-based applications.Publicación Deep Attention-Driven HSI Scene Classification Based on Inverted Dot-Product(Institute of Electrical and Electronics Engineers Inc., 2022) Paoletti, Mercedes Eugenia; Tao, Xuanwen; han, lirong; Wu, Zhaoyue; Moreno Álvarez, Sergio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-1093-0079; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0002-6797-2440; https://orcid.org/0000-0001-6701-961XCapsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attentionPublicación Cloud Implementation of Extreme Learning Machine for Hyperspectral Image Classification(IEEE, 2023) Haut, Juan M.; Moreno Álvarez, Sergio; Moreno Ávila, Enrique; Ayma Quirita, Victor Andrés; Pastor Vargas, Rafael; Paoletti, Mercedes Eugenia; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0003-2987-2761; https://orcid.org/0000-0002-4089-9538; https://orcid.org/0000-0003-1030-3729Classifying remotely sensed hyperspectral images (HSIs) became a computationally demanding task given the extensive information contained throughout the spectral dimension. Furthermore, burgeoning data volumes compound inherent computational and storage challenges for data processing and classification purposes. Given their distributed processing capabilities, cloud environments have emerged as feasible solutions to handle these hurdles. This encourages the development of innovative distributed classification algorithms that take full advantage of the processing capabilities of such environments. Recently, computational-efficient methods have been implemented to boost network convergence by reducing the required training calculations. This letter develops a novel cloud-based distributed implementation of the extreme learning machine ( CC-ELM ) algorithm for efficient HSI classification. The proposal implements a fault-tolerant and scalable computing design while avoiding traditional batch-based backpropagation. CC-ELM has been evaluated over state-of-the-art HSI classification benchmarks, yielding promising results and proving the feasibility of cloud environments for large remote sensing and HSI data volumes processing. The code available at https://github.com/mhaut/scalable-ELM-HSI