Persona:
Moreno Álvarez, Sergio

Foto de perfil
Dirección de correo electrónico
ORCID
0000-0002-1858-9920
Fecha de nacimiento
Proyectos de investigación
Unidades organizativas
Puesto de trabajo
Apellidos
Moreno Álvarez
Nombre de pila
Sergio
Nombre

Filtros

2021 - 202415
Restablecer filtros

Ajustes

Autor: Haut, Juan M. × Tiene archivos: true ×

Resultados de la búsqueda

Mostrando 1 - 10 de 15
  • Miniatura
    Publicación
    Optimizing Distributed Deep Learning in Heterogeneous Computing Platforms for Remote Sensing Data Classification
    (IEEE, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Cavallaro, Gabriele; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0002-3239-9904; https://orcid.org/0000-0001-6701-961X
    Applications from Remote Sensing (RS) unveiled unique challenges to Deep Learning (DL) due to the high volume and complexity of their data. On the one hand, deep neural network architectures have the capability to automatically ex-tract informative features from RS data. On the other hand, these models have massive amounts of tunable parameters, re-quiring high computational capabilities. Distributed DL with data parallelism on High-Performance Computing (HPC) sys-tems have proved necessary in dealing with the demands of DL models. Nevertheless, a single HPC system can be al-ready highly heterogeneous and include different computing resources with uneven processing power. In this context, a standard data parallelism strategy does not partition the data efficiently according to the available computing resources. This paper proposes an alternative approach to compute the gradient, which guarantees that the contribution to the gradi-ent calculation is proportional to the processing speed of each DL model's replica. The experimental results are obtained in a heterogeneous HPC system with RS data and demon-strate that the proposed approach provides a significant training speed up and gain in the global accuracy compared to one of the state-of-the-art distributed DL framework.
  • No hay miniatura disponible
    Publicación
    Multiple Attention-Guided Capsule Networks for Hyperspectral Image Classification
    (IEEE, 2022) Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X
    The profound impact of deep learning and particularly of convolutional neural networks (CNNs) in automatic image processing has been decisive for the progress and evolution of remote sensing (RS) hyperspectral imaging (HSI) processing. Indeed, CNNs have stated themselves as the current state of the art, reaching unparalleled results in HSI classification. However, most CNNs were designed for RGB images, and their direct application to HSI data analysis could lead to nonoptimal solutions. Moreover, CNNs perform classification based on the identification of specific features, neglecting the spatial relationships between different features (i.e., their arrangement) due to pooling techniques. The capsule network (CapsNet) architecture is an attempt to overcome this drawback by nesting several neural layers within a capsule, connected by dynamic routing, both to identify not only the presence of a feature but also its instantiation parameters and to learn the relationships between different features. Although this mechanism improves the data representations, enhancing the classification of HSI data, it still acts as a black box, without control of the most relevant features for classification purposes. Indeed, important features could be discriminated against. In this article, a new multiple attention-guided CapsNet is proposed to improve feature processing for RS-HSIs’ classification, both to improve computational efficiency (in terms of parameters) and increase accuracy. Hence, the most representative visual parts of the images are identified using a detailed feature extractor coupled with attention mechanisms. Extensive experimental results have been obtained on five real datasets, demonstrating the great potential of the proposed method compared to other state-of-the-art classifiers.
  • No hay miniatura disponible
    Publicación
    Remote Sensing Image Classification Using CNNs With Balanced Gradient for Distributed Heterogeneous Computing
    (IEEE, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Cavallaro, Gabriele; Rico Gallego, Juan Antonio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-3239-9904; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0001-6701-961X
    Land-cover classification methods are based on the processing of large image volumes to accurately extract representative features. Particularly, convolutional models provide notable characterization properties for image classification tasks. Distributed learning mechanisms on high-performance computing platforms have been proposed to speed up the processing, while achieving an efficient feature extraction. High-performance computing platforms are commonly composed of a combination of central processing units (CPUs) and graphics processing units (GPUs) with different computational capabilities. As a result, current homogeneous workload distribution techniques for deep learning (DL) become obsolete due to their inefficient use of computational resources. To address this, new computational balancing proposals, such as heterogeneous data parallelism, have been implemented. Nevertheless, these techniques should be improved to handle the peculiarities of working with heterogeneous data workloads in the training of distributed DL models. The objective of handling heterogeneous workloads for current platforms motivates the development of this work. This letter proposes an innovative heterogeneous gradient calculation applied to land-cover classification tasks through convolutional models, considering the data amount assigned to each device in the platform while maintaining the acceleration. Extensive experimentation has been conducted on multiple datasets, considering different deep models on heterogeneous platforms to demonstrate the performance of the proposed methodology.
  • Miniatura
    Publicación
    AAtt-CNN: Automatic Attention-Based Convolutional Neural Networks for Hyperspectral Image Classification
    (IEEE, 2023) Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; xue, yu; Haut, Juan M.; Plaza, Antonio; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-9069-7547; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-9613-1659
    Convolutional models have provided outstanding performance in the analysis of hyperspectral images (HSIs). These architectures are carefully designed to extract intricate information from nonlinear features for classification tasks. Notwithstanding their results, model architectures are manually engineered and further optimized for generalized feature extraction. In general terms, deep architectures are time-consuming for complex scenarios, since they require fine-tuning. Neural architecture search (NAS) has emerged as a suitable approach to tackle this shortcoming. In parallel, modern attention-based methods have boosted the recognition of sophisticated features. The search for optimal neural architectures combined with attention procedures motivates the development of this work. This article develops a new method to automatically design and optimize convolutional neural networks (CNNs) for HSI classification using channel-based attention mechanisms. Specifically, 1-D and spectral–spatial (3-D) classifiers are considered to handle the large amount of information contained in HSIs from different perspectives. Furthermore, the proposed automatic attention-based CNN ( AAtt-CNN ) method meets the requirement to lower the large computational overheads associated with architectural search. It is compared with current state-of-the-art (SOTA) classifiers. Our experiments, conducted using a wide range of HSI images, demonstrate that AAtt-CNN succeeds in finding optimal architectures for classification, leading to SOTA results.
  • Miniatura
    Publicación
    Heterogeneous gradient computing optimization for scalable deep neural networks
    (Springer, 2022) Moreno Álvarez, Sergio; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473; https://orcid.org/0000-0001-6701-961X
    Nowadays, data processing applications based on neural networks cope with the growth in the amount of data to be processed and with the increase in both the depth and complexity of the neural networks architectures, and hence in the number of parameters to be learned. High-performance computing platforms are provided with fast computing resources, including multi-core processors and graphical processing units, to manage such computational burden of deep neural network applications. A common optimization technique is to distribute the workload between the processes deployed on the resources of the platform. This approach is known as data-parallelism. Each process, known as replica, trains its own copy of the model on a disjoint data partition. Nevertheless, the heterogeneity of the computational resources composing the platform requires to unevenly distribute the workload between the replicas according to its computational capabilities, to optimize the overall execution performance. Since the amount of data to be processed is different in each replica, the influence of the gradients computed by the replicas in the global parameter updating should be different. This work proposes a modification of the gradient computation method that considers the different speeds of the replicas, and hence, its amount of data assigned. The experimental results have been conducted on heterogeneous high-performance computing platforms for a wide range of models and datasets, showing an improvement in the final accuracy with respect to current techniques, with a comparable performance.
  • Miniatura
    Publicación
    Cloud Implementation of Extreme Learning Machine for Hyperspectral Image Classification
    (IEEE, 2023) Haut, Juan M.; Moreno Álvarez, Sergio; Moreno Ávila, Enrique; Ayma Quirita, Victor Andrés; Pastor Vargas, Rafael; Paoletti, Mercedes Eugenia; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0003-2987-2761; https://orcid.org/0000-0002-4089-9538; https://orcid.org/0000-0003-1030-3729
    Classifying remotely sensed hyperspectral images (HSIs) became a computationally demanding task given the extensive information contained throughout the spectral dimension. Furthermore, burgeoning data volumes compound inherent computational and storage challenges for data processing and classification purposes. Given their distributed processing capabilities, cloud environments have emerged as feasible solutions to handle these hurdles. This encourages the development of innovative distributed classification algorithms that take full advantage of the processing capabilities of such environments. Recently, computational-efficient methods have been implemented to boost network convergence by reducing the required training calculations. This letter develops a novel cloud-based distributed implementation of the extreme learning machine ( CC-ELM ) algorithm for efficient HSI classification. The proposal implements a fault-tolerant and scalable computing design while avoiding traditional batch-based backpropagation. CC-ELM has been evaluated over state-of-the-art HSI classification benchmarks, yielding promising results and proving the feasibility of cloud environments for large remote sensing and HSI data volumes processing. The code available at https://github.com/mhaut/scalable-ELM-HSI
  • Miniatura
    Publicación
    Heterogeneous model parallelism for deep neural networks
    (ELSEVIER, 2021-06-21) Moreno Álvarez, Sergio; Haut, Juan M.; Paoletti, Mercedes Eugenia; Rico Gallego, Juan Antonio; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0002-4264-7473
    Deep neural networks (DNNs) have transformed computer vision, establishing themselves as the current state-of-the-art for image processing. Nevertheless, the training of current large DNN models is one of the main challenges to be solved. In this sense, data-parallelism has been the most widespread distributed training strategy since it is easy to program and can be applied to almost all cases. However, this solution suffers from several limitations, such as its high communication requirements and the memory constraints when training very large models. To overcome these limitations model-parallelism has been proposed, solving the most substantial problems of the former strategy. However, describing and implementing the parallelization of the training of a DNN model across a set of processes deployed on several devices is a challenging task. Current proposed solutions assume a homogeneous distribution, being impractical when working with devices of different computational capabilities, which is quite common on high performance computing platforms. To address previous shortcomings, this work proposes a novel model-parallelism technique considering heterogeneous platforms, where a load balancing mechanism between uneven devices of an HPC platform has been implemented. Our proposal takes advantage of the Google Brain’s Mesh-TensorFlow for convolutional networks, splitting computing tensors across filter dimension in order to balance the computational load of the available devices. Conducted experiments show an improvement in the exploitation of heterogeneous computational resources, enhancing the training performance. The code is available on: https://github.com/mhaut/HeterogeneusModelDNN.
  • Miniatura
    Publicación
    A Comprehensive Survey of Imbalance Correction Techniques for Hyperspectral Data Classification
    (IEEE, 2023) Paoletti, Mercedes Eugenia; Mogollón Gutiérrez, Óscar; Moreno Álvarez, Sergio; Sancho, José Carlos; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-2980-9236; https://orcid.org/0000-0002-4584-6945; https://orcid.org/0000-0001-6701-961X
    Land-cover classification is an important topic for remotely sensed hyperspectral (HS) data exploitation. In this regard, HS classifiers have to face important challenges, such as the high spectral redundancy, as well as noise, present in the data, and the fact that obtaining accurate labeled training data for supervised classification is expensive and time-consuming. As a result, the availability of large amounts of training samples, needed to alleviate the so-called Hughes phenomenon, is often unfeasible in practice. The class-imbalance problem, which results from the uneven distribution of labeled samples per class, is also a very challenging factor for HS classifiers. In this article, a comprehensive review of oversampling techniques is provided, which mitigate the aforementioned issues by generating new samples for the minority classes. More specifically, this article pursues a twofold objective. First, it reviews the most relevant oversampling methods that can be adopted according to the nature of HS data. Second, it provides a comprehensive experimental study and comparison, which are useful to derive practical conclusions about the performance of oversampling techniques in different HS image-based applications.
  • Miniatura
    Publicación
    Deep Attention-Driven HSI Scene Classification Based on Inverted Dot-Product
    (Institute of Electrical and Electronics Engineers Inc., 2022) Paoletti, Mercedes Eugenia; Tao, Xuanwen; han, lirong; Wu, Zhaoyue; Moreno Álvarez, Sergio; Haut, Juan M.; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0003-1093-0079; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0002-6797-2440; https://orcid.org/0000-0001-6701-961X
    Capsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attention
  • No hay miniatura disponible
    Publicación
    Deep shared proxy construction hashing for cross-modal remote sensing image fast target retrieval
    (ELSEVIER, 2024) han, lirong; Paoletti, Mercedes Eugenia; Moreno Álvarez, Sergio; Haut, Juan M.; Plaza, Antonio; https://orcid.org/0000-0002-8613-7037; https://orcid.org/0000-0003-1030-3729; https://orcid.org/0000-0001-6701-961X; https://orcid.org/0000-0002-9613-1659
    The diversity of remote sensing (RS) image modalities has expanded alongside advancements in RS technologies. A plethora of optical, multispectral, and hyperspectral RS images offer rich geographic class information. The ability to swiftly access multiple RS image modalities is crucial for fully harnessing the potential of RS imagery. In this work, an innovative method, called Deep Shared Proxy Construction Hashing (DSPCH), is introduced for cross-modal hyperspectral scene target retrieval using accessible RS images such as optical and sketch. Initially, a shared proxy hash code is generated in the hash space for each land use class. Subsequently, an end-to-end deep hash network is built to generate hash codes for hyperspectral pixels and accessible RS images. Furthermore, a proxy hash loss function is designed to optimize the proposed deep hashing network, aiming to generate hash codes that closely resemble the corresponding proxy hash code. Finally, two benchmark datasets are established for cross-modal hyperspectral and accessible RS image retrieval, allowing us to conduct extensive experiments with these datasets. Our experimental results validate that the novel DSPCH method can efficiently and effectively achieve RS image cross-modal target retrieval, opening up new avenues in the field of cross-modal RS image retrieval