Persona: Prieto Rumeau, Tomás
Cargando...
Dirección de correo electrónico
ORCID
0000-0003-4677-4725
Fecha de nacimiento
Proyectos de investigación
Unidades organizativas
Puesto de trabajo
Apellidos
Prieto Rumeau
Nombre de pila
Tomás
Nombre
8 resultados
Resultados de la búsqueda
Mostrando 1 - 8 de 8
Publicación Maximizing the probability of visiting a set infinitely often for a Markov decision process with Borel state and action spaces(Cambridge University Press, 2024) François Dufour; Prieto Rumeau, Tomás; https://orcid.org/0000-0002-8062-1346We consider a Markov control model with Borel state space, metric compact action space, and transitions assumed to have a density function with respect to some probability measure satisfying some continuity conditions. We study the optimization problem of maximizing the probability of visiting some subset of the state space infinitely often, and we show that there exists an optimal stationary Markov policy for this problem. We endow the set of stationary Markov policies and the family of strategic probability measures with adequate topologies (namely, the narrow topology for Young measures and the ws∞ -topology, respectively) to obtain compactness and continuity properties, which allow us to obtain our main results.Publicación Constrained Markov decision processes with non-constant discount factor(Springer Nature, 2024-05-30) Jasso Fuentes, Héctor; Prieto Rumeau, TomásThis paper studies constrained Markov decision processes under the total expected discounted cost optimality criterion, with a state-action dependent discount factor that may take any value between zero and one. Both the state and the action space are assumed to be Borel spaces. By using the linear programming approach, consisting in stating the control problem as a linear problem on a set of occupation measures, we show the existence of an optimal stationary Markov policy. Our results are based on the study of both weak-strong topologies in the space of occupation measures and Young measures in the space of Markov policiesPublicación Discrete-time control with non-constant discount factor(Springer Nature, 2020-06-20) Jasso Fuentes, Héctor; Menaldi, José Luis; Prieto Rumeau, TomásThis paper deals with discrete-time Markov decision processes (MDPs) with Borel state and action spaces, and total expected discounted cost optimality criterion. We assume that the discount factor is not constant: it may depend on the state and action; moreover, it can even take the extreme values zero or one. We propose sufficient conditions on the data of the model ensuring the existence of optimal control policies and allowing the characterization of the optimal value function as a solution to the dynamic programming equation. As a particular case of these MDPs with varying discount factor, we study MDPs with stopping, as well as the corresponding optimal stopping times and contact set. We show applications to switching MDPs models and, in particular, we study a pollution accumulation problem.Publicación Nash equilibria for total expected reward absorbing Markov games: The constrained and unconstrained cases(Springer Nature, 2024-01-17) Dufour, François; Prieto Rumeau, TomásWe consider a nonzero-sum N -player Markov game on an abstract measurable state space with compact metric action spaces. The payoff functions are bounded Carathéodory functions and the transitions of the system are assumed to have a density function satisfying some continuity conditions. The optimality criterion of the players is given by a total expected payoff on an infinite discrete-time horizon. Under the condition that the game model is absorbing, we establish the existence of Markov strategies that are a noncooperative equilibrium in the family of all history-dependent strategies of the players for both the constrained and the unconstrained problems. We obtain, as a particular case of results, the existence of Nash equilibria for discounted constrained and unconstrained game models.Publicación Discrete-time hybrid control in Borel spaces(Springer Nature, 2018-05-18) Jasso Fuentes, Héctor; Menaldi, José-Luis; Prieto Rumeau, TomásA discrete-time hybrid control model with Borel state and action spaces is introduced. In this type of models, the dynamic of the system is composed by two sub-dynamics affecting the evolution of the state; one is of a standard-type that runs almost every time and another is of a special-type that is active under special circumstances. The controller is able to use two different type of actions, each of them is applied to each of the two sub-dynamics, and the activations of these sub-dynamics are possible according to an activation rule that can be handled by the controller. The aim for the controller is to find a control policy, containing a mix of actions (of either standard- or special-type), with the purpose of minimizing an infinite-horizon discounted cost criterion whose discount factor is dependent on the state-action history and may be equal one at some stages. Two different sets of conditions are proposed to guarantee (i) the finiteness of the cost criterion, (ii) the characterization of the optimal value function and (iii) the existence of optimal control policies; to do so, we employ the dynamic programming approach. A useful characterization that signalizes the accurate times between changes of sub-dynamics in terms of the so-named contact set is also provided. Finally, we introduce two examples that illustrate our results and also show that control models such as discrete-time impulse control models and discrete-time switching control models become special cases of our present hybrid model.Publicación Stationary Markov Nash equilibria for Nonzero-Sum constrained ARAT Markov Games(Society for Industrial and Applied Mathematics, 2022) Dufour, François; Prieto Rumeau, TomásWe consider a nonzero-sum Markov game on an abstract measurable state space with compact metric action spaces. The goal of each player is to maximize his respective discounted payoff function under the condition that some constraints on a discounted payoff are satisfied. We are interested in the existence of a Nash or noncooperative equilibrium. Under suitable conditions, which include absolute continuity of the transitions with respect to some reference probability measure, additivity of the payoffs and the transition probabilities (ARAT condition), and continuity in action of the payoff functions and the density function of the transitions of the system, we establish the existence of a constrained stationary Markov Nash equilibrium, that is, the existence of stationary Markov strategies for each of the players yielding an optimal profile within the class of all history-dependent profiles.Publicación Absorbing Markov decision processes(EDP Sciences, 2024-02-09) Dufour, François; Prieto Rumeau, TomásIn this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not occupation measures may exist. Several necessary and sufficient conditions are provided to guarantee that any solution to the characteristic equation is an occupation measure. Under the so-called continuity-compactness conditions, we first show that a measure is precisely an occupation measure if and only if it satisfies the characteristic equation and an additional absolute continuity condition. Secondly, it is shown that the set of occupation measures is compact in the weak-strong topology if and only if the model is uniformly absorbing. Several examples are provided to illustrate our results.Publicación Maximizing the probability of visiting a set infinitely often for a countable state space Markov decision process(Elsevier, 2022-01-15) Dufour, François; Prieto Rumeau, TomásWe consider a Markov decision process with countable state space and Borel action space. We are interested in maximizing the probability that the controlled Markov chain visits some subset of the state space infinitely often. We provide sufficient conditions, based on continuity and compactness requirements, together with a stability condition on a parametrized family of auxiliary control models, which imply the existence of an optimal policy that is deterministic and stationary. We compare our hypotheses with those existing in the literature.