Asymptotic properties of constrained Markov Decision Processes

We present in this paper several asymptotic properties of constrained Markov Decision Processes (MDPs) with a countable state space. We treat both the discounted and the expected average cost, with unbounded cost. We are interested in (1) the convergence of finite horizon MDPs to the infinite horizon MDP, (2) convergence of MDPs with a truncated state space to the problem with infinite state space, (3) convergence of MDPs as the discount factor goes to a limit. In all these cases we establish the convergence of optimal values and policies. Moreover, based on the optimal policy for the limiting problem, we construct policies which are almost optimal for the other (approximating) problems. Based on the convergence of MDPs with a truncated state space to the problem with infinite state space, we show that an optimal stationary policy exists such that the number of randomisations it uses is less or equal to the number of constraints plus one. We finally apply the results to a dynamic scheduling problem.This work was partially supported by the Chateaubriand fellowship from the French embassy in Israel and by the European Grant BRA-QMIPS of CEC DG XIII     

Global Convergence of Algorithms with Nonmonotone Line Search Strategy in Unconstrained Optimization

In this paper we state some nonmonotone line search strategies for unconstrained optimization algorithms. Abstracting from the concrete line search strategy we prove two general convergence results. Using this theory we can show the global convergence of the BFGS method with nonmonotone line search strategy. In contrast to some former results about nonmonotone line search strategies, both our convergence results and their proofs are natural generalizations of known results for the monotone case.     

On the convergence of finite state mean-field games through Γ-convergence

In this study, we consider the long-term convergence (trend toward an equilibrium) of finite state mean-field games using Γ-convergence. Our techniques are based on the observation that an important class of mean-field games can be viewed as the Euler–Lagrange equation of a suitable functional. Therefore, using a scaling argument, one can convert a long-term convergence problem into a Γ-convergence problem. Our results generalize previous results related to long-term convergence for finite state problems.     

Globalizing a nonsmooth Newton method via nonmonotone path search

We give a framework for the globalization of a nonsmooth Newton method. In part one we start with recalling B. Kummer’s approach to convergence analysis of a nonsmooth Newton method and state his results for local convergence. In part two we give a globalized version of this method. Our approach uses a path search idea to control the descent. After elaborating the single steps, we analyze and prove the global convergence resp. the local superlinear or quadratic convergence of the algorithm. In the third part we illustrate the method for nonlinear complementarity problems.     

A priori error estimates for optimal control problems with pointwise constraints on the gradient of the state

We analyze a finite element approximation of an elliptic optimal control problem with pointwise bounds on the gradient of the state variable. We derive convergence rates if the control space is discretized implicitly by the state equation. In contrast to prior work we obtain these results directly from classical results for the W ^1,∞-error of the finite element projection, without using adjoint information. If the control space is discretized directly, we first prove a regularity result for the optimal control to control the approximation error, based on which we then obtain analogous convergence rates.     

Optimal control in coefficients of elliptic equations with state constraints

This paper is concerned with state constrained optimal control problems of elliptic equations, the control being a coefficient of the partial differential equation. Existence of an optimal control is proved and optimality conditions are derived. We perform finite-element approximations of optimal control problems and state some convergence results: we prove convergence of optimal controls and states as well as convergence of Lagrange multipliers.This research was partially supported by the Dirección General de Investigación Científica y Técnica (Madrid).     

A combination of energy method and spectral analysis for study of equations of gas motion

There have been extensive studies on the large time behavior of solutions to systems on gas motions, such as the Navier-Stokes equations and the Boltzmann equation. Recently, an approach is introduced by combining the energy method and the spectral analysis to the study of the optimal rates of convergence to the asymptotic profiles. In this paper, we will first illustrate this method by using some simple model and then we will present some recent results on the Navier-Stokes equations and the Boltzmann equation. Precisely, we prove the stability of the non-trivial steady state for the Navier-Stokes equations with potential forces and also obtain the optimal rate of convergence of solutions toward the steady state. The same issue was also studied for the Boltzmann equation in the presence of the general time-space dependent forces. It is expected that this approach can also be applied to other dissipative systems in fluid dynamics and kinetic models such as the model system of radiating gas and the Vlasov-Poisson-Boltzmann system.      

Exact bound for the convergence of metropolis chains

In this note, we present a calculation which gives us the exact bound for the convergence of Metropolis chains in a finite state space and therefore improves the existing results which are only for the upper bounds of such convergence (see the references below). Our result is based on an interesting observation on the transition probability of Metropolis chains     

Domain branching in uniaxial ferromagnets: asymptotic behavior of the energy

In this article we analyze the ground state energy of a ferromagnetic bulk sample with strong uniaxial anisotropy in a regime featuring domain branching. We derive the convergence of the micromagnetic energy towards a sharp interface energy in the spirit of Γ-limits. Then we use this convergence to rigorously justify the notion of a minimal energy per cross-section area. Compared to known results, the scaling bounds for the minimal energy are improved to an asymptotic equality.     

Convergence of analytic gradient-type systems with periodicity and its applications in Kuramoto models

We consider the convergence of gradient-type systems with periodic and analytic potentials. The main tool is the celebrated Łojasiewicz inequality which is valid for any analytic function. Our results show that the convergence of such systems with periodic and analytic potentials is unconditional to the initial data; in other words, any trajectory converges to some equilibrium. As direct applications, we can show that any trajectory converges to phase-locked state for the first- and second-order Kuramoto models on a symmetric network with attractive–repulsive forces and identical natural frequencies. In particular, the inertial Kuramoto model with identical oscillators converges to phase-locked state for any initial configuration.     

Automatic differentiation of explicit Runge-Kutta methods for optimal control

This paper considers the numerical solution of optimal control problems based on ODEs. We assume that an explicit Runge-Kutta method is applied to integrate the state equation in the context of a recursive discretization approach. To compute the gradient of the cost function, one may employ Automatic Differentiation (AD). This paper presents the integration schemes that are automatically generated when differentiating the discretization of the state equation using AD. We show that they can be seen as discretization methods for the sensitivity and adjoint differential equation of the underlying control problem. Furthermore, we prove that the convergence rate of the scheme automatically derived for the sensitivity equation coincides with the convergence rate of the integration scheme for the state equation. Under mild additional assumptions on the coefficients of the integration scheme for the state equation, we show a similar result for the scheme automatically derived for the adjoint equation. Numerical results illustrate the presented theoretical results.     

Enhanced Euler's method to a free boundary porous media flow problem

We study an ODE‐based iterative method, the residual velocity method, for steady state free boundary problems. The convergence analysis of the method, as well as the numerical implementation based on Euler's method were provided by Donaldson and Wetton (J Appl Math 71 (2006), 877–897). In this article, we develop an enhanced Euler's method which is nearly as simple as the modified Euler's method but can achieve a rapid convergence rate similar to the fourth‐order Runge‐Kutta method. Numerical results are also provided to verify the validity of our method. © 2011 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 2012     

On Hashin–Shtrikman bounds for mixtures of two isotropic materials

In the context of stationary diffusion equation we calculate explicitly the optimal microstructure for the Hashin–Shtrikman energy bound in the case of two isotropic phases with prescribed ratio, in three dimensions. A similar, but more general problem arises in the study of optimal design in conductivity with multiple state equations. Here, the necessary condition of optimality leads to a finite-dimensional optimisation problem which extends the problem of Hashin–Shtrikman bounds, which can be solved explicitly, as well.These calculations have important applications to the optimality criteria method for numerical solution of optimal design problems with multiple state equations. In this iterative algorithm, the presented results enable one to calculate explicitly the update of design variables, similar to the problems with one state equation. Therefore, its implementation is simple, showing nice convergence results on a number of examples, two of them being demonstrated here.     

Convergence and quasi-optimality of an adaptive finite element method for optimal control problems with integral control constraint

In this paper we study the convergence of an adaptive finite element method for optimal control problems with integral control constraint. For discretization, we use piecewise constant discretization for the control and continuous piecewise linear discretization for the state and the co-state. The contraction, between two consecutive loops, is proved. Additionally, we find the adaptive finite element method has the optimal convergence rate. In the end, we give some examples to support our theoretical analysis.     

Optimality conditions with state constraints for semilinear systems determined by operator valued measures

In this paper we develop the necessary conditions of optimality for a class of distributed parameter systems (partial differential equations) determined by operator valued measures and controlled by vector measures. Based on some recent results on existence of optimal controls from the space of vector measures, we develop necessary conditions of optimality for a class of control problems. The main results are the necessary conditions of optimality for problems without state constraints and those with state constraints. Also, a conceptual algorithm along with a brief discussion of its convergence is presented.     

Vitesse de convergence vers l'état d'équilibre pour des dynamiques markoviennes non höldériennes

We study the convergence speed to equilibrium state for Markovian non hölderian dynamics. In particular, an estimation of the mixing speed is obtained on a subspace B which is dense in the space of continuous functions. Moreover, we show that the spectrum of the Perron-Frobenius operator as acting on B is a whole elosed disk of which each point is an eigenvalue. This implies that the convergence speed cannot be exponential.     

Online Surrogate Problem Methodology for Stochastic Discrete Resource Allocation Problems

We consider stochastic discrete optimization problems where the decision variables are nonnegative integers. We propose and analyze an online control scheme which transforms the problem into a surrogate continuous optimization problem and proceeds to solve the latter using standard gradient-based approaches, while simultaneously updating both the actual and surrogate system states. It is shown that the solution of the original problem is recovered as an element of the discrete state neighborhood of the optimal surrogate state. For the special case of separable cost functions, we show that this methodology becomes particularly efficient. Finally, convergence of the proposed algorithm is established under standard technical conditions; numerical results are included in the paper to illustrate the fast convergence of this approach.     

Rate of convergence estimates for the spectral approximation of a generalized eigenvalue problem

Summary. The aim of this work is to derive rate of convergence estimates for the spectral approximation of a mathematical model which describes the vibrations of a solid-fluid type structure. First, we summarize the main theoretical results and the discretization of this variational eigenvalue problem. Then, we state some well known abstract theorems on spectral approximation and apply them to our specific problem, which allow us to obtain the desired spectral convergence. By using classical regularity results, we are able to establish estimates for the rate of convergence of the approximated eigenvalues and for the gap between generalized eigenspaces. Received February 6, 1996 / Revised version received November 28, 1996     

Optimal error estimates for finite element discretization of elliptic optimal control problems with finitely many pointwise state constraints

In this paper we consider a model elliptic optimal control problem with finitely many state constraints in two and three dimensions. Such problems are challenging due to low regularity of the adjoint variable. For the discretization of the problem we consider continuous linear elements on quasi-uniform and graded meshes separately. Our main result establishes optimal a priori error estimates for the state, adjoint, and the Lagrange multiplier on the two types of meshes. In particular, in three dimensions the optimal second order convergence rate for all three variables is possible only on properly refined meshes. Numerical examples at the end of the paper support our theoretical results.     

Hybrid Projection Algorithm for Two Finite Families of Asymptotically Quasi ϕ-Nonexpansive Mappings in Reflexive Banach Spaces

In this paper, we introduce a hybrid projection algorithm for two finite families of asymptotically quasi ?-nonexpansive mappings to establish a convergence theorem in smooth, strictly convex, and reflexive Banach spaces. As applications, we state the convergence of hybrid projection algorithm for two finite families of asymptotically quasi-nonexpansive mappings in Hilbert spaces and the convergence for a system of equilibrium problems.