Robust optimal strategies in Markov decision problems

An optimal strategy in a Markov decision problem is robust if it is optimal in every decision problem (not necessarily stationary) that is close to the original problem. We prove that when the state and action spaces are finite, an optimal strategy is robust if and only if it is the unique optimal strategy.     

Strong controllability and optimal control of the heat equation with a thermal source

In this paper we consider an optimal control system described byn-dimensional heat equation with a thermal source. Thus problem is to find an optimal control which puts the system in a finite time T, into a stationary regime and to minimize a general objective function. Here we assume there is no constraints on control. This problem is reduced to a moment problem.We modify the moment problem into one consisting of the minimization of a positive linear functional over a set of Radon measures and we show that there is an optimal measure corresponding to the optimal control. The above optimal measure approximated by a finite combination of atomic measures. This construction gives rise to a finite dimensional linear programming problem, where its solution can be used to determine the optimal combination of atomic measures. Then by using the solution of the above linear programming problem we find a piecewise-constant optimal control function which is an approximate control for the original optimal control problem. Finally we obtain piecewise-constant optimal control for two examples of heat equations with a thermal source in one-dimensional.     

Time optimal sampled-data controls for the heat equation

In this paper, we first design a time optimal control problem for the heat equation with sampled-data controls, and then use it to approximate a time optimal control problem for the heat equation with distributed controls.The study of such a time optimal sampled-data control problem is not easy, because it may have infinitely many optimal controls. We find connections among this problem, a minimal norm sampled-data control problem and a minimization problem, and obtain some properties on these problems. Based on these, we not only build up error estimates for optimal time and optimal controls between the time optimal sampled-data control problem and the time optimal distributed control problem, in terms of the sampling period, but we also prove that such estimates are optimal in some sense.     

Optimal singular and chattering modes in the problem of controlling the vibrations of a string with clamped ends

The problem of minimizing the root mean square deviation of a uniform string with clamped ends from an equilibrium position is investigated. It is assumed that the initial conditions are specified and the ends of the string are clamped. The Fourier method is used, which enables the control problem with a partial differential equation to be reduced to a control problem with a denumerable system of ordinary differential equations. For the optimal control problem in the l₂ space obtained, it is proved that the optimal synthesis contains singular trajectories and chattering trajectories. For the initial problem of the optimal control of the vibrations of a string it is also proved that there is a unique solution for which the optimal control has a denumerable number of switchings in a finite time interval.     

The inverse optimal value problem

This paper considers the following inverse optimization problem: given a linear program, a desired optimal objective value, and a set of feasible cost vectors, determine a cost vector such that the corresponding optimal objective value of the linear program is closest to the desired value. The above problem, referred here as the inverse optimal value problem, is significantly different from standard inverse optimization problems that involve determining a cost vector for a linear program such that a pre-specified solution vector is optimal. In this paper, we show that the inverse optimal value problem is NP-hard in general. We identify conditions under which the problem reduces to a concave maximization or a concave minimization problem. We provide sufficient conditions under which the associated concave minimization problem and, correspondingly, the inverse optimal value problem is polynomially solvable. For the case when the set of feasible cost vectors is polyhedral, we describe an algorithm for the inverse optimal value problem based on solving linear and bilinear programming problems. Some preliminary computational experience is reported.Mathematics Subject Classification (1999):49N45, 90C05, 90C25, 90C26, 90C31, 90C60Acknowledgement This research has been supported in part by the National Science Foundation under CAREER Award DMII-0133943. The authors thank two anonymous reviewers for valuable comments.     

Deterministic Time-inconsistent Optimal Control Problems - an Essentially Cooperative Approach

A general deterministic time-inconsistent optimal control problem is formulated for ordinary differential equations. To find a time-consistent equilibrium value function and the corresponding time-consistent equilibrium control, a non-cooperative N-person differential game (but essentially cooperative in some sense) is introduced. Under certain conditions, it is proved that the open-loop Nash equilibrium value function of the N -person differential game converges to a time-consistent equilibrium value function of the original problem, which is the value function of a time-consistent optimal control problem. Moreover, it is proved that any optimal control of the time-consistent limit problem is a time-consistent equilibrium control of the original problem.     

Optimality conditions for a class of inverse optimal control problems with partial differential equations

ABSTRACT

We consider bilevel optimization problems which can be interpreted as inverse optimal control problems. The lower-level problem is an optimal control problem with a parametrized objective function. The upper-level problem is used to identify the parameters of the lower-level problem. Our main focus is the derivation of first-order necessary optimality conditions. We prove C-stationarity of local solutions of the inverse optimal control problem and give a counterexample to show that strong stationarity might be violated at a local minimizer.     

Optimal assignment of NOP due-dates and sequencing in a single machine shop

We consider the problem of optimal assignment of NOP due-dates ton jobs and sequencing them on a single machine to minimize a penalty function depending on the values of assigned constant waiting allowance and maximum job tardiness. It is shown that the earliest due date (EDD) order is an optimal sequence. For finding optimal constant waiting allowance, we reduce the problem to a multiple objective piecewise linear programming with single variable. An efficient algorithm for optimal solution of the problem is given.     

A Lagrangian search method for the <Emphasis Type="Italic">P</Emphasis>-median problem

In this paper, we propose a novel algorithm for solving the classical P-median problem. The essential aim is to identify the optimal extended Lagrangian multipliers corresponding to the optimal solution of the underlying problem. For this, we first explore the structure of the data matrix in P-median problem to recast it as another equivalent global optimization problem over the space of the extended Lagrangian multipliers. Then we present a stochastic search algorithm to find the extended Lagrangian multipliers corresponding to the optimal solution of the original P-median problem. Numerical experiments illustrate that the proposed algorithm can effectively find a global optimal or very good suboptimal solution to the underlying P-median problem, especially for the computationally challenging subclass of P-median problems with a large gap between the optimal solution of the original problem and that of its Lagrangian relaxation.     

A general method for finding the optimal threshold in discrete time

We develop an approach for solving one-sided optimal stopping problems in discrete time for general underlying Markov processes on the real line. The main idea is to transform the problem into an auxiliary problem for the ladder height variables. In case that the original problem has a one-sided solution and the auxiliary problem has a monotone structure, the corresponding myopic stopping time is optimal for the original problem as well. This elementary line of argument directly leads to a characterization of the optimal boundary in the original problem. The optimal threshold is given by the threshold of the myopic stopping time in the auxiliary problem. Supplying also a sufficient condition for our approach to work, we obtain solutions for many prominent examples in the literature, among others the problems of Novikov-Shiryaev, Shepp-Shiryaev, and the American put in option pricing under general conditions. As a further application we show that for underlying random walks (and Lévy processes in continuous time), general monotone and log-concave reward functions g lead to one-sided stopping problems.     

Real-Time Solution of Bi-Level Optimal Control Problems

Bi-level optimal control problems are presented as an extension to classical optimal control problems. Hereby, additional constraints for the primary problem are considered, which depend on the optimal solution of a secondary optimal control problem. A demanding problem is the numerical complexity, since at any point in time the solution of the optimal control problem as well as a complete solution of the secondary problem have to be determined. Hence we deal with two dependent variables in time. The numerical solution of the bi-level problem is illustrated by an application of a container crane. Jerk and energy optimal trajectories with free final time are calculated under the terminal condition that the crane system comes to be at rest at a predefined location. In enlargement additional constraints are investigated to ensure that the crane system can be brought to a rest position by a safety stop at a free but admissible location in minimal time from any state of the trajectory. (© 2005 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A penalty function method for solving inverse optimal value problem

In order to consider the inverse optimal value problem under more general conditions, we transform the inverse optimal value problem into a corresponding nonlinear bilevel programming problem equivalently. Using the Kuhn–Tucker optimality condition of the lower level problem, we transform the nonlinear bilevel programming into a normal nonlinear programming. The complementary and slackness condition of the lower level problem is appended to the upper level objective with a penalty. Then we give via an exact penalty method an existence theorem of solutions and propose an algorithm for the inverse optimal value problem, also analysis the convergence of the proposed algorithm. The numerical result shows that the algorithm can solve a wider class of inverse optimal value problem.     

Optimal paths in bi-attribute networks with fractional cost functions

An important routing problem is to determine an optimal path through a multi-attribute network which minimizes a cost function of path attributes. In this paper, we study an optimal path problem in a bi-attribute network where the cost function for path evaluation is fractional. The problem can be equivalently formulated as the “bi-attribute rational path problem” which is known to be NP-complete. We develop an exact approach to find an optimal simple path through the network when arc attributes are non-negative. The approach uses some path preference structures and elimination techniques to discard, from further consideration, those (partial) paths that cannot be parts of an optimal path. Our extensive computational results demonstrate that the proposed method can find optimal paths for large networks in very attractive times.     

David G. Luenberger,Linear and nonlinear programming, 2nd edition,Addison-Wesley,Reading (1984), xv + 492 pages, $35.50

This paper considers a stochastic version of the linear continuous type knapsack problem in which the cost coefficients are random variables. The problem is to find an optimal solution and an optimal probability level of the chance constraint. This problem P₀ is first transformed into a deterministic equivalent problem P. Then a subproblem with a positive parameter is introduced and a close relation between P and its subproblem is shown. Further, an auxiliary problem of the subproblem is introduced and a direct relation between P and the auxiliary problem is derived through a relation connecting the subproblem and its auxiliary problem. Fully utilizing these relations, an efficient algorithm is proposed that finds an optimal solution of P in at most O(n⁴) computational time where n is the number of decision variables. Finally, further research problems are discussed.     

A variational problem determined by probability measures

ABSTRACT

An optimization problem of maximizing an integral of a function over a family of probability measures is considered. The problem is a generalization of a well-studied variational problem in mathematical economics, concerning optimal allocations. The specific generalization that we examine arises also in the limit of singularly perturbed optimal control problems. We examine the mathematical problem and allude to the singular perturbation motivation.     

Optimal XL-insurance under Wasserstein-type ambiguity

We study the problem of optimal insurance contract design for risk management under a budget constraint. The contract holder takes into consideration that the loss distribution is not entirely known and therefore faces an ambiguity problem. For a given set of models, we formulate a minimax optimization problem of finding an optimal insurance contract that minimizes the distortion risk functional of the retained loss with premium limitation. We demonstrate that under the average value-at-risk measure, the entrance-excess of loss contracts are optimal under ambiguity, and we solve the distributionally robust optimal contract-design problem. It is assumed that the insurance premium is calculated according to a given baseline loss distribution and that the ambiguity set of possible distributions forms a neighborhood of the baseline distribution. To this end, we introduce a contorted Wasserstein distance. This distance is finer in the tails of the distributions compared to the usual Wasserstein distance.     

On non-permutation solutions to some two machine flow shop scheduling problems

In this paper, we study two versions of the two machine flow shop scheduling problem, where schedule length is to be minimized. First, we consider the two machine flow shop with setup, processing, and removal times separated. It is shown that an optimal solution need not be a permutation schedule, and that the problem isNP-hard in the strong sense, which contradicts some known results. The tight worst-case bound for an optimal permutation solution in proportion to a global optimal solution is shown to be 3/2. An O(n) approximation algorithm with this bound is presented. Secondly, we consider the two machine flow shop with finite storage capacity. Again, it is shown that there may not exist an optimal solution that is a permutation schedule, and that the problem isNP-hard in the strong sense.     

Optimal multiple quantum statistical hypothesis testing

This paper is concerned with the problem of optimal M-alternative determination of quantum statistical states. A review of newest achievement of solving this problem is given. A notion of an effective decision Hilbert space is introduced and necessary and sufficient condkions for optimality of multiple quantum hypothesis testing in this space are formulated. The general solution is found for the case of a two-dimensional decision space. Another problem solved is that of discrimination of quantum pure non-orthogonal states. The result is represented in explicit analytical form for an "equidiagonal" case, which is quite general. In particular, we find explicit solutions of optimal discrimination problem of homogeneous and equiangle sets of pure states. These results are used for the M-ary detection problem in solving for the quantum coherent non-orthogonal signals. It is proved that the simplex signals are optimal elso in quantum case. The optimal estimatesof phaseandamplitude of quantum coherent signals are found. For decision operators a notion of IT-representation is introduced to get a general quasi-classical (optimal in quasi-classical limit) M-ary detection procedure of stochastic fields and particles, which submits to Bose-Einstein statistics. An optimal solution of problem of non-coherent detection of quantum stochastic (including optical) signals are found in the extreme quantum limit (weaknoise and signals with unknown phase).     

Finding preferred subsets of Pareto optimal solutions

Multi-objective optimization algorithms can generate large sets of Pareto optimal (non-dominated) solutions. Identifying the best solutions across a very large number of Pareto optimal solutions can be a challenge. Therefore it is useful for the decision-maker to be able to obtain a small set of preferred Pareto optimal solutions. This paper analyzes a discrete optimization problem introduced to obtain optimal subsets of solutions from large sets of Pareto optimal solutions. This discrete optimization problem is proven to be NP-hard. Two exact algorithms and five heuristics are presented to address this problem. Five test problems are used to compare the performances of these algorithms and heuristics. The results suggest that preferred subset of Pareto optimal solutions can be efficiently obtained using the heuristics, while for smaller problems, exact algorithms can be applied.     

Finite-horizon optimal investment with transaction costs: A parabolic double obstacle problem

This paper concerns optimal investment problem of a CRRA investor who faces proportional transaction costs and finite time horizon. From the angle of stochastic control, it is a singular control problem, whose value function is governed by a time-dependent HJB equation with gradient constraints. We reveal that the problem is equivalent to a parabolic double obstacle problem involving two free boundaries that correspond to the optimal buying and selling policies. This enables us to make use of the well-developed theory of obstacle problem to attack the problem. The C2,1 regularity of the value function is proven and the behaviors of the free boundaries are completely characterized.