Generalized solution in singular stochastic control: The nondegenerate problem

This paper is concerned with singular stochastic control for non-degenerate problems. It generalizes the previous work in that the model equation is nonlinear and the cost function need not be convex. The associated dynamic programming equation takes the form of variational inequalities. By combining the principle of dynamic programming and the method of penalization, we show that the value function is characterized as a unique generalized (Sobolev) solution which satisfies the dynamic programming variational inequality in the almost everywhere sense. The approximation for our singular control problem is given in terms of a family of penalized control problems. As a result of such a penalization, we obtain that the value function is also the minimum cost available when only the admissible pairs with uniformly Lipschitz controls are admitted in our cost criterion.     

Convex duality for finite-fuel problems in singular stochastic control

Upon introducing a finite-fuel constraint in a stochastic control system, the convex duality formulation can be set up to represent the original singular control problem as a minimization problem over the space of vector measures at each level of available fuel. This minimization problem is imbedded tightly into a related weak problem, which is actually a mathematical programming problem over a convex,w*-compact space of vector-valued Radon measures. Then, through the Fenchel duality principle, the dual for the finite-fuel control problems is to seek the maximum of smooth subsolutions to a dynamic programming variational inequality. The approach is basically in the spirit of Fleming and Vermes, and the results of this paper extend those of Vinter and Lewis in deterministic control problems to the finite-fuel problems in singular stochastic control. Meanwhile, we also obtain the characterization of the value function as a solution to the dynamic programming variational inequality in the sense of the Schwartz distribution.The author is much indebted to Professor Wendell H. Fleming for his constant support and many helpful discussions during the preparation of this paper.     

Robust finite-time H∞ control of singular stochastic systems via static output feedback

This paper addresses the problem of robust finite-time stabilization of singular stochastic systems via static output feedback. Firstly, sufficient conditions of singular stochastic finite-time boundedness on static output feedback are obtained for the family of singular stochastic systems with parametric uncertainties and time-varying norm-bounded disturbance. Then the results are extended to singular stochastic H_∞ finite-time boundedness for the class of singular stochastic systems. Designed algorithm for static output feedback controller is provided to guarantee that the underlying closed-loop singular stochastic system is singular stochastic H_∞ finite-time boundedness in terms of strict linear matrix equalities with a fixed parameter. Finally, an illustrative example is presented to show the validity of the developed methodology.     

Singular stochastic control of a singular diffusion process

This paper studies the monotone follower problem for a one-dimensional singular diffusion process. The dynamic programming principle is established. It is shown that the value function is continuous and satisfies the Hamilton-Jacobi-Bellman equation in the viscosity sense     

Numerical methods for singular optimal control

Methods are described for the numerical solution of singular optimal control problems. A simple method is given for solving a class of problems which form a transition from nonsingular to singular cases. A procedure is given for determining the structure of a singular problem if it is initially unknown. Several numerical examples are presented.This work is based on the author's PhD Dissertation at The Hatfield Polytechnic, Hatfield, Hertfordshire, England.     

Numerical solution of the finite horizon stochastic linear quadratic control problem

The treatment of the stochastic linear quadratic optimal control problem with finite time horizon requires the solution of stochastic differential Riccati equations. We propose efficient numerical methods, which exploit the particular structure and can be applied for large‐scale systems. They are based on numerical methods for ordinary differential equations such as Rosenbrock methods, backward differentiation formulas, and splitting methods. The performance of our approach is tested in numerical experiments.     

Necessary conditions for optimal singular stochastic control problems

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state, extending earlier results of the literature.     

Mean reversal for stochastic hybrid systems

Consider two discrete time Markov chains on a finite state space with ±1 win or lose payoff subject to transition between the states. We introduce a class of processes whose cumulative expected payoffs are decreasing in time but, whenever the processes are chosen at random by flipping a fair coin, the expected payoff for the randomized process becomes increasing in time. The seemingly counterintuitive long time run mean reversal generalizes the idea of combining two losing games into a winning one, known as Parrondo’s Paradox.     

Numerical solution of a time-optimal parabolic boundary-value control problem

A special time-optimal parabolic boundary-value control problem describing a one-dimensional heat-diffusion process is solved numerically. Using a bang-bang principle recently proved by Lempio, this problem can be transformed in such a way that the variables are jumps of bang-bang controls. A discretization is performed in two steps, and the convergence of the approximate solutions is proved. Finally, an algorithm to solve the discrete problem is developed and some numerical results are discussed.The author would like to thank Prof. F. Lempio, who pointed out this problem to him, and Prof. K. Glashoff for many helpful comments and suggestions.     

LP-stabilization problem for linear stochastic control systems with multiplicative noise

For the deterministic case, a linear controlled system is alwayspth order stable as long as we use the control obtained as the solution of the so-called LQ-problem. For the stochastic case, however, a linear controlled system with multiplicative noise is not alwayspth mean stable for largep, even if we use the LQ-optimal control. Hence, it is meaningful to solve the LP-optimal control problem (i.e., linear system,pth order cost functional) for eachp. In this paper, we define the LP-optimal control problem and completely solve it for the scalar case. For the multidimensional case, we get some results, but the general solution of this problem seems to be impossible. So, we consider thepth mean stabilization problem more intensively and give a sufficient condition for the existence of apth mean stabilizing control by using the contraction mapping method in a Hilbert space. Some examples are also given.This research was conducted while the author was a visitor at the Forschungsschwerpunkt Dynamische Systeme, Universität Bremen, Bremen, West Germany. The author is grateful to Professor L. Arnold for providing interesting seminars and excellent working conditions during his stay. The financial assistance given by the Alexander von Humboldt Foundation during the author's stay is also gratefully acknowledged.     

Dynamic programming for multidimensional stochastic control problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem, singular control problem and impulse control problem as special cases. Using a unified treatment of dynamic programming, we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasivariational inequality. The uniqueness of such a quasi-variational inequality is proved. Supported in part by USA Office of Naval Research grant #N00014-96-1-0262. Supported in part by the NSFC Grant #79790130, the National Distinguished Youth Science Foundation of China Grant #19725106 and the Chinese Education Ministry Science Foundation.     

Numerical solution of a singular integral equation arising in a cruciform crack problem

A singular integral equation arising in a cruciform crack problem is investigated in the present paper. Based on the convex technique, the piecewise Taylor-series expansion method is extended by introducing a weight parameter. An approximate solution of the singular integral equation is constructed and its convergence and error estimate are made. The variations of the approximate solutions associating with stress intensity factors are analyzed by considering internal pressures of power and sine functions, respectively. By comparing with the known methods, the observations reveal that a good approximation can be achieved using less derivative times, less discretization points, and a suitable weight parameter. The obtained results show that the crack growth is dependent on applied mechanical loadings.     

Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.     

Approximate solution of a class of singular control problems

A class of singular control problems involving amplitude constraints on the controls is examined. IfL is the space of control functionsU, the control constraint setS can be identified with the unit ball inL . Now, for anyn (1, ), an analogous problem may be set up withL ⁿ forU and the unit ball inL ⁿ forS. This modified problem is necessarily nonsingular for controllable systems. It is shown that, by takingn sufficiently large, the solution to the modified problem also solves the original problem arbitrarily closely (in a sense made precise). Behavior asn is investigated.This research was supported by the Science Research Council of Great Britain and the Commonwealth Fund (Harkness Fellowship).     

Event-triggered sliding mode control for singular systems with disturbance

This paper investigates the event-triggered sliding mode control (SMC) problem for singular systems with disturbance. Firstly, an event-triggered sliding mode control law is designed to guarantee the reachability of sliding surface. Different from the related methods, in order to deal with the difficulty caused by event-triggered SMC strategy, a novel Lemma is proposed in this paper. Secondly, the admissibility of sliding motion is presented, which is used to solve the controller gain. Then, a positive lower bound of the inter execution time can be guaranteed and the Zeno behavior is avoided. Finally, two simulation examples are presented to show the effectiveness of derived theoretical results.     

A scenario‐based stochastic programming model for the control or dummy wafers downgrading problem

The subject of this paper is to study a realistic planning environment in wafer fabrication for the control or dummy (C/D) wafers problem with uncertain demand. The demand of each product is assumed with a geometric Brownian motion and approximated by a finite discrete set of scenarios. A two‐stage stochastic programming model is developed based on scenarios and solved by a deterministic equivalent large linear programming model. The model explicitly considers the objective to minimize the total cost of C/D wafers. A real‐world example is given to illustrate the practicality of a stochastic approach. The results are better in comparison with deterministic linear programming by using expectation instead of stochastic demands. The model improved the performance of control and dummy wafers management and the flexibility of determining the downgrading policy. Copyright © 2008 John Wiley & Sons, Ltd.     

Thresholds of moving average of stationary processes for given on target significant levels

A Markov chain is associated to a finite order one-sided moving average of a discrete time stationary Gaussian process. A method is developed to specify thresholds for given on target significant levels in the sense that in the long run the probability that the moving average process lies in [L _i , L _i+1), will be π _i , i = 0,. . . ,m. Special inputs, AR(1) and MA(1) are treated in details. This article extends the work of Soltani et al. in (Commun Stat Theory Methods 36(14):2595–2606) where the inputs were assumed to be i.i.d.; and a single threshold was considered. This research was supported by Kuwait University, Research Administration, Research Grant No.[SS08/06].     

Recurrence and strong stochastic persistence of a stochastic single-species model

A stochastic generalized logistic model is considered in this paper. The condition of the existence of its stationary distribution is generalized. Recurrence and strong stochastic persistence are obtained. Finally some numerical simulations are carried out to support our results.     

Optimal feedback control policies for stochastic,distributed-parameter systems

This paper considers optimal feedback control policies for a class of discrete stochastic distributed-parameter systems. The class under consideration has the property that the random variable in the dynamic systems depends only on the time and possesses the Markovian property with stationary transition probabilities. A necessary condition for optimality of a feedback control policy, which has form similar to the Hamiltonian form in the deterministic case, is derived via a dynamic programming approach.     

Numerical solution of positive control problem via linear programming

The reachability problem for linear time-invariant discrete-time control systems with sign-restricted input is considered. The time-optimal control is constructed by an iterative procedure. Each step of the iteration is defined as a linear programming problem. This problem is solved by the simplex algorithm. The initial feasible solution for the simplex algorithm is provided by the preceding step of the iteration. The inversion of the basis matrix is reduced to a bordering procedure. The structural stability of the solution is investigated.