A stochastic algorithm using one sample point per iteration and diminishing stepsizes

A stochastic algorithm for finding stationary points of real-valued functions defined on a Euclidean space is analyzed. It is based on the Robbins-Monro stochastic approximation procedure. Gradient evaluations are done by means of Monte Carlo simulations. At each iteratex _i , one sample point is drawn from an underlying probability space, based on which the gradient is approximated. The descent direction is against the approximation of the gradient, and the stepsize is 1/i. It is shown that, under broad conditions, w.p.1 if the sequence of iteratesx ₁,x ₂,...generated by the algorithm is bounded, then all of its accumulation points are stationary.     

Explicit Formulas for Generalized Action–Angle Variables in a Neighborhood of an Isotropic Torus and Their Application

Different versions of the Darboux–Weinstein theorem guarantee the existence of action–angle-type variables and the harmonic-oscillator variables in a neighborhood of isotropic tori in the phase space. The procedure for constructing these variables is reduced to solving a rather complicated system of partial differential equations. We show that this system can be integrated in quadratures, which permits reducing the problem of constructing these variables to solving a system of quadratic equations. We discuss several applications of this purely geometric fact in problems of classical and quantum mechanics.     

A Method for Establishing Optimality Conditions for a Nonsmooth Vector-Valued Minimax Problem

Using a method of uniform approximations, necessary and sufficient conditions for a nonsmooth constrained vector-valued minimax problem are established in terms of Mordukhovich subdifferentials.     

Approximating Clarke’s subgradients of semismooth functions by divided differences

We show that the algorithm presented in an earlier paper by Studniarski (Numer. Math., 55:685–693, 1989) can be applied, after only a small modification, to approximate numerically Clarke’s subgradients of semismooth functions of two variables. Results of computational testing of this modified algorithm are also reported.      

Approximation Methods for Solving the Cauchy Problem

In this paper we give some new results concerning solvability of the 1-dimensional differential equation y′ = f(x, y) with initial conditions. We study the basic theorem due to Picard. First we prove that the existence and uniqueness result remains true if f is a Lipschitz function with respect to the first argument. In the second part we give a contractive method for the proof of Picard theorem. These considerations allow us to develop two new methods for finding an approximation sequence for the solution. Finally, some applications are given.     

On a certain two-sided symmetric condition in magnetic field analysis and computations

A special two-sided condition for the incremental magnetic reluctivity is introduced which guarantees the unique existence of both the weak and the approximate solutions of the nonlinear stationary magnetic field distributed on a region composed of different media, as well as a certain estimate of the error between the two solutions. The condition, being discussed from the physical as well as the mathematical point of view, can be easily verified and is fulfilled for various magnetic reluctivity models used in electrotechnical practice.     

Asymptotic properties of constrained Markov Decision Processes

We present in this paper several asymptotic properties of constrained Markov Decision Processes (MDPs) with a countable state space. We treat both the discounted and the expected average cost, with unbounded cost. We are interested in (1) the convergence of finite horizon MDPs to the infinite horizon MDP, (2) convergence of MDPs with a truncated state space to the problem with infinite state space, (3) convergence of MDPs as the discount factor goes to a limit. In all these cases we establish the convergence of optimal values and policies. Moreover, based on the optimal policy for the limiting problem, we construct policies which are almost optimal for the other (approximating) problems. Based on the convergence of MDPs with a truncated state space to the problem with infinite state space, we show that an optimal stationary policy exists such that the number of randomisations it uses is less or equal to the number of constraints plus one. We finally apply the results to a dynamic scheduling problem.This work was partially supported by the Chateaubriand fellowship from the French embassy in Israel and by the European Grant BRA-QMIPS of CEC DG XIII     

A theoretical approximation scheme for Stackelberg problems

We consider the Stackelberg problem corresponding to a two-player game in which one of the two players has the leadership in playing the game. We present a general approach for approximating the considered hierarchical programming problem by a sequence of two-level optimization problems. From a practical point of view, we also give some results for asymptotically Stackelberg approximating sequences and for problems with perturbed constraints.This paper is based upon results first presented at Journées Fermat: Mathematics for Optimization, Toulouse, France, May 1985.     

Hereditary optimal control problems: Numerical method based upon a padé approximation

In this paper, we consider a particular approximation scheme which can be used to solve hereditary optimal control problems. These problems are characterized by variables with a time-delayed argumentx(t – ). In our approximation scheme, we first replace the variable with an augmented statey(t) x(t - ). The two-sided Laplace transform ofy(t) is a product of the Laplace transform ofx(t) and an exponential factor. This factor is approximated by a first-order Padé approximation, and a differential relation fory(t) can be found. The transformed problem, without any time-delayed argument, can then be solved using a gradient algorithm in the usual way. Four problems are solved to illustrate the validity and usefulness of this technique.This research was supported in part by the National Aeronautics and Space Administration under NASA Grant NCC-2-106.     

On sequential procedures for the point estimation of the mean of a normal population

The sequential procedures developed by Starr (1966, Ann. Math. Statist., 37, 1173–1185) for estimating the mean of a normal population are further analyzed. Asymptotic properties of the regret and first two moments of the stopping rules are studied and second-order approximations are derived.