A reduced Hessian method for constrained optimization

Reduced Hessian methods have been shown to be successful for equality constrained problems. However there are few results on reduced Hessian methods for general constrained problems. In this paper we propose a method for general constrained problems, based on Byrd and Schnabel's basis-independent algorithm. It can be regarded as a smooth extension of the standard reduced Hessian Method.Research supported in part by NSF, AFORS and ONR through NSF grant DMS-8920550.     

Inexact proximal stochastic gradient method for convex composite optimization

We study an inexact proximal stochastic gradient (IPSG) method for convex composite optimization, whose objective function is a summation of an average of a large number of smooth convex functions and a convex, but possibly nonsmooth, function. Variance reduction techniques are incorporated in the method to reduce the stochastic gradient variance. The main feature of this IPSG algorithm is to allow solving the proximal subproblems inexactly while still keeping the global convergence with desirable complexity bounds. Different subproblem stopping criteria are proposed. Global convergence and the component gradient complexity bounds are derived for the both cases when the objective function is strongly convex or just generally convex. Preliminary numerical experiment shows the overall efficiency of the IPSG algorithm.     

A practical termination criterion for the conjugate gradient method

The conjugate gradient method for the iterative solution of a set of linear equationsAx=b is essentially equivalent to the Lanczos method, which implies that approximations to certain eigen-values ofA can be obtained at low cost. In this paper it is shown how the smallest active eigenvalue ofA can be cheaply approximated, and the usefulness of this approximation for a practical termination criterion for the conjugate gradient method is studied. It is proved that this termination criterion is reliable in many relevant situations.     

A reduced Hessian SQP method for inequality constrained optimization

This paper develops a reduced Hessian method for solving inequality constrained optimization problems. At each iteration, the proposed method solves a quadratic subproblem which is always feasible by introducing a slack variable to generate a search direction and then computes the steplength by adopting a standard line search along the direction through employing the l _∞ penalty function. And a new update criterion is proposed to generate the quasi-Newton matrices, whose dimensions may be variable, approximating the reduced Hessian of the Lagrangian. The global convergence is established under mild conditions. Moreover, local R-linear and superlinear convergence are shown under certain conditions.     

A stochastic method for global optimization

A stochastic method for global optimization is described and evaluated. The method involves a combination of sampling, clustering and local search, and terminates with a range of confidence intervals on the value of the global optimum. Computational results on standard test functions are included as well.     

A modification of the stochastic ruler method for discrete stochastic optimization

We propose a modified stochastic ruler method for finding a global optimal solution to a discrete optimization problem in which the objective function cannot be evaluated analytically but has to be estimated or measured. Our method generates a Markov chain sequence taking values in the feasible set of the underlying discrete optimization problem; it uses the number of visits this sequence makes to the different states to estimate the optimal solution. We show that our method is guaranteed to converge almost surely (a.s.) to the set of global optimal solutions. Then, we show how our method can be used for solving discrete optimization problems where the objective function values are estimated using either transient or steady-state simulation. Finally, we provide some numerical results to check the validity of our method and compare its performance with that of the original stochastic ruler method.     

The conjugate gradient method for computing all the extremal stationary probability vectors of a stochastic matrix

Summary The conjugate gradient method is developed for computing stationary probability vectors of a large sparse stochastic matrixP, which often arises in the analysis of queueing system. When unit vectors are chosen as the initial vectors, the iterative method generates all the extremal probability vectors of the convex set formed by all the stationary probability vectors ofP, which are expressed in terms of the Moore-Penrose inverse of the matrix (P−I). A numerical method is given also for classifying the states of the Markov chain defined byP. One particular advantage of this method is to handle a very large scale problem without resorting to any special form ofP. The Institute of Statistical Mathematics     

A stochastic conjugate gradient method for the approximation of functions

A stochastic conjugate gradient method for the approximation of a function is proposed. The proposed method avoids computing and storing the covariance matrix in the normal equations for the least squares solution. In addition, the method performs the conjugate gradient steps by using an inner product that is based on stochastic sampling. Theoretical analysis shows that the method is convergent in probability. The method has applications in such fields as predistortion for the linearization of power amplifiers.     

Approximate parametric optimization of infinite-dimensional systems

We consider the problem of finding g M_n such that where M_n is the n-dimensional subspace of the complexHilbert space L²(0, ) spanned by an n-tuple of normalized eigenvectoesof the operator , corresponding to eigenvalues. The solution is g = P_nf and P_n denotesthe orthoprojector onto M_n. From Grabowski (1991) we know thatP_n can be expressed in terms of the Malmquist functions. Wegive an alternative approach, more convenient for applicationof the standard mathematical software. The problem of convergenceas n is discussed from both theoretical and numerical viewpoint.The reslts are illustrated by the problems of finding the optimaladjustment of the proportional controller stabilizing a distributedplant. Email: pgrab{at}ia.agh.edu.pl     

A gradient method for neutral systems

This paper derives a gradient method for the iterative solution of control problems described by neutral equations. Three numerical examples are considered, including one with terminal constraints.     

A curvilinear optimization method based upon iterative estimation of the eigensystem of the Hessian matrix

An algorithm was recently presented that minimizes a nonlinear function in several variables using a Newton-type curvilinear search path. In order to determine this curvilinear search path the eigenvalue problem of the Hessian matrix of the objective function has to be solved at each iteration of the algorithm. In this paper an iterative procedure requiring gradient information only is developed for the approximation of the eigensystem of the Hessian matrix. It is shown that for a quadratic function the approximated eigenvalues and eigenvectors tend rapidly to the actual eigenvalues and eigenvectors of its Hessian matrix. The numerical tests indicate that the resulting algorithm is very fast and stable. Moreover, the fact that some approximations to the eigenvectors of the Hessian matrix are available is used to get past saddle points and accelerate the rate of convergence on flat functions.     

Interior gradient and Hessian estimates for the Dirichlet problem of semi-linear degenerate elliptic systems: A probabilistic approach

In this paper, we give interior gradient and Hessian estimates for systems of semi-linear degenerate elliptic partial differential equations on bounded domains, using both tools of backward stochastic differential equations and quasi-derivatives.     

A branch and bound method for stochastic global optimization

A stochastic branch and bound method for solving stochastic global optimization problems is proposed. As in the deterministic case, the feasible set is partitioned into compact subsets. To guide the partitioning process the method uses stochastic upper and lower estimates of the optimal value of the objective function in each subset. Convergence of the method is proved and random accuracy estimates derived. Methods for constructing stochastic upper and lower bounds are discussed. The theoretical considerations are illustrated with an example of a facility location problem.     

Eigenvalue sensitivity and parametric optimization of the large rotating systems

The contribution presents the modal synthesis method of the mathematical modelling of the large rotating systems. The condensed mathematical model is used for deriving analytical formula for eigenvalue sensitivity calculation. According to the sensitivity analysis suitable design parameters are chosen and the optimization process for the purpose of steady state response minimization in a certain operating speed area is performed. The theory is applied to a simple test‐gearbox. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

A stochastic approach for the optimization of open-loop engine control systems

The operation of sensors and actuators in engine control systems is always affected by errors, which are stochastic in nature. In this paper it is shown that, because of the non-linear interactions between engine performance and control laws in an open-loop engine control system, these errors can give rise to unexpected deviations of control variables, fuel consumption and emissions from the optimal values, which are not predictable in an elementary way.A model for vehicle performance evaluation on a driving cycle is presented, which provides the expected values of fuel consumption and emissions in the case of stochastic errors in sensors and actuators, utilizing only steady-state engine data.The stochastic model is utilized to obtain the optimal control laws; the resultant non-linear constrained minimization problem is solved by an Augmented Lagrangian approach, using a Quasi-Newton technique. The results of the stochastic optimization analysis indicate that significant reductions in performance degradation may be achieved with respect to the solutions provided by the classical deterministic approach.     

Active parametric identification of stochastic linear discrete systems in the frequency domain

Under consideration are some theoretical and applied aspects of the active identification of the multidimensional stochastic linear discrete systems in the frequency domain which are described by models in the state space. Original results are obtained in the case that the parameters of mathematical models to be estimated appear in various combinations in the state and control matrices, as well as in the perturbation matrix and the covariation matrices of the dynamic noise and measurement errors. By an example of a system controlling a boring machine, the efficiency and sensibility is shown of applying the active identification procedure.     

A new variant of the memory gradient method for unconstrained optimization

In this paper, we present a new memory gradient method such that the direction generated by this method provides a sufficient descent direction for the objective function at every iteration. Then, we analyze its global convergence under mild conditions and convergence rate for uniformly convex functions. Finally, we report some numerical results to show the efficiency of the proposed method.     

A preconditioning technique for Schur complement systems arising in stochastic optimization

Deterministic sample average approximations of stochastic programming problems with recourse are suitable for a scenario-based parallelization. In this paper the parallelization is obtained by using an interior-point method and a Schur complement mechanism for the interior-point linear systems. However, the direct linear solves involving the dense Schur complement matrix are expensive, and adversely affect the scalability of this approach. We address this issue by proposing a stochastic preconditioner for the Schur complement matrix and by using Krylov iterative methods for the solution of the dense linear systems. The stochastic preconditioner is built based on a subset of existing scenarios and can be assembled and factorized on a separate process before the computation of the Schur complement matrix finishes on the remaining processes. The expensive factorization of the Schur complement is removed from the parallel execution flow and the scaling of the optimization solver is considerably improved with this approach. The spectral analysis indicates an exponentially fast convergence in probability to 1 of the eigenvalues of the preconditioned matrix with the number of scenarios incorporated in the preconditioner. Numerical experiments performed on the relaxation of a unit commitment problem show good performance, in terms of both the accuracy of the solution and the execution time.