Local properties of inexact methods for minimizing nonsmooth composite functions

Methods for minimization of composite functions with a nondifferentiable polyhedral convex part are considered. This class includes problems involving minimax functions and norms. Local convergence results are given for “active set” methods, in which an equality-constrained quadratic programming subproblem is solved at each iteration. The active set consists of components of the polyhedral convex function which are active or near-active at the current iteration. The effects of solving the subproblem inexactly at each iteration are discussed; rate-of-convergence results which depend on the degree of inexactness are given.     

Practical convergence conditions for unconstrained optimization

Convergence properties of descent methods are investigated for the case where the usual requirement that an exact line search be performed at each iteration is relaxed. The error at each iteration is measured by the relative decrease in the directional derivative in the search direction. The objective function is assumed to have continuous second derivatives and the eigenvalues of the Hessian are assumed to be bounded above and below by positive constants. Sufficient conditions are given for establishing that a method converges, or that a method converges at a linear rate.These results are used to prove that the order of convergence for a specific conjugate gradient method is linear, provided the error at each iteration is suitably restricted.     

Two-Stage Multisplitting Iteration Methods Using Modulus-Based Matrix Splitting as Inner Iteration for Linear Complementarity Problems

The matrix multisplitting iteration method is an effective tool for solving large sparse linear complementarity problems. However, at each iteration step we have to solve a sequence of linear complementarity sub-problems exactly. In this paper, we present a two-stage multisplitting iteration method, in which the modulus-based matrix splitting iteration and its relaxed variants are employed as inner iterations to solve the linear complementarity sub-problems approximately. The convergence theorems of these two-stage multisplitting iteration methods are established. Numerical experiments show that the two-stage multisplitting relaxation methods are superior to the matrix multisplitting iteration methods in computing time, and can achieve a satisfactory parallel efficiency.     

A variable dimension solution approach for the general spatial price equilibrium problem

Algorithms are presented that are specifically designed for solving general nonlinear multicommodity spatial price equilibrium problems, i.e., problems with nonlinear transportation cost functions, nonlinear supply and demand functions, inter-commodity congestion effects, intercommodity substitution and complementarity effects and interactions among transportation links and among spatially separated markets. The algorithms are specializations of an iterative method for solving nonlinear complementarity problems that requires solving a system of nonlinear equations at each iteration. The algorithms exploit the network structure of the problems to reduce the size of the system of equations to be solved at each iteration. The decision rules for determining which equations are to be included in the system at each iteration are extremely simple, and the remainder of the computational work is carried out by the nonlinear equation solver. Because of this, the algorithms are very easy to implement with readily available software. In addition, since the decision rules only require sign information, only the final system needs to be solved with precision.     

Reliable and efficient error control for an adaptive Galerkin-characteristic method for convection-dominated diffusion problems

An efficient and reliable a-posteriori error estimator is developed for a characteristic-Galerkin finite element method for time-dependent convection-dominated problems. An adaptive algorithm with variable time and space steps is proposed and studied. At each time step in this algorithm grid coarsening occurs solely at the final iteration of the adaptive procedure, meaning that only time and space refinement is allowed before the final iteration. It is proved that at each time step this adaptive algorithm is capable of reducing errors below a given tolerance in a finite number of iteration steps. Numerical results are presented to check the theoretical analysis.     

Simultaneous Iteration for Partial Eigensolution of Real Matrices

Simultaneous iteration methods are presented for obtaining dominanteigenvalues and corresponding eigenvectors of real unsymmetricmatrices. These methods involve the accurate eigensolution ofthe smaller interaction matrix at each iteration, the dimensionsof which depend on the number of vectors processed simultaneously.A bi-iteration procedure is proposed when both left and righteigenvectors are required, and a lopsided iteration procedurewhen only one set of eigenvectors is required. Numerical testsare discussed.     

Time-inhomogeneous discrete stochastic search methods for optimal Bernoulli parameters

We present two time-inhomogeneous search processes for finding the optimal Bernoulli parameters, where the performance measure cannot be evaluated exactly but must be estimated through Monte Carlo simulation. At each iteration, two neighbouring alternatives are compared and the one that appears to be better is passed on to the next iteration. The first search process uses an increasing sample size of each configuration at each iteration. The second search process uses a sequential sampling procedure with increasing boundaries as the number of iterations increases. At each iteration the acceptance of a new configuration depends on the iterate number, therefore, the search process turns out to be inhomogeneous Markov chain. We show that if the increase occurs slower than a certain rate, these search processes will converge to the optimal set with probability one. © 1998 John Wiley & Sons, Ltd.     

A Class of Nested Iteration Schemes for Linear Systems with a Coefficient Matrix with a Dominant Positive Definite Symmetric Part

We present a class of nested iteration schemes for solving large sparse systems of linear equations with a coefficient matrix with a dominant symmetric positive definite part. These new schemes are actually inner/outer iterations, which employ the classical conjugate gradient method as inner iteration to approximate each outer iterate, while each outer iteration is induced by a convergent and symmetric positive definite splitting of the coefficient matrix. Convergence properties of the new schemes are studied in depth, possible choices of the inner iteration steps are discussed in detail, and numerical examples from the finite-difference discretization of a second-order partial differential equation are used to further examine the effectiveness and robustness of the new schemes over GMRES and its preconditioned variant. Also, we show that the new schemes are, at least, comparable to the variable-step generalized conjugate gradient method and its preconditioned variant.     

Practical convergence conditions for the Davidon-Fletcher-Powell method

The convergence properties of the Davidon-Fletcher-Powell method when applied to the minimization of convex functions are considered for the case where the one-dimensional minimization required at each iteration is not solved exactly. Conditions on the error incurred at each iteration are given which are sufficient for the original method to have a linear or superlinear rate of convergence, and for the restarted version to have ann-step quadratic rate of convergence.Sponsored by the United States Army under Contract No. DA-31-124-ARO-D-462.     

Nested splitting CG-like iterative method for solving the continuous Sylvester equation and preconditioning

We present a nested splitting conjugate gradient iteration method for solving large sparse continuous Sylvester equation, in which both coefficient matrices are (non-Hermitian) positive semi-definite, and at least one of them is positive definite. This method is actually inner/outer iterations, which employs the Sylvester conjugate gradient method as inner iteration to approximate each outer iterate, while each outer iteration is induced by a convergent and Hermitian positive definite splitting of the coefficient matrices. Convergence conditions of this method are studied and numerical experiments show the efficiency of this method. In addition, we show that the quasi-Hermitian splitting can induce accurate, robust and effective preconditioned Krylov subspace methods.     

Numerical implementation of iteration processes applied in the study of nonlinear boundary value problems of the theory of elasticity and filtration

Nonlinear elastic problems for hardening media are solved by applying the universal iteration process proposed by A.I. Koshelev in his works on the regularity of solutions to quasilinear elliptic and parabolic systems. This requires numerically solving a linear elliptic system at each step of the iteration procedure. The method is numerically implemented in the MATLAB environment by using its PDE Toolbox. A modification of the finite-element procedure is suggested in order to solve a linear algebraic system at each iteration step. The computer model is tested on simple examples. The same nonlinear problems are also solved by the method of elastic solutions, which consists in replacing the Laplace operator in the universal iteration process by the Lamé operator of linear elasticity. As is known, this iteration process converges to a weak solution of the nonlinear problem, provided that the displacements are fixed on the boundary. The method is tested on examples with stresses on the boundary. The third part of the paper is devoted to the nonlinear filtration problem. General properties of the iteration process for nonlinear parabolic systems have been studied by A.I. Koshelev and V.M. Chistyakov. The numerical implementation is based on slightly modified PDE Toolbox procedures. The program is tested on simple examples.     

A dimensional split preconditioner for Stokes and linearized Navier-Stokes equations

In this paper we introduce a new preconditioner for linear systems of saddle point type arising from the numerical solution of the Navier-Stokes equations. Our approach is based on a dimensional splitting of the problem along the components of the velocity field, resulting in a convergent fixed-point iteration. The basic iteration is accelerated by a Krylov subspace method like restarted GMRES. The corresponding preconditioner requires at each iteration the solution of a set of discrete scalar elliptic equations, one for each component of the velocity field. Numerical experiments illustrating the convergence behavior for different finite element discretizations of Stokes and Oseen problems are included.     

An additional projection step to He and Liao's method for solving variational inequalities

In this paper, we proposed a modified extragradient method for solving variational inequalities. The method can be viewed as an extension of the method proposed by He and Liao [Improvement of some projection methods for monotone variational inequalities, J. Optim. Theory Appl. 112 (2002) 111–128], by performing an additional projection step at each iteration and another optimal step length is employed to reach substantial progress in each iteration. We used a self-adaptive technique to adjust parameter ρ$\rho$ at each iteration. Under certain conditions, the global convergence of the proposed method is proved. Preliminary numerical experiments are included to compare our method with some known methods.     

一类带非单调线搜索的信赖域算法

通过将非单调Wolfe线搜索技术与传统的信赖域算法相结合,我们提出了一类新的求解无约束最优化问题的信赖域算法.新算法在每一迭代步只需求解一次信赖域子问题,而且在每一迭代步Hesse阵的近似都满足拟牛顿条件并保持正定传递.在一定条件下,证明了算法的全局收敛性和强收敛性.数值试验表明新算法继承了非单调技术的优点,对于求解某...     

Relaxed Cutting Plane Method for Solving Linear Semi-Infinite Programming Problems

One of the major computational tasks of using the traditional cutting plane approach to solve linear semi-infinite programming problems lies in finding a global optimizer of a nonlinear and nonconvex program. This paper generalizes the Gustafson and Kortanek scheme to relax this requirement. In each iteration, the proposed method chooses a point at which the infinite constraints are violated to a degree, rather than a point at which the violations are maximized. A convergence proof of the proposed scheme is provided. Some computational results are included. An explicit algorithm which allows the unnecessary constraints to be dropped in each iteration is also introduced to reduce the size of computed programs.     

On inexact alternating direction implicit iteration for continuous Sylvester equations

In this paper, we study the alternating direction implicit (ADI) iteration for solving the continuous Sylvester equation AX + XB = C , where the coefficient matrices A and B are assumed to be positive semi‐definite matrices (not necessarily Hermitian), and at least one of them to be positive definite. We first analyze the convergence of the ADI iteration for solving such a class of Sylvester equations, then derive an upper bound for the contraction factor of this ADI iteration. To reduce its computational complexity, we further propose an inexact variant of the ADI iteration, which employs some Krylov subspace methods as its inner iteration processes at each step of the outer ADI iteration. The convergence is also analyzed in detail. The numerical experiments are given to illustrate the effectiveness of both ADI and inexact ADI iterations.     

A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization

By using the Moreau-Yosida regularization and proximal method, a new trust region algorithm is proposed for nonsmooth convex minimization. A cubic subproblem with adaptive parameter is solved at each iteration. The global convergence and Q-superlinear convergence are established under some suitable conditions. The overall iteration bound of the proposed algorithm is discussed. Preliminary numerical experience is reported.     

A superharmonic approach to solving infinite horizon partially observable Markov decision problems

In this paper we use an approach which uses a superharmonic property of a sequence of functions generated by an algorithm to show that these functions converge in a non-increasing manner to the optimal value function for our problem, and bounds are given for the loss of optimality if the computational process is terminated at any iteration. The basic procedure is to add an additional linear term at each iteration, selected by solving a particular optimisation problem, for which primal and dual linear programming formulations are given.     

Efficient fourth orderP-stable formulae

In this paper, a family of fourth orderP-stable methods for solving second order initial value problems is considered. When applied to a nonlinear differential system, all the methods in the family give rise to a nonlinear system which may be solved using a modified Newton method. The classical methods of this type involve at least three (new) function evaluations per iteration (that is, they are 3-stage methods) and most involve using complex arithmetic in factorising their iteration matrix. We derive methods which require only two (new) function evaluations per iteration and for which the iteration matrix is a true real perfect square. This implies that real arithmetic will be used and that at most one real matrix must be factorised at each step. Also we consider various computational aspects such as local error estimation and a strategy for changing the step size.     

Readily implementable conjugate gradient methods

Conjugate gradient methods have been extensively used to locate unconstrained minimum points of real-valued functions. At present, there are several readily implementable conjugate gradient algorithms that do not require exact line search and yet are shown to be superlinearly convergent. However, these existing algorithms usually require several trials to find an acceptable stepsize at each iteration, and their inexact line search can be very timeconsuming.In this paper we present new readily implementable conjugate gradient algorithms that will eventually require only one trial stepsize to find an acceptable stepsize at each iteration.Making usual continuity assumptions on the function being minimized, we have established the following properties of the proposed algorithms. Without any convexity assumptions on the function being minimized, the algorithms are globally convergent in the sense that every accumulation point of the generated sequences is a stationary point. Furthermore, when the generated sequences converge to local minimum points satisfying second-order sufficient conditions for optimality, the algorithms eventually demand only one trial stepsize at each iteration, and their rate of convergence isn-step superlinear andn-step quadratic.This research was supported in part by the National Science Foundation under Grant No. ENG 76-09913.