Linear Systems Associated with Numerical Methods for Constrained Optimization

Linear systems associated with numerical methods for constrained optimization are discussed in thia paper ,It is shown that the corresponding subproblems arise in most well-known methods,no matter line search methods or trust region methods for constrained optimization can be expressed as similar systems of linear equations.All these linear systems can be viewed as some kinds of approximation to the linear system derived by the Lagrange-Newton method .Some properties of these linear systems are analyzed.     

A unified approach to global convergence of trust region methods for nonsmooth optimization

This paper investigates the global convergence of trust region (TR) methods for solving nonsmooth minimization problems. For a class of nonsmooth objective functions called regular functions, conditions are found on the TR local models that imply three fundamental convergence properties. These conditions are shown to be satisfied by appropriate forms of Fletcher's TR method for solving constrained optimization problems, Powell and Yuan's TR method for solving nonlinear fitting problems, Zhang, Kim and Lasdon's successive linear programming method for solving constrained problems, Duff, Nocedal and Reid's TR method for solving systems of nonlinear equations, and El Hallabi and Tapia's TR method for solving systems of nonlinear equations. Thus our results can be viewed as a unified convergence theory for TR methods for nonsmooth problems.Research supported by AFOSR 89-0363, DOE DEFG05-86ER25017 and ARO 9DAAL03-90-G-0093.Corresponding author.     

Optimization of a Class of Nonlinear Dynamic Systems: New Efficient Method without Lagrange Multipliers

This paper deals with optimization of a class of nonlinear dynamic systems with n states and m control inputs commanded to move between two fixed states in a prescribed time. Using conventional procedures with Lagrange multipliers, it is well known that the optimal trajectory is the solution of a two-point boundary-value problem. In this paper, a new procedure for dynamic optimization is presented which relies on tools of feedback linearization to transform nonlinear dynamic systems into linear systems. In this new form, the states and controls can be written as higher derivatives of a subset of the states. Using this new form, it is possible to change constrained dynamic optimization problems into unconstrained problems. The necessary conditions for optimality are then solved efficiently using weighted residual methods.     

Solving a class of linear projection equations

Summary. Many interesting and important constrained optimization problems in mathematical programming can be translated into an equivalent linear projection equation Here, is the orthogonal projection on some convex set (e.g. ) and is a positive semidefinite matrix. This paper presents some new methods for solving a class of linear projection equations. The search directions of these methods are straighforward extensions of the directions in traditional methods for unconstrained optimization. Based on the idea of a projection and contraction method [7] we get a simple step length rule and are able to obtain global linear convergence. Received April 19, 1993 / Revised version received February 9, 1994     

ON AUGMENTED LAGRANGIAN METHODS FOR SADDLE-POINT LINEAR SYSTEMS WITH SINGULAR OR SEMIDEFINITE （1, 1） BLOCKS

An effective algorithm for solving large saddle-point linear systems, presented by Krukier et al., is applied to the constrained optimization problems. This method is a modification of skew-Hermitian triangular splitting iteration methods. We consider the saddle-point linear systems with singular or semidefinite （1, 1） blocks. Moreover, this method is applied to precondition the GMRES. Numerical results have confirmed the effectiveness of the method and showed that the new method can produce high-quality preconditioners for the Krylov subspace methods for solving large sparse saddle-point linear systems.     

A sequential equality constrained quadratic programming algorithm for inequality constrained optimization

In this paper, the feasible type SQP method is improved. A new SQP algorithm is presented to solve the nonlinear inequality constrained optimization. As compared with the existing SQP methods, per single iteration, in order to obtain the search direction, it is only necessary to solve equality constrained quadratic programming subproblems and systems of linear equations. Under some suitable conditions, the global and superlinear convergence can be induced.     

Properties of certain zero column-sum matrices with applications to the optimization of chemical reactors

This work is motivated by linear chemical reactor systems. The mathematical model of these systems employs a finite dimensional concentration vector which yields the properties of a discrete probability distribution. Central in the response of the system is a rate matrix. The properties of these matrices are analyzed in terms of the theories of Markoff and M-matrices. A linear objective function is selected and the optimization of a cascade system relative to changes of the sizes of the tanks is pursued. This amounts to the optimization of the objective function on R₊^m. The global optimum is shown to lie on the diagonal of the domain. Hence, the search for optimum can be simplified to a single dimension. Other related topics such as the effect of the number of tanks in the cascade on the optimum, conditions for off-diagonal stationary points and the constrained optimization are also considered.     

Preconditioning Indefinite Systems in Interior Point Methods for Optimization

Every Newton step in an interior-point method for optimization requires a solution of a symmetric indefinite system of linear equations. Most of today's codes apply direct solution methods to perform this task. The use of logarithmic barriers in interior point methods causes unavoidable ill-conditioning of linear systems and, hence, iterative methods fail to provide sufficient accuracy unless appropriately preconditioned. Two types of preconditioners which use some form of incomplete Cholesky factorization for indefinite systems are proposed in this paper. Although they involve significantly sparser factorizations than those used in direct approaches they still capture most of the numerical properties of the preconditioned system. The spectral analysis of the preconditioned matrix is performed: for convex optimization problems all the eigenvalues of this matrix are strictly positive. Numerical results are given for a set of public domain large linearly constrained convex quadratic programming problems with sizes reaching tens of thousands of variables. The analysis of these results reveals that the solution times for such problems on a modern PC are measured in minutes when direct methods are used and drop to seconds when iterative methods with appropriate preconditioners are used.     

New LP bound in multivariate Lipschitz optimization: Theory and applications

Most numerically promising methods for solving multivariate unconstrained Lipschitz optimization problems of dimension greater than two use rectangular or simplicial branch-and-bound techniques with computationally cheap but rather crude lower bounds.Generalizations to constrained problems, however, require additional devices to detect sufficiently many infeasible partition sets. In this article, a new lower bounding procedure is proposed for simplicial methods yielding considerably better bounds at the expense of two linear programs in each iteration. Moreover, the resulting approach can solve easily linearly constrained problems, since in this case infeasible partition sets are automatically detected by the lower bounding procedure.Finally, it is shown that the lower bounds can be further improved when the method is applied to solve systems of inequalities. Implementation issues, numerical experiments, and comparisons are discussed in some detail.The authors are indebted to the Editor-in-Chief of this journal for his valuable suggestions which have considerably improved the final version of this article.     

A line search filter algorithm with inexact step computations for equality constrained optimization

In this paper, a new line search filter algorithm for equality constrained optimization is presented. The approach belongs to the class of inexact Newton-like methods. It can also be regarded as an inexact version of generic sequential quadratic programming (SQP) methods. The trial step is obtained by truncatedly solving the primal-dual system based on any robust and efficient linear system solver. Practical termination tests for the linear system solver are established to ensure global convergence. Preliminary numerical results demonstrate the approach is potentially useful.     

Newton-type methods for constrained optimization with nonregular constraints

The most important classes of Newton-type methods for solving constrained optimization problems are discussed. These are the sequential quadratic programming methods, active set methods, and semismooth Newton methods for Karush-Kuhn-Tucker systems. The emphasis is placed on the behavior of these methods and their special modifications in the case where assumptions concerning constraint qualifications are relaxed or altogether dropped. Applications to optimization problems with complementarity constraints are examined.     

线性无关假设如何影响非线性约束最优化算法

首先综述非线性约束最优化最近的一些进展. 首次定义了约束最优化算法的全局收敛性. 注意到最优性条件的精确性和算法近似性之间的差异, 并回顾等式约束最优化的原始的Newton 型算法框架, 即可理解为什么约束梯度的线性无关假设应该而且可以被弱化. 这些讨论被扩展到不等式约束最优化问题. 然后在没有线性无关假设条件下, 证明了一个使用精确罚函数和二阶校正技术的算法可具有超线性收敛性. 这些认知有助于接下来开发求解包括非线性半定规划和锥规划等约束最优化问题的更加有效的新算法.     

线性与非线性规划算法与理论

线性规划与非线性规划是数学规划中经典而重要的研究方向. 主要介绍该研究方向的背景知识,并介绍线性规划、无约束优化和约束优化的最新算法与理论以及一些前沿与热点问题. 交替方向乘子法是一类求解带结构的约束优化问题的方法,近年来倍受重视. 全局优化是一个对于应用优化领域非常重要的研究方向. 因此也试图介绍这两个方面的一些最新研究进展和问题.     

ABS methods for continuous and integer linear equations and optimization

ABS methods are a large class of algorithms for solving continuous and integer linear algebraic equations, and nonlinear continuous algebraic equations, with applications to optimization. Recent work by Chinese researchers led by Zunquan Xia has extended these methods also to stochastic, fuzzy and infinite systems, extensions not considered here. The work on ABS methods began almost thirty years. It involved an international collaboration of mathematicians especially from Hungary, England, China and Iran, coordinated by the university of Bergamo. The ABS method are based on the rank reducing matrix update due to Egerváry and can be considered as the most fruitful extension of such technique. They have led to unification of classes of methods for several problems. Moreover they have produced some special algorithms with better complexity than the standard methods. For the linear integer case they have provided the most general polynomial time class of algorithms so far known; such algorithms have been extended to other integer problems, as linear inequalities and LP problems, in over a dozen papers written by Iranian mathematicians led by Nezam Mahdavi-Amiri. ABS methods can be implemented generally in a stable way, techniques existing to enhance their accuracy. Extensive numerical experiments have shown that they can outperform standard methods in several problems. Here we provide a review of their main properties, for linear systems and optimization. We also give the results of numerical experiments on some linear systems. This paper is dedicated to Professor Egerváry, developer of the rank reducing matrix update, that led to ABS methods.     

A Single Component Mutation Evolutionary Programming

In this paper, a novel evolutionary programming is proposed for solving the upper and lower bound optimization problems as well as the linear constrained optimization problems. There are two characteristics of the algorithm: first, only one component of the current solution is mutated in each iteration; second, it can solve the linear constrained optimization problems directly without converting it into unconstrained problems. By solving two kinds of the optimization problems, the algorithm can not only effectively find optimal or close to optimal solutions but also reduce the number of function evolutions compared with the other heuristic algorithms.     

Probability-one homotopies in computational science

Probability-one homotopy algorithms are a class of methods for solving nonlinear systems of equations that, under mild assumptions, are globally convergent for a wide range of problems in science and engineering. Convergence theory, robust numerical algorithms, and production quality mathematical software exist for general nonlinear systems of equations, and special cases such as Brouwer fixed point problems, polynomial systems, and nonlinear constrained optimization. Using a sample of challenging scientific problems as motivation, some pertinent homotopy theory and algorithms are presented. The problems considered are analog circuit simulation (for nonlinear systems), reconfigurable space trusses (for polynomial systems), and fuel-optimal orbital rendezvous (for nonlinear constrained optimization). The mathematical software packages HOMPACK90 and POLSYS_PLP are also briefly described.     

Optimal Noncollapsing Space-Filling Designs for Irregular Experimental Regions

Space-filling and noncollapsing are two important properties in designing computer experiments. We study how the noncollapsing, space-filling designs for irregular experimental regions can be generated efficiently by the proposed metaheuristic methods. We solve this optimal design problem using variants of the discrete particle swarm optimization (DPSO) approaches. Numerical results, including an application in data center thermal management, are used to illustrate the performances of the proposed algorithms. Based on these numerical results, we assert that the most efficient approach is to reformulate the target optimal design problem as a constrained optimization problem and then use a modified DPSO to solve the constrained optimization problem.     

Generalized S-Lemma and strong duality in nonconvex quadratic programming

On the basis of a new topological minimax theorem, a simple and unified approach is developed to Lagrange duality in nonconvex quadratic programming. Diverse generalizations as well as equivalent forms of the S-Lemma, providing a thorough study of duality for single constrained quadratic optimization, are derived along with new strong duality conditions for multiple constrained quadratic optimization. The results allow many quadratic programs to be solved by solving one or just a few SDP’s (semidefinite programs) of about the same size, rather than solving a sequence, often infinite, of SDP’s or linear programs of a very large size as in most existing methods.     

Deriving crisp and consistent priorities for fuzzy AHP-based multicriteria systems using non-linear constrained optimization

Fuzzy optimization models are used to derive crisp weights (priority vectors) for the fuzzy analytic hierarchy process (AHP) based multicriteria decision making systems. These optimization models deal with the imprecise judgements of decision makers by formulating the optimization problem as the system of constrained non linear equations. Firstly, a Genetic Algorithm based heuristic solution for this optimization problem is implemented in this paper. It has been found that the crisp weights derived from this solution for fuzzy-AHP system, sometimes lead to less consistent or inconsistent solutions. To deal with this problem, we have proposed a consistency based constraint for the optimization models. A decision maker can set the consistency threshold value and if the solution exists for that threshold value then crisp weights can be derived, otherwise it can be concluded that the fuzzy comparison matrix for AHP is not consistent for the given threshold. Three examples are considered to demonstrate the effectiveness of the proposed method. Results with the proposed constraint based fuzzy optimization model are more consistent than the existing optimization models.