互补约束规划问题的一个广义梯度投影算法

本文研究了一类均衡约束最优化问题.利用广义梯度投影法,结合罚函数思想,得到了一个初始点可以任意的广义梯度投影算法.在较弱的条件下,证明了算法的全局收敛性.     

互补约束数学规划问题的一个广义梯度投影罚算法

结合罚函数思想和广义梯度投影技术,提出求解非线性互补约束数学规划问题的一个广义梯度投影罚算法.首先,通过扰动技术和广义互补函数,将原问题转化为序列带参数的近似的标准非线性规划;其次,利用广义梯度投影矩阵构造搜索方向的显式表达式.一个特殊的罚函数作为效益函数,而且搜索方向能保证效益函数的下降性.在适当的假设条件下算法具有全局收敛性.     

冲击接触问题的一种双共振投影梯度算法

根据冲击接触计算模型所需满足的基本控制方程和非线性互补条件,应用非线性互补问题与约束优化的等价关系将非线性互补接触问题转变成一个非线性规划问题,系统地推导建立了冲击接触问题的一种双共轭投影梯度计算方法．增广Lagrange乘子法克服了罚函数要求减小迭代步长以达到计算稳定的限制,即使对于冲击接触问题亦可以采用较大迭代步长,在形成的与原互补问题等价的无约束规划模式下,应用双共轭投影梯度算法提高非线性搜索速度和计算效率．算法模型计算结果表明,所建立的双共轭投影梯度计算理论及方法是正确有效的．     

一般线性或非线性约束下的共轭投影变尺度方法

梯度投影法是一类有效的约束最优化算法，在最优化领域中占有重要的地位．但是，梯度投影法所采用的投影是正交投影，不包含目标函数和约束函数的二阶导数信息·因而；收敛速度不太令人满意．本文介绍一种共轭投影概念，利用共轭投影构造了一般线性或非线性约束下的共轭投影变尺度算法，并证明了算法在一定条件下具有全局收敛性．由于算法中的共轭投影恰当地包含了目标函数和约束函数的二阶导数信息，因而收敛速度有希望加快．数值试验的结果表明算法是有效的．     

Minimizing Quadratic Functions Subject to Bound Constraints with the Rate of Convergence and Finite Termination

A new active set based algorithm is proposed that uses the conjugate gradient method to explore the face of the feasible region defined by the current iterate and the reduced gradient projection with the fixed steplength to expand the active set. The precision of approximate solutions of the auxiliary unconstrained problems is controlled by the norm of violation of the Karush-Kuhn-Tucker conditions at active constraints and the scalar product of the reduced gradient with the reduced gradient projection. The modifications were exploited to find the rate of convergence in terms of the spectral condition number of the Hessian matrix, to prove its finite termination property even for problems whose solution does not satisfy the strict complementarity condition, and to avoid any backtracking at the cost of evaluation of an upper bound for the spectral radius of the Hessian matrix. The performance of the algorithm is illustrated on solution of the inner obstacle problems. The result is an important ingredient in development of scalable algorithms for numerical solution of elliptic variational inequalities.     

不等式约束优化一个基于滤子思想的广义梯度投影算法

讨论非线性不等式约束优化问题, 借鉴于滤子算法思想,提出了一个新型广义梯度投影算法.该方法既不使用罚函数又无真正意义下的滤子.每次迭代通过一个简单的显式广义投影法产生搜索方向,步长由目标函数值或者约束违反度函数值充分下降的Armijo型线搜索产生.算法的主要特点是: 不需要迭代序列的有界性假设;不需要传统滤子算法所必需的可行恢复阶段;使用了ε积极约束集减小计算量.在合适的假设条件下算法具有全局收敛性, 最后对算法进行了初步的数值实验.     

On the convergence rate for a penalty function method of exponential type

Recently, Kort and Bertsekas (Ref. 1) and Hartman (Ref. 2) presented independently a new penalty function algorithm of exponential type for solving inequality-constrained minimization problems. The main purpose of this work is to give a proof on the rate of convergence of a modification of the exponential penalty method proposed by these authors. We show that the sequence of points generated by the modified algorithm converges to the solution of the original nonconvex problem linearly and that the sequence of estimates of the optimal Lagrange multiplier converges to this multiplier superlinearly. The question of convergence of the modified method is discussed. The present paper hinges on ideas of Mangasarian (Ref. 3), but the case considered here is not covered by Mangasarian's theory.     

等式约束优化问题的一类新的简单光滑精确罚函数

精确罚函数方法是求解优化问题的一类经典方法,传统的精确罚函数不可能既是简单的又是光滑的,这里简单的是指罚函数中不包含目标函数和约束函数的梯度信息。针对等式约束问题提出了不同与传统罚函数的一类新的简单光滑罚函数并证明了它是精确的。给出了以新的罚函数为基础的罚函数方法并用数值例子说明算法是可行的。     

A gradient projection-multiplier method for nonlinear programming

This paper describes a gradient projection-multiplier method for solving the general nonlinear programming problem. The algorithm poses a sequence of unconstrained optimization problems which are solved using a new projection-like formula to define the search directions. The unconstrained minimization of the augmented objective function determines points where the gradient of the Lagrangian function is zero. Points satisfying the constraints are located by applying an unconstrained algorithm to a penalty function. New estimates of the Lagrange multipliers and basis constraints are made at points satisfying either a Lagrangian condition or a constraint satisfaction condition. The penalty weight is increased only to prevent cycling. The numerical effectiveness of the algorithm is demonstrated on a set of test problems.The author gratefully acknowledges the helpful suggestions of W. H. Ailor, J. L. Searcy, and D. A. Schermerhorn during the preparation of this paper. The author would also like to thank D. M. Himmelblau for supplying a number of interesting test problems.     

求解约束优化问题的记忆梯度Goldstein-Lavintin-Polyak投影算法

给求解无约束规划问题的记忆梯度算法中的参数一个特殊取法,得到目标函数的记忆梯度G o ldste in-L av in tin-Po lyak投影下降方向,从而对凸约束的非线性规划问题构造了一个记忆梯度G o ldste in-L av in tin-Po lyak投影算法,并在一维精确步长搜索和去掉迭代点列有界的条件下,分析了算法的全局收敛性,得到了一些较为深刻的收敛性结果.同时给出了结合FR,PR,HS共轭梯度算法的记忆梯度G o ldste in-L av in tin-Po lyak投影算法,从而将经典共轭梯度算法推广用于求解凸约束的非线性规划问题.数值例子表明新算法比梯度投影算法有效.     

任意初始点下的广义梯度投影滤子算法

提出了一个任意初始点的广义梯度滤子方法. 该方法不使用罚函数以避免由此带来的缺陷并可以减少计算量. 方法的另一个特点是不因使用了滤子技术而使算法早熟或陷入循环. 算法对初始点没有要求并在比较合理的条件下具有全局收敛性.     

凸约束非光滑方程组一个新的谱梯度投影算法

基于寻找分离超平面的三种经典线搜索技术,本文提出了一种自适应线搜索技术.结合谱梯度投影法,提出了凸约束非光滑单调方程组的一个谱梯度投影算法.该算法不需要计算和存储任何矩阵,因而适合求解大规模非光滑的非线性单调方程组.在较弱的条件下,证明了方法的全局收敛性,并分析了算法的收敛率.数值试验结果表明算法是有效的和鲁棒的.     

Firefly penalty-based algorithm for bound constrained mixed-integer nonlinear programming

In this article, we aim to extend the firefly algorithm (FA) to solve bound constrained mixed-integer nonlinear programming (MINLP) problems. An exact penalty continuous formulation of the MINLP problem is used. The continuous penalty problem comes out by relaxing the integrality constraints and by adding a penalty term to the objective function that aims to penalize integrality constraint violation. Two penalty terms are proposed, one is based on the hyperbolic tangent function and the other on the inverse hyperbolic sine function. We prove that both penalties can be used to define the continuous penalty problem, in the sense that it is equivalent to the MINLP problem. The solutions of the penalty problem are obtained using a variant of the metaheuristic FA for global optimization. Numerical experiments are given on a set of benchmark problems aiming to analyze the quality of the obtained solutions and the convergence speed. We show that the firefly penalty-based algorithm compares favourably with the penalty algorithm when the deterministic DIRECT or the simulated annealing solvers are invoked, in terms of convergence speed.     

A method for determining the equilibrium states of dynamic systems

A heuristic method is presented for determining the equilibrium states of motion of dynamic systems, in particular, spacecraft. The method can also be applied to the solution of sets of linear or nonlinear algebraic equations. A positive-semidefinite functional is formed to convert the problem to that of finding those minimum points where the functional vanishes. The process is initiated within a selecteddomain of interest by random search; convergence to a minimum is obtained by a modified Davidon's deflected gradient technique. To render this approach feasible in the presence of constraints, the functional is modified to include penalty terms which cause the functional to approach infinity at the constraint boundaries. Close approximations to solutions near the constraint boundaries are found by applying Carroll's approach in successively reducing the weighting factors of the penalty terms. After finding a minimum, the local domain around this point is eliminated by adding to the functional an interior constraint term, representing the surface under a hypersphere centered at the minimum point. The domain of consideration now becomes the subdomain formed by subtracting the space contained within this hypersphere from the previous domain of interest. Minima are now sought within the remaining space, as before.This paper is derived from research performed by the author while employed by TRW Systems Group, Redondo Beach, California.The author acknowledges the helpful suggestions of Dr. G. Bekey, University of Southern California, and those of Mr. E. A. Quast, Dr. M. P. Scher, and Dr. R. J. Wiley, Dynamics Department, TRW Systems Group, Redondo Beach, California.     

A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations

ABSTRACT

In this paper, we develop a three-term conjugate gradient method involving spectral quotient, which always satisfies the famous Dai-Liao conjugacy condition and quasi-Newton secant equation, independently of any line search. This new three-term conjugate gradient method can be regarded as a variant of the memoryless Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method with regard to spectral quotient. By combining this method with the projection technique proposed by Solodov and Svaiter in 1998, we establish a derivative-free three-term projection algorithm for dealing with large-scale nonlinear monotone system of equations. We prove the global convergence of the algorithm and obtain the R-linear convergence rate under some mild conditions. Numerical results show that our projection algorithm is effective and robust, and is more competitive with the TTDFP algorithm proposed Liu and Li [A three-term derivative-free projection method for nonlinear monotone system of equations. Calcolo. 2016;53:427–450].     

Spectral Gradient Methods for Linearly Constrained Optimization

Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasi-Newton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right-hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be applied and exploited effectively. Encouraging numerical experiments are presented.This research was supported by FAPESP Grant 2001-04597-4 and Grant 903724-6, FINEP and FAEP-UNICAMP, and the Scientific Computing Center of UCV. The authors thank two anonymous referees whose comments helped us to improve the final version of this paper.     

Parallel algorithms for nonlinear programming problems

This paper describes several parallel algorithms for solving nonlinear programming problems. Two approaches where parallelism can successfully be introduced have been explored: a quadratic approximation method based on penalty function and a dual method. These methods are improved by using two algorithms originally proposed for solving unconstrained problems: the parallel variable metric algorithm and the parallel Jacobson-Oksman algorithm. Even though general problems are dealt with, particular emphasis is placed on the potential of these parallel methods for separable programming problems. The numerical effectiveness of the algorithms is demonstrated on a set of test problems using a Cray-1S vector computer and serial computers (with respect to sequential versions of the same methods).These studies were sponsored in part by the CERT. The author would particularly like to thank Ph. Berger (LSI-ENSEEIHT), the researchers of the DERI (CERT) and of the Groupe Structures, Aerospatiale, for their assistance.     

并行技术在约束凸规划化问题的对偶算法中的应用

用 Rosen(196 1)的投影梯度的方法求解约束凸规划化问题的对偶问题 ,在计算投影梯度方向时 ,涉及求关于原始变量的最小化问题的最优解 .我们用并行梯度分布算法 (PGD)计算出这一极小化问题的近似解 ,证明近似解可以达到任何给定的精度 ,并说明当精度选取合适时 ,Rosen方法仍然是收敛的     

Continuous Gradient Projection Method in Hilbert Spaces

This paper is concerned with the asymptotic analysis of the trajectories of some dynamical systems built upon the gradient projection method in Hilbert spaces. For a convex function with locally Lipschitz gradient, it is proved that the orbits converge weakly to a constrained minimizer whenever it exists. This result remains valid even if the initial condition is chosen out of the feasible set and it can be extended in some sense to quasiconvex functions. An asymptotic control result, involving a Tykhonov-like regularization, shows that the orbits can be forced to converge strongly toward a well-specified minimizer. In the finite-dimensional framework, we study the differential inclusion obtained by replacing the classical gradient by the subdifferential of a continuous convex function. We prove the existence of a solution whose asymptotic properties are the same as in the smooth case.     

Projector preconditioning for partially bound‐constrained quadratic optimization

Preconditioning by a conjugate projector is combined with the recently proposed modified proportioning with reduced gradient projection (MPRGP) algorithm for the solution of bound‐constrained quadratic programming problems. If applied to the partially bound‐constrained problems, such as those arising from the application of FETI‐based domain decomposition methods to the discretized elliptic boundary variational inequalities, the resulting algorithm is shown to have better bound on the rate of convergence than the original MPRGP algorithm. The performance of the algorithm is illustrated on the solution of a model boundary variational inequality. Copyright © 2007 John Wiley & Sons, Ltd.