非线性最优化的广义梯度投影法

梯度投影法已有许多有效算法,但这些算法还存在三个问题:1)为了保证算法的收敛性,在算法的每一迭代步,需要选取δ-主动约束集,计算量较大.2)在迭代过程中,需要跟踪主动约束集.3)只能处理非线性不等式约束问题.本文讨论非线性等式与不等式约束的优化问题,给出了一个广义梯度投影法,证明了算法的收敛性并且完满地解决了上述三个问题.本文算法结构简单且其处理技巧有普遍意义.     

具有精确线性搜索的改进共轭梯度法

其中g_k=f(x_k),β_k为参数.β_k的不同选法形成了各种共轭梯度法,其中Fletcher-Reeves法(简记为FR法)是理论较完整的一个方法,对水平集有界的二阶连续可微函数,Powell和Baali分别在精确和不精确线搜索下证明了其全局收敛性.Polak-Ribiere法     

广义梯度和ε-严有效解的最优性条件

在锥序Banach空间中引入了集值映射ε-严有效意义下的广义梯度.在连通性条件下,利用凸集分离定理证明了该广义梯度的存在性.作为应用,给出了用广义梯度刻画集值优化问题ε-严有效解的充分和必要条件.     

求解非线性系统的有效集信赖域方法

该文给出了一个求解非线性系统的信赖域方法.主要思想是通过引入松弛变量,将问题等价地转化为带非负约束的最优化问题.作者利用有效集策略,在每次迭代中只需求解一个低维的信赖域子问题,该信赖域子问题是通过截断共轭梯度法来近似求解的.在较弱的条件下,获得了一个更一般的收敛性结果.     

集值映射的广义梯度和全局真有效解

本文利用集值映射的上图导数引进了全局真有效意义下的广义梯度和广义次微分的概念，并且给出了集值映射全局真有效次微分的存在定理，还建立了集值向量优化问题全局真有效解在次微分形式下的最优性条件．     

集值映射$\epsilon$-超次梯度的性质及应用[英文]

本文讨论集值映射$\epsilon$-超次梯度的性质，建立$\epsilon$-超次梯度意义下的Moreau-Rockafellar定理.作为应用， 借助$\epsilon$-超次梯度分别得到集值优化取得$\epsilon$-超有效元的充分和必要条件.     

Chan-Vese模型的共轭梯度算法

随着图像采集设备的发展和对图像分辨率要求的提高,人们对图像处理算法在收敛速度和鲁棒性方面提出了更高的要求.从优化的角度对Chan-Vese模型进行算法上的改进,即将共轭梯度法应用到该模型中,使得新算法有更快的收敛速度.首先,简单介绍了Chan-Vese模型的变分水平集方法的理论框架;其次,将共轭梯度算法引入到该模型的求解,得到了模型的新的数值解方法;最后,将得到的算法与传统求解Chan-Vese模型的最速下降法进行了比较.数值实验表明,提出的共轭梯度算法在保持精度的前提下有更快的收敛速度.     

一种自适应Dai-Liao共轭梯度法

共轭梯度法是一种解决大规模无约束优化问题的重要方法.本文对Dai-Liao (DL)共轭梯度法的参数进行了研究,提出了一种新的自适应DL共轭梯度法.在适当的条件下,证明了该方法的全局收敛性.数值结果表明,我们的方法对给定的测试问题是有效的.     

大规模非线性互补问题的共轭梯度法

提出求解大规模非线性互补问题NCP(F)的PRP型共轭梯度法,算法自然满足充分下降条件.当F是可微P_0+R_0函数且F'(χ)在水平集上全局Lipschitz连续条件下,证明了算法的全局收敛性.数值结果表明算法的有效性.     

拓广的Rosen梯度投影法及其整体收敛性证明

§1.引言 Rosen梯度投影法是求解非线性规划问题的基本方法之一,方法简便,实际应用的数值效果好,而且许多近代的更有效的算法继续采用了它的基本思想和技巧。在这些算法中最有代表性的是Goldfarb方法和Murtagh-Sargents方法,其收敛性自然在某种程度上依赖于Rosen方法的收敛性。但是,Rosen方法之严格的收敛性证明尚未取得。尽管D.G.Luenberger在其著作中应用Zangwill的总体收敛性充分性定理说     

A factorization with update procedures for a KKT matrix arising in direct optimal control

Quadratic programs obtained for optimal control problems of dynamic or discrete-time processes usually involve highly block structured Hessian and constraints matrices, to be exploited by efficient numerical methods. In interior point methods, this is elegantly achieved by the widespread availability of advanced sparse symmetric indefinite factorization codes. For active set methods, however, conventional dense matrix techniques suffer from the need to update base matrices in every active set iteration, thereby loosing the sparsity structure after a few updates. This contribution presents a new factorization of a KKT matrix arising in active set methods for optimal control. It fully respects the block structure without any fill-in. For this factorization, matrix updates are derived for all cases of active set changes. This allows for the design of a highly efficient block structured active set method for optimal control and model predictive control problems with long horizons or many control parameters.     

梯度投影算子的广义陡度引理及其应用

在一般闭凸集上建立了梯度投影算子的广义陡度引理,利用它证明了几种松 弛搜索下梯度投影算法的全局收敛性、强收敛性以及若干良好的收敛性质.     

Modified active set projected spectral gradient method for bound constrained optimization

In this paper, by means of an active set strategy, we present a projected spectral gradient algorithm for solving large-scale bound constrained optimization problems. A nice property of the active set estimation technique is that it can identify the active set at the optimal point without requiring strict complementary condition, which is potentially used to solve degenerated optimization problems. Under appropriate conditions, we show that this proposed method is globally convergent. We also do some numerical experiments by using some bound constrained problems from CUTEr library. The numerical comparisons with SPG, TRON, and L-BFGS-B show that the proposed method is effective and promising.     

Subspace Barzilai-Borwein Gradient Method for Large-Scale Bound Constrained Optimization

An active set subspace Barzilai-Borwein gradient algorithm for large-scale bound constrained optimization is proposed. The active sets are estimated by an identification technique. The search direction consists of two parts: some of the components are simply defined; the other components are determined by the Barzilai-Borwein gradient method. In this work, a nonmonotone line search strategy that guarantees global convergence is used. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known method SPG on a subset of bound constrained problems from CUTEr collection. This work was supported by the 973 project granted 2004CB719402 and the NSF project of China granted 10471036.     

Three-term conjugate gradient method for the convex optimization problem over the fixed point set of a nonexpansive mapping

Many constrained sets in problems such as signal processing and optimal control can be represented as a fixed point set of a certain nonexpansive mapping, and a number of iterative algorithms have been presented for solving a convex optimization problem over a fixed point set. This paper presents a novel gradient method with a three-term conjugate gradient direction that is used to accelerate conjugate gradient methods for solving unconstrained optimization problems. It is guaranteed that the algorithm strongly converges to the solution to the problem under the standard assumptions. Numerical comparisons with the existing gradient methods demonstrate the effectiveness and fast convergence of this algorithm.     

A SUBSPACE PROJECTED CONJUGATE GRADIENT ALGORITHM FOR LARGE BOUND CONSTRAINED QUADRATIC PROGRAMMING

A subspace projected conjugate gradient method is proposed for solving large bound constrained quadratic programming. The conjugate gradient method is used to update the variables with indices outside of the active set, while the projected gradient method is used to update the active variables. At every iterative level, the search direction consists of two parts, one of which is a subspace trumcated Newton direction, another is a modified gradient direction. With the projected search the algorithm is suitable to large problems. The convergence of the method is proved and same numerical tests with dimensions ranging from 5000 to 20000 are given.     

The Dai–Liao nonlinear conjugate gradient method with optimal parameter choices

Minimizing two different upper bounds of the matrix which generates search directions of the nonlinear conjugate gradient method proposed by Dai and Liao, two modified conjugate gradient methods are proposed. Under proper conditions, it is briefly shown that the methods are globally convergent when the line search fulfills the strong Wolfe conditions. Numerical comparisons between the implementations of the proposed methods and the conjugate gradient methods proposed by Hager and Zhang, and Dai and Kou, are made on a set of unconstrained optimization test problems of the CUTEr collection. The results show the efficiency of the proposed methods in the sense of the performance profile introduced by Dolan and Moré.     

Acceleration of conjugate gradient algorithms for unconstrained optimization

Conjugate gradient methods are important for large-scale unconstrained optimization. This paper proposes an acceleration of these methods using a modification of steplength. The idea is to modify in a multiplicative manner the steplength α_k, computed by Wolfe line search conditions, by means of a positive parameter η_k, in such a way to improve the behavior of the classical conjugate gradient algorithms. It is shown that for uniformly convex functions the convergence of the accelerated algorithm is still linear, but the reduction in function values is significantly improved. Numerical comparisons with some conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that the accelerated computational scheme outperform the corresponding conjugate gradient algorithms.     

约束优化问题的修正共轭梯度投影算法

对闭凸集约束的非线性规划问题构造了一个修正共轭梯度投影下降算法,在去掉迭代点列有界的条件下,分析了算法的全局收敛性.新算法与共轭梯度参数结合,给出了三类结合共轭梯度参数的修正共轭梯度投影算法.数值例子表明算法是有效的.     

Newton-type methods for constrained optimization with nonregular constraints

The most important classes of Newton-type methods for solving constrained optimization problems are discussed. These are the sequential quadratic programming methods, active set methods, and semismooth Newton methods for Karush-Kuhn-Tucker systems. The emphasis is placed on the behavior of these methods and their special modifications in the case where assumptions concerning constraint qualifications are relaxed or altogether dropped. Applications to optimization problems with complementarity constraints are examined.