A Quasi-Newton Penalty Barrier Method for Convex Minimization Problems

We describe an infeasible interior point algorithm for convex minimization problems. The method uses quasi-Newton techniques for approximating the second derivatives and providing superlinear convergence. We propose a new feasibility control of the iterates by introducing shift variables and by penalizing them in the barrier problem. We prove global convergence under standard conditions on the problem data, without any assumption on the behavior of the algorithm.     

Convergence of the Gradient Projection Method for Generalized Convex Minimization

This paper develops convergence theory of the gradient projection method by Calamai and Moré (Math. Programming, vol. 39, 93–116, 1987) which, for minimizing a continuously differentiable optimization problem min{f(x) : x } where is a nonempty closed convex set, generates a sequence x_k+1 = P(x_k – _k f(x_k)) where the stepsize _k > 0 is chosen suitably. It is shown that, when f(x) is a pseudo-convex (quasi-convex) function, this method has strong convergence results: either x_k x^* and x^* is a minimizer (stationary point); or x_k arg min{f(x) : x } = , and f(x_k) inf{f(x) : x }.     

A General Iterative Method for Solving Constrained Convex Minimization Problems

It is well known that the gradient-projection algorithm plays an important role in solving minimization problems. In this paper, we will use the idea of regularization to establish a general method so that the sequence generated by the general method can be strongly convergent to a minimizer of constrained convex minimization problems, which solves a variational inequality under suitable conditions.     

一个结构型凸优化问题的分裂算法

考虑带线性约束的三块变量的凸优化模型,目标函数是可分的三个函数和.给出了一个新的分裂算法.首先,对每个块变量解极小化增广拉格朗日函数.然后,通过一个校正步得到新的迭代点.证明了新算法的整体收敛性和O(1/t)的收敛阶.     

Expected Residual Minimization Method for Stochastic Variational Inequality Problems

This paper considers a stochastic variational inequality problem (SVIP). We first formulate SVIP as an optimization problem (ERM problem) that minimizes the expected residual of the so-called regularized gap function. Then, we focus on a SVIP subclass in which the function involved is assumed to be affine. We study the properties of the ERM problem and propose a quasi-Monte Carlo method for solving the problem. Comprehensive convergence analysis is included as well. This work was supported in part by SRF for ROCS, SEM and Project 10771025 supported by NSFC.     

A Dynamical System Associated with Newton's Method for Parametric Approximations of Convex Minimization Problems

We study the existence and asymptotic convergence when t→+∞ for the trajectories generated by where is a parametric family of convex functions which approximates a given convex function f we want to minimize, and ε(t) is a parametrization such that ε(t)→ 0 when t→+∞ . This method is obtained from the following variational characterization of Newton's method: where H is a real Hilbert space. We find conditions on the approximating family and the parametrization to ensure the norm convergence of the solution trajectories u(t) toward a particular minimizer of f . The asymptotic estimates obtained allow us to study the rate of convergence as well. The results are illustrated through some applications to barrier and penalty methods for linear programming, and to viscosity methods for an abstract noncoercive variational problem. Comparisons with the steepest descent method are also provided. Accepted 5 December 1996     

Minimization of A Nondifferentiable Convex Function Defined not Everywhere

We examine an oracle-type method to minimize a convex function f over a convex polyhedron G . The method is an extension of the level-method to the case, when f is a not everywhere finite function, i.e., it may equal to + X at some points of G . An estimate of its efficiency is given, and some modifications of the method are mentioned. Finally, we indicate possible ways of its employment and report results of a numerical experiment.     

求解约束优化问题的记忆梯度Goldstein-Lavintin-Polyak投影算法

给求解无约束规划问题的记忆梯度算法中的参数一个特殊取法,得到目标函数的记忆梯度G o ldste in-L av in tin-Po lyak投影下降方向,从而对凸约束的非线性规划问题构造了一个记忆梯度G o ldste in-L av in tin-Po lyak投影算法,并在一维精确步长搜索和去掉迭代点列有界的条件下,分析了算法的全局收敛性,得到了一些较为深刻的收敛性结果.同时给出了结合FR,PR,HS共轭梯度算法的记忆梯度G o ldste in-L av in tin-Po lyak投影算法,从而将经典共轭梯度算法推广用于求解凸约束的非线性规划问题.数值例子表明新算法比梯度投影算法有效.     

Approximate Level Method for Nonsmooth Convex Minimization

In this paper, we propose and analyse an approximate variant of the level method of Lemaréchal, Nemirovskii and Nesterov for minimizing nonsmooth convex functions. The main per-iteration work of the level method is spent on (i) minimizing a piecewise-linear model of the objective function and (ii) projecting onto the intersection of the feasible region and a level set of the model function. We show that, by replacing exact computations in both cases by approximate computations, in relative scale, the theoretical iteration complexity increases only by a small factor which depends on the approximation level and reduces to one in the exact case.     

The Newton Bracketing Method for Convex Minimization

An iterative method for the minimization of convex functions f :ⁿ , called a Newton Bracketing (NB) method, is presented. The NB method proceeds by using Newton iterations to improve upper and lower bounds on the minimum value. The NB method is valid for n = 1, and in some cases for n > 1 (sufficient conditions given here). The NB method is applied to large scale Fermat–Weber location problems.     

A Global Optimization Method for Solving Convex Quadratic Bilevel Programming Problems

We use the merit function technique to formulate a linearly constrained bilevel convex quadratic problem as a convex program with an additional convex-d.c. constraint. To solve the latter problem we approximate it by convex programs with an additional convex-concave constraint using an adaptive simplicial subdivision. This approximation leads to a branch-and-bound algorithm for finding a global optimal solution to the bilevel convex quadratic problem. We illustrate our approach with an optimization problem over the equilibrium points of an n-person parametric noncooperative game.     

A Strongly Convergent Method for Nonsmooth Convex Minimization in Hilbert Spaces

In this article, we propose a strongly convergent variant on the projected subgradient method for constrained convex minimization problems in Hilbert spaces. The advantage of the proposed method is that it converges strongly when the problem has solutions, without additional assumptions. The method also has the following desirable property: the sequence converges to the solution of the problem which lies closest to the initial iterate.     

Global Convergence of a Modified Gradient Projection Method for Convex Constrained Problems

In this paper, the continuously differentiable optimization problem min{f（x） ： x∈Ω}, where Ω ∈ R^n is a nonempty closed convex set, the gradient projection method by Calamai and More （Math. Programming, Vol.39. P.93-116, 1987） is modified by memory gradient to improve the convergence rate of the gradient projection method is considered. The convergence of the new method is analyzed without assuming that the iteration sequence {x^k} of bounded. Moreover, it is shown that, when f（x） is pseudo-convex （quasiconvex） function, this new method has strong convergence results. The numerical results show that the method in this paper is more effective than the gradient projection method.     

An Extragradient-Based Alternating Direction Method for Convex Minimization

In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processing and other fields have the structure that while one of the two functions has an easy proximal mapping, the other function is smoothly convex but does not have an easy proximal mapping. Therefore, the classical alternating direction methods cannot be applied. To deal with the difficulty, we propose in this paper an alternating direction method based on extragradients. Under the assumption that the smooth function has a Lipschitz continuous gradient, we prove that the proposed method returns an \(\epsilon \)-optimal solution within \(O(1/\epsilon )\) iterations. We apply the proposed method to solve a new statistical model called fused logistic regression. Our numerical experiments show that the proposed method performs very well when solving the test problems. We also test the performance of the proposed method through solving the lasso problem arising from statistics and compare the result with several existing efficient solvers for this problem; the results are very encouraging.     

A Continuous Method for Convex Programming Problems

In this paper, we present a continuous method for convex programming (CP) problems. Our approach converts first the convex problem into a monotone variational inequality (VI) problem. Then, a continuous method, which includes both a merit function and an ordinary differential equation (ODE), is introduced for the resulting variational inequality problem. The convergence of the ODE solution is proved for any starting point. There is no Lipschitz condition required in our proof. We show also that this limit point is an optimal solution for the original convex problem. Promising numerical results are presented.This research was supported in part by Grants FRG/01-02/I-39 and FRG/01-02/II-06 of Hong Kong Baptist University and Grant HKBU2059/02P from the Research Grant Council of Hong Kong.The author thanks Professor Bingsheng He for many helpful suggestions and discussions. The author is also grateful for the comments and suggestions of two anonymous referees. In particular, the author is indebted to one referee who drew his attention to References 15, 17, 18.     

Regularized Newton Methods for Convex Minimization Problems with Singular Solutions

This paper studies convergence properties of regularized Newton methods for minimizing a convex function whose Hessian matrix may be singular everywhere. We show that if the objective function is LC², then the methods possess local quadratic convergence under a local error bound condition without the requirement of isolated nonsingular solutions. By using a backtracking line search, we globalize an inexact regularized Newton method. We show that the unit stepsize is accepted eventually. Limited numerical experiments are presented, which show the practical advantage of the method.     

Subgradient Method with Entropic Projections for Convex Nondifferentiable Minimization

We replace orthogonal projections in the Polyak subgradient method for nonnegatively constrained minimization with entropic projections, thus obtaining an interior-point subgradient method. Inexact entropic projections are quite cheap. Global convergence of the resulting method is established.     

凸约束非线性方程组的一种无导数投影方法

推广LCG共轭梯度方法并建立一种求解凸约束非线性单调方程组问题的无导数投影方法.在适当的条件下,证明了方法的全局收敛性.方法不需要任何导数信息,而且继承了共轭梯度方法储存量小的特征,因此它特别适合求解大规模非光滑的非线性单调方程组问题.大量数值结果和比较表明方法是有效的和稳定的.     

An Exact Penalty Function Algorithm for Non-smooth Convex Constrained Minimization Problems

This paper presents a readily implementable algorthm for solvingconstrained minimization problems invlving (not necessarilysmooth) convex functions. The algorithm minimizes an exact penaltyfunction via the aggregater subgradient method for unconstrainedminimization, A scheme for automatic limitaion of penalty growthis given. The algorithm is globally convergent under mild assumptions.