无约束不可微观划的信赖域算法

提出了一种易实施的求无约束不可微规划的信赖域算法,并在一定条件下证明了该算法所产生的点列的任何聚点都是原问题的稳定点。     

A Quasi-Newton Method with Memory for Unconstrained Function Minimization

A quasi-Newton method for unconstrained function minimizationis described which requires very little additional programmingto that required for the memory gradient method of Miele &Cantrell and which usually converges in far fewer iterations.The new method is essentially that of Fletcher & Powellwith memory; computational experience shows that it can be moreefficient than the method of Fletcher & Powell.     

一个解大规模无约束优化问题的全局梯度法(英文)

本文提出一种新的解大规模无约束优化问题的全局收敛的梯度法.新算法沿着负梯度方向选择步长,而初始步长根据目标函数的海赛矩阵的近似数量矩阵来确定.理论上证明了新算法产生的点列的每个聚点都是稳定的,数值试验表明新算法是可靠且有效的.     

A New Direction Set Method for Unconstrained Minimization without Evaluating Derivatives

A new direction set method for unconstrained minimization withoutevaluating derivatives is presented. The algorithm can be regardedas an application to function minimization of Jacobi's methodfor determining the eigenvalues and eigenvectors of a real symmetricmatrix. Numerical results are presented, illustrating the performanceof the new algorithm on well-known test problems; a comparisonwith other methods is also given.     

On Convergence Properties of Algorithms for Unconstrained Minimization

Suppose F is a convex function on R" for which there is a sequenceof points on which the function values are bounded below andthe gradients converge to zero. Is it possible that F is unboundedbelow? The answer, perhaps surprisingly, is yes for n > 1.     

泊松化图像复原的交替最小化算法

为了快速地去除图像中的泊松噪声,本文在传统的交替方向算法基础上,结合松弛算法提出了一个改进的快速交替最小化算法.与经典的数值算法相比,数值试验表明提出的新算法不但能有效地实现泊松化图像复原,还能大幅度地提高数值计算的速率,并显著地减少电脑的CPU运行时间.     

处理噪声问题的泰勒展开交替最小化算法

为了处理图像、计算机视觉和生物信息等领域中广泛存在的稀疏大噪声和高斯噪声问题,提出了一种利用交替方向最小化思想求解主成分追求松弛模型的泰勒展开交替最小化算法(TEAM).采用推广泰勒展开和收缩算子等技术推导出低秩矩阵和稀疏大噪声矩阵的迭代方向矩阵,加入连续技术提高算法的收敛速率,设计出TEAM算法的求解步骤.实验中,将TEAM算法与该领域的顶级算法作分析对比.结果表明,TEAM算法时间优势明显,误差优势略好.     

An Extragradient-Based Alternating Direction Method for Convex Minimization

In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processing and other fields have the structure that while one of the two functions has an easy proximal mapping, the other function is smoothly convex but does not have an easy proximal mapping. Therefore, the classical alternating direction methods cannot be applied. To deal with the difficulty, we propose in this paper an alternating direction method based on extragradients. Under the assumption that the smooth function has a Lipschitz continuous gradient, we prove that the proposed method returns an \(\epsilon \)-optimal solution within \(O(1/\epsilon )\) iterations. We apply the proposed method to solve a new statistical model called fused logistic regression. Our numerical experiments show that the proposed method performs very well when solving the test problems. We also test the performance of the proposed method through solving the lasso problem arising from statistics and compare the result with several existing efficient solvers for this problem; the results are very encouraging.     

Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization

A special variable metric method is given for finding the stationary points of locally Lipschitz continuous functions which are not necessarily convex or differentiable. Time consuming quadratic programming subproblems do not need to be solved. Global convergence of the method is established. Some encouraging numerical experience is reported.     

Scalar Correction Method for Solving Large Scale Unconstrained Minimization Problems

We introduce a gradient descent algorithm for solving large scale unconstrained nonlinear optimization problems. The computation of the initial trial steplength is based on the usage of both the quasi-Newton property and the Hessian inverse approximation by an appropriate scalar matrix. The nonmonotone line search technique for the steplength calculation is applied later. The computational and storage complexity of the new method is equal to the computational and storage complexity of the Barzilai and Borwein method. On the other hand, the reported numerical results indicate improvements in favor of the new method with respect to the well known global Barzilai and Borwein method.     

无约束最优化锥模型拟牛顿信赖域方法的收敛性(英)

本文研究无约束最优化雄模型拟牛顿信赖域方法的全局收敛性．文章给出了确保这类方法全局收敛的条件．文章还证明了，当用拆线法来求这类算法中锥模型信赖域子问题的近似解时，确保全局收敛的条件得到满足     

Unconstrained Optimization Techniques for the Acceleration of Alternating Projection Methods

Alternating projection methods have been extensively used to find the closest point, to a given point, in the intersection of several given sets that belong to a Hilbert space. One of the characteristics of these schemes is the slow convergence that can be observed in practical applications. To overcome this difficulty, several techniques, based on different ideas, have been developed to accelerate their convergence. Recently, a successful acceleration scheme was developed specially for Cimmino's method when applied to the solution of large-scale saddle point problems. This specialized acceleration scheme is based on the use of the well-known conjugate gradient method for minimizing a related convex quadratic map. In this work, we extend and further analyze this optimization approach for several alternating projection methods on different scenarios. In particular, we include a specialized analysis and treatment for the acceleration of von Neumann-Halperin's method and Cimmino's method on subspaces, and Kaczmarz method on linear varieties. For some specific applications we illustrate the advantages of our acceleration schemes with encouraging numerical experiments.     

Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1

A special variable metric method is given for finding minima of convex functions that are not necessarily differentiable. Time-consuming quadratic programming subproblems do not need to be solved. Global convergence of the method is established. Some encouraging numerical experience is reported.     

采用不定Dogleg路径的非单调自适应信赖域算法解无约束极小化(英文)

In this paper, we combine the nonmonotone and adaptive techniques with trust region method for unconstrained minimization problems. We set a new ratio of the actual descent and predicted descent. Then, instead of the monotone sequence, the nonmonotone sequence of function values are employed. With the adaptive technique, the radius of trust region △k can be adjusted automatically to improve the efficiency of trust region methods. By means of the Bunch-Parlett factorization, we construct a method with indefinite dogleg path for solving the trust region subproblem which can handle the indefinite approximate Hessian Bk. The convergence properties of the algorithm are established. Finally, detailed numerical results are reported to show that our algorithm is efficient.     

A Dynamic Alternating Direction of Multipliers for Nonconvex Minimization with Nonlinear Functional Equality Constraints

Journal of Optimization Theory and Applications - This paper studies the minimization of a broad class of nonsmooth nonconvex objective functions subject to nonlinear functional equality...     

解不等式约束优化的新的序列线性方程组方法

提出一种新的序列线性方程组(SSLE)算法解非线性不等式约束优化问题.在算法的每步迭代,子问题只需解四个简化的有相同的系数矩阵的线性方程组.证明算法是可行的,并且不需假定聚点的孤立性、严格互补条件和积极约束函数的梯度的线性独立性得到算法的全局收敛性.在一定条件下,证明算法的超线性收敛率.     

包含FR方法的一类无约束极小化方法的全局收敛性

本文对包含Fletcher-Reeves共轭梯度法的一类无约束最优化方法的全局收敛性进行了研究．Fletcher-Reeves方法的某些性质在收敛性分析中起着重要的作用．我们以一种简单的方式证明了这类方法在一种Wolfe型非精确线搜索条件下对光滑的非凸函数具有下降性和全局收敛性．全局收敛性结果也被推广到了一种广义Wolfe型非精确线搜索．     

Two-Phase Image Segmentation by Nonconvex Nonsmooth Models with Convergent Alternating Minimization Algorithms

Two-phase image segmentation is a fundamental task to partition an image into foreground and background. In this paper, two types of nonconvex and nonsmooth regularization models are proposed for basic two-phase segmentation. They extend the convex regularization on the characteristic function on the image domain to the nonconvex case, which are able to better obtain piecewise constant regions with neat boundaries. By analyzing the proposed non-Lipschitz model, we combine the proximal alternating minimization framework with support shrinkage and linearization strategies to design our algorithm. This leads to two alternating strongly convex subproblems which can be easily solved. Similarly, we present an algorithm without support shrinkage operation for the nonconvex Lipschitz case. Using the Kurdyka-Łojasiewicz property of the objective function, we prove that the limit point of the generated sequence is a critical point of the original nonconvex nonsmooth problem. Numerical experiments and comparisons illustrate the effectiveness of our method in two-phase image segmentation.     

求解两块可分离凸极小化问题的惯性交替方向乘子法

The alternating direction method of multipliers(ADMM)is a widely used method for solving many convex minimization models arising in signal and image processing.In this paper,we propose an inertial ADMM for solving a two-block separable convex minimization problem with linear equality constraints.This algorithm is obtained by making use of the inertial Douglas-Rachford splitting algorithm to the corresponding dual of the primal problem.We study the convergence analysis of the proposed algorithm in infinite-dimensional Hilbert spaces.Furthermore,we apply the proposed algorithm on the robust principal component analysis problem and also compare it with other state-of-the-art algorithms.Numerical results demonstrate the advantage of the proposed algorithm.     

交替最小化算法求解强凸函数与弱凸函数和的极小值问题

交替最小化算法(简称AMA)最早由[SIAM J.Control Optim.,1991,29(1):119-138]提出,并能用于求解强凸函数与凸函数和的极小值问题.本文直接利用AMA算法来求解强凸函数与弱凸函数和的极小值问题.在强凸函数的模大于弱凸函数的模的假设下,我们证明了AMA生成的点列全局收敛到优化问题的解,并且若该优化问题中的某个函数是光滑函数时,AMA所生成的点列的收敛率是线性的.