Minimization by coordinate descent

We examine the minimization of anN-dimensional real-valued function using the coordinate descent method. We impose conditions on the function under which the method converges; furthermore, by specializing our class of functions, we obtain the rate of convergence. We also present some examples from classical approximation theory where this method applies. A computational example is also given.     

包含FR方法的一类无约束极小化方法的全局收敛性

本文对包含Fletcher-Reeves共轭梯度法的一类无约束最优化方法的全局收敛性进行了研究．Fletcher-Reeves方法的某些性质在收敛性分析中起着重要的作用．我们以一种简单的方式证明了这类方法在一种Wolfe型非精确线搜索条件下对光滑的非凸函数具有下降性和全局收敛性．全局收敛性结果也被推广到了一种广义Wolfe型非精确线搜索．     

Expected Residual Minimization Formulation for a Class of Stochastic Vector Variational Inequalities

This paper considers a class of vector variational inequalities. First, we present an equivalent formulation, which is a scalar variational inequality, for the deterministic vector variational inequality. Then we concentrate on the stochastic circumstance. By noting that the stochastic vector variational inequality may not have a solution feasible for all realizations of the random variable in general, for tractability, we employ the expected residual minimization approach, which aims at minimizing the expected residual of the so-called regularized gap function. We investigate the properties of the expected residual minimization problem, and furthermore, we propose a sample average approximation method for solving the expected residual minimization problem. Comprehensive convergence analysis for the approximation approach is established as well.     

Random Gradient-Free Minimization of Convex Functions

In this paper, we prove new complexity bounds for methods of convex optimization based only on computation of the function value. The search directions of our schemes are normally distributed random Gaussian vectors. It appears that such methods usually need at most n times more iterations than the standard gradient methods, where n is the dimension of the space of variables. This conclusion is true for both nonsmooth and smooth problems. For the latter class, we present also an accelerated scheme with the expected rate of convergence \(O\Big ({n^2 \over k^2}\Big )\), where k is the iteration counter. For stochastic optimization, we propose a zero-order scheme and justify its expected rate of convergence \(O\Big ({n \over k^{1/2}}\Big )\). We give also some bounds for the rate of convergence of the random gradient-free methods to stationary points of nonconvex functions, for both smooth and nonsmooth cases. Our theoretical results are supported by preliminary computational experiments.     

A New Approximation of the Matrix Rank Function and Its Application to Matrix Rank Minimization

The matrix rank minimization problem is widely applied in many fields such as control, signal processing and system identification. However, the problem is NP-hard in general and is computationally hard to directly solve in practice. In this paper, we provide a new approximation function of the matrix rank function, and the corresponding approximation problems can be used to approximate the matrix rank minimization problem within any level of accuracy. Furthermore, the successive projected gradient method, which is designed based on the monotonicity and the Fréchet derivative of these new approximation function, can be used to solve the matrix rank minimization this problem by using the projected gradient method to find the stationary points of a series of approximation problems. Finally, the convergence analysis and the preliminary numerical results are given.     

A particle swarm pattern search method for bound constrained global optimization

In this paper we develop, analyze, and test a new algorithm for the global minimization of a function subject to simple bounds without the use of derivatives. The underlying algorithm is a pattern search method, more specifically a coordinate search method, which guarantees convergence to stationary points from arbitrary starting points. In the optional search phase of pattern search we apply a particle swarm scheme to globally explore the possible nonconvexity of the objective function. Our extensive numerical experiments showed that the resulting algorithm is highly competitive with other global optimization methods also based on function values. Support for Luís N. Vicente was provided by Centro de Matemática da Universidade de Coimbra and by FCT under grant POCI/MAT/59442/2004.     

Accelerated Non-Overlapping Domain Decomposition Method for Total Variation Minimization

We concern with fast domain decomposition methods for solving the total variation minimization problems in image processing. By decomposing the image domain into non-overlapping subdomains and interfaces, we consider the primal-dual problem on the interfaces such that the subdomain problems become independent problems and can be solved in parallel. Suppose both the interfaces and subdomain problems are uniformly convex, we can apply the acceleration method to achieve an $\mathcal{O}(1 / n^2)$ convergent domain decomposition algorithm. The convergence analysis is provided as well. Numerical results on image denoising, inpainting, deblurring, and segmentation are provided and comparison results with existing methods are discussed, which not only demonstrate the advantages of our method but also support the theoretical convergence rate.     

不可微规划的邻近控制簇算法

本文提出了一个带非线性约束的凸不可微规划的邻近控制簇算法，并给出了一种加权技术．在Ｓｌａｔｅｒ约束规格满足的条件下，证明了算法的整体收敛性．数字例子表明，该算法是处理该类问题的一种有效方法．     

Nonmonotone Strategy for Minimization of Quadratics with Simple Constraints

An algorithm for quadratic minimization with simple bounds is introduced, combining, as many well-known methods do, active set strategies and projection steps. The novelty is that here the criterion for acceptance of a projected trial point is weaker than the usual ones, which are based on monotone decrease of the objective function. It is proved that convergence follows as in the monotone case. Numerical experiments with bound-constrained quadratic problems from CUTE collection show that the modified method is in practice slightly more efficient than its monotone counterpart and has a performance superior to the well-known code LANCELOT for this class of problems.     

Complexity estimates of some cutting plane methods based on the analytic barrier

In this paper we establish the efficiency estimates for two cutting plane methods based on the analytic barrier. We prove that the rate of convergence of the second method is optimal uniformly in the number of variables. We present a modification of the second method. In this modified version each test point satisfies an approximate centering condition. We also use the standard strategy for updating approximate Hessians of the logarithmic barrier function. We prove that the rate of convergence of the modified scheme remains optimal and demonstrate that the number of Newton steps in the auxiliary minimization processes is bounded by an absolute constant. We also show that the approximate Hessian strategy significantly improves the total arithmetical complexity of the method.     

Hidden Convex Minimization

A class of nonconvex minimization problems can be classified as hidden convex minimization problems. A nonconvex minimization problem is called a hidden convex minimization problem if there exists an equivalent transformation such that the equivalent transformation of it is a convex minimization problem. Sufficient conditions that are independent of transformations are derived in this paper for identifying such a class of seemingly nonconvex minimization problems that are equivalent to convex minimization problems. Thus, a global optimality can be achieved for this class of hidden convex optimization problems by using local search methods. The results presented in this paper extend the reach of convex minimization by identifying its equivalent with a nonconvex representation.     

On the Convergence of Some Constrained Minimization Algorithms Based on Recursive Quadratic Programming

This paper considers the convergence of the method of recursiveequality quadratic programming (REQP) for constrained minimization.A theorem of Wolfe (1969) gives conditions on the search directionsand step lengths used by a minimization algorithm which ensurethat it will locate an unconstrained stationary point of a function.It is shown here that, under suitable circumstances, the iterationsof REQP satisfy these conditions both with respect to the conventionalpenalty function P(x, r) and also with respect to the augmentedpenalty function proposed by Fletcher (1969) which has a minimumat the solution to the constrained problem. The behaviour ofREQP in the neighbourhood of the solution is also considered,and it is shown that the algorithm is capable of superlinearconvergence.     

Strong Convergence of Projected Subgradient Methods for Nonsmooth and Nonstrictly Convex Minimization

In this paper, we establish a strong convergence theorem regarding a regularized variant of the projected subgradient method for nonsmooth, nonstrictly convex minimization in real Hilbert spaces. Only one projection step is needed per iteration and the involved stepsizes are controlled so that the algorithm is of practical interest. To this aim, we develop new techniques of analysis which can be adapted to many other non-Fejérian methods.     

采用不定Dogleg路径的非单调自适应信赖域算法解无约束极小化(英文)

In this paper, we combine the nonmonotone and adaptive techniques with trust region method for unconstrained minimization problems. We set a new ratio of the actual descent and predicted descent. Then, instead of the monotone sequence, the nonmonotone sequence of function values are employed. With the adaptive technique, the radius of trust region △k can be adjusted automatically to improve the efficiency of trust region methods. By means of the Bunch-Parlett factorization, we construct a method with indefinite dogleg path for solving the trust region subproblem which can handle the indefinite approximate Hessian Bk. The convergence properties of the algorithm are established. Finally, detailed numerical results are reported to show that our algorithm is efficient.     

关于DC函数最小化邻近点算法的注记

本文证明DC函数最小化问题邻近点算法的一个收敛性定理,并对此问题提出一类非精确邻近点算法．     

A New Direction Set Method for Unconstrained Minimization without Evaluating Derivatives

A new direction set method for unconstrained minimization withoutevaluating derivatives is presented. The algorithm can be regardedas an application to function minimization of Jacobi's methodfor determining the eigenvalues and eigenvectors of a real symmetricmatrix. Numerical results are presented, illustrating the performanceof the new algorithm on well-known test problems; a comparisonwith other methods is also given.     

线性l1模极小化问题的熵函数延拓法

本文通过利用极大熵函数构造同伦映射，建立了求解无约束线性l1模问题的熵函数延拓算法，证明了方法的收敛性，并给出了数值算例．     

求解变量带简单界约束的非线性规划问题的信赖域方法

1．引言。本文考虑下述变量带简单界约束的非线性规划问题：问题（1．1）不仅是实际应用中出现的简单的约束最优化问题，而且相当一部分最优化问题可以把变量限制在有意义的区间内181．因此，无论在理论方面还是在实际应用方面，都有必要研究此种问题．给出简便而且有效的算法．有些文章提出了一些特殊的方法．如011和［2］．14］及16］提出了一类信赖域方法，它们都借助于某种辅助点，证明了算法的全局收敛性．在收敛速度的分析方面，除要求在＊－T点满足严格互补松弛外，它们还要求另一个条件，即在每次迭代中，辅助点的有效约束必须在尝…     

Minimization methods for approximating tensors and their comparison

Application of various minimization methods to trilinear approximation of tensors is considered. These methods are compared based on numerical calculations. For the Gauss-Newton method, an efficient implementation is proposed, and the local rate of convergence is estimated for the case of completely symmetric tensors.     

Properties and numerical testing of a parallel global optimization algorithm

In the present paper, we have considered three methods with which to control the error in the homotopy analysis of elliptic differential equations and related boundary value problems, namely, control of residual errors, minimization of error functionals, and optimal homotopy selection through appropriate choice of auxiliary function H(x). After outlining the methods in general, we consider three applications. First, we apply the method of minimized residual error in order to determine optimal values of the convergence control parameter to obtain solutions exhibiting central symmetry for the Yamabe equation in three or more spatial dimensions. Secondly, we apply the method of minimizing error functionals in order to obtain optimal values of the convergnce control parameter for the homotopy analysis solutions to the Brinkman?CForchheimer equation. Finally, we carefully selected the auxiliary function H(x) in order to obtain an optimal homotopy solution for Liouville??s equation.