Symmetric Alternating Direction Method with Indefinite Proximal Regularization for Linearly Constrained Convex Optimization

The proximal alternating direction method of multipliers is a popular and useful method for linearly constrained, separable convex problems, especially for the linearized case. In the literature, convergence of the proximal alternating direction method has been established under the assumption that the proximal regularization matrix is positive semi-definite. Recently, it was shown that the regularizing proximal term in the proximal alternating direction method of multipliers does not necessarily have to be positive semi-definite, without any additional assumptions. However, it remains unknown as to whether the indefinite setting is valid for the proximal version of the symmetric alternating direction method of multipliers. In this paper, we confirm that the symmetric alternating direction method of multipliers can also be regularized with an indefinite proximal term. We theoretically prove the global convergence of the indefinite method and establish its worst-case convergence rate in an ergodic sense. In addition, the generalized alternating direction method of multipliers proposed by Eckstein and Bertsekas is a special case in our discussion. Finally, we demonstrate the performance improvements achieved when using the indefinite proximal term through experimental results.     

求複數根的舗D法

<正> 一. 引言 代數方程f(x)=0的實數根的逐步接近法已有多種,其中計算簡單收斂最快的是用牛頓公式     

非正则线性增广拉格朗日乘子法的收敛速率研究CSCD

Recently,an indefinite linearized augmented Lagrangian method(IL-ALM)was proposed for the convex programming problems with linear constraints.The IL-ALM differs from the linearized augmented Lagrangian method in that the augmented Lagrangian is linearized by adding an indefinite quadratic proximal term.But,it preserves the algorithmic feature of the linearized ALM and usually has the advantage to improve the performance.The IL-ALM is proved to be convergent from contraction perspective,but its convergence rate is still missing.This is mainly because that the indefinite setting destroys the structures when we directly employ the contraction frameworks.In this paper,we derive the convergence rate for this algorithm by using a different analysis.We prove that a worst-case O(1/t)convergence rate is still hold for this algorithm,where t is the number of iterations.Additionally we show that the customized proximal point algorithm can employ larger step sizes by proving its equivalence to the linearized ALM.     

General fractional variational problem depending on indefinite integrals

In this report, we consider two kind of general fractional variational problem depending on indefinite integrals include unconstrained problem and isoperimetric problem. These problems can have multiple dependent variables, multiorder fractional derivatives, multiorder integral derivatives and boundary conditions. For both problems, we obtain the Euler-Lagrange type necessary conditions which must be satisfied for the given functional to be extremum. Also, we apply the Rayleigh-Ritz method for solving the unconstrained general fractional variational problem depending on indefinite integrals. By this method, the given problem is reduced to the problem for solving a system of algebraic equations using shifted Legendre polynomials basis functions. An approximate solution for this problem is obtained by solving the system. We discuss the analytic convergence of this method and finally by some examples will be showing the accurately and applicability for this technique.     

Über die näherungsweise Lösung nichtlinearer Integralgleichungen

Summary In this paper we study the iterative solvability of nonlinear systems of equations which arise from the discretization of Hammerstein integral equations. It is shown that, for a large class of equations satisfying monotonicity assumptions, it is possible to solve these systems by means of a linearly convergent iteration method. Moreover, for general monotone operators on a Hilbert space a globally convergent variant of Newton's method is given. Finally it is shown that this method effectively can be applied in a natural way to the systems of equations under consideration.     

The Impact of Equal Weighting of Low- and High-Confidence Observations on Robust Linear Regression Computations

Equal weighting of low- and high-confidence observations occurs for Huber, Talwar, and Barya weighting functions when Newton's method is used to solve robust linear regression problems. This leads to easy updates and/or downdates of existing matrix factorizations or easy computation of coefficient matrices in linear systems from previous ones. Thus Newton's method based on these functions has been shown to be computationally cheap. In this paper we show that a combination of Newton's method and an iterative method is a promising approach for solving robust linear regression problems. We show that Newton's method based on the Talwar function is an active set method. Further we show that it is possible to obtain improved estimates of the solution vector by combining a line search method like Newton's method with an active set method.This revised version was published online in October 2005 with corrections to the Cover Date.     

Uniform convergence of the rectangle method for singular integral equation with the Hölder density

We study an approximate method for solving singular integral equations. It implies an approximation of a singular operator by means of a compound quadrature formula similar to the rectangle one. The corresponding systems of linear algebraic equations are solvable if so is the integral equation, while its coefficients satisfy the strong ellipticity condition. Under these restrictions we obtain a bound for the rate of convergence of solutions of systems of linear equations to the solution of the considered integral equation in the uniform vector norm.     

An interior-point method for semi-infinite programming problems

This work examines the generalization of a certain interior-point method, namely the method of analytic centers, to semi-infinite linear programming problems. We define an analytic center for these problems and an appropriate norm to examine Newton's method for computing this center. A simple algorithm of order zero is constructed and a convergence proof for that algorithm is given. Finally, we describe a more practical implementation of a predictor-corrector method and give some numerical results. In particular we concentrate on practical integration rules that take care of the specific structure of the integrals.     

Gauss-Newton法的收敛球与解的唯一性球

0　引　言本文研究非线性最小二乘问题min F( x)∶ =12 f( x) Tf ( x) ( EP)的 Gauss-Newton法的局部收敛性 ,其中 f:Rn→ Rm是 Frechet可微的 ,m≥ n.非线性最小二乘问题在数据拟合 ,参数估计和函数逼近等方面有广泛的应用 .在工程应用中也起到很大作用 ,例如在神经网络中 ,对小波问题 ,FP网络等方面的数据 (图形 )传输 ,数据 (图形 )压缩等方面有极其重要的理论和实际意义 .目前 ,求解最小二乘问题的最基本的方法之一是 Gauss-Newton法 [1 ]xn+1 =xn -[f′( xn) Tf′( x) ] - 1 f′( xn) Tf( xn) . ( GN)就我们所知 ,目前关于 Gau…     

Global convergence of some differential equation algorithms for solving equations involving positive variables

This paper describes some sufficient conditions for global convergence in five differential equation algorithms for solving systems of non-linear algebraic equations involving positive variables. The algorithms are continuous time versions of a modified steepest descent method, Newton's method, a modified Newton's method and two algorithms using the transpose of the Jacobian in place of the inverse of the Jacobian in Newton's method. It is shown that under a set of mildly restrictive conditions the Jacobian transpose algorithm has qualitatively the same convergence as Newton's method.