Solving the split equality problem without prior knowledge of operator norms

The split equality problem has extraordinary utility and broad applicability in many areas of applied mathematics. Recently, Moudafi proposed an alternating CQ algorithm and its relaxed variant to solve it. However, to employ Moudafi’s algorithms, one needs to know a priori norm (or at least an estimate of the norm) of the bounded linear operators (matrices in the finite-dimensional framework). To estimate the norm of an operator is very difficult, but not an impossible task. It is the purpose of this paper to introduce a projection algorithm with a way of selecting the stepsizes such that the implementation of the algorithm does not need any priori information about the operator norms. We also practise this way of selecting stepsizes for variants of the projection algorithm, including a relaxed projection algorithm where the two closed convex sets are both level sets of convex functions, and a viscosity algorithm. Both weak and strong convergence are investigated.     

基于交替投影算法求解单变量线性约束矩阵方程问题

研究如下线性约束矩阵方程求解问题:给定A∈R~(m×n),B∈R~(n×p)和C∈R~(m×p),求矩阵X∈R(?)R~(n×n)"使得A×B=C以及相应的最佳逼近问题,其中集合R为如对称阵,Toeplitz阵等构成的线性子空间,或者对称半(ε)正定阵,(对称)非负阵等构成的闭凸集.给出了在相容条件下求解该问题的交替投影算法及算法收敛性分析.通过大量数值算例说明该算法的可行性和高效性,以及该算法较传统的矩阵形式的Krylov子空间方法(可行前提下)在迭代效率上的明显优势,本文也通过寻求加速技巧进一步提高算法的收敛速度.     

Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems

In this paper, we study the generalized Douglas–Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas–Rachford algorithm and the alternating projection algorithm. Specifically, we establish several local linear convergence results for the algorithm in solving feasibility problems with finitely many closed possibly nonconvex sets under different assumptions. Our findings not only relax some regularity conditions but also improve linear convergence rates in the literature. In the presence of convexity, the linear convergence is global.     

Numerical methods for solving some matrix feasibility problems

In this paper, we design two numerical methods for solving some matrix feasibility problems, which arise in the quantum information science. By making use of the structured properties of linear constraints and the minimization theorem of symmetric matrix on manifold, the projection formulas of a matrix onto the feasible sets are given, and then the relaxed alternating projection algorithm and alternating projection algorithm on manifolds are designed to solve these problems. Numerical examples show that the new methods are feasible and effective.     

Additional Results on Convergence of Alternating Iterations Involving Rectangular Matrices

Different classes of matrix splittings of semi-monotone matrices lead to many comparison results, which are useful tools in examining the convergence rate of iterative methods for solving rectangular systems of linear equations in a faster way. In this context, the theory of alternating iterations for rectangular matrices was introduced recently, in order to arrive at the desired solution of accuracy or at the exact solution. In this article, we expand convergence theory of such alternating iterations and obtain comparison results for such iterations.     

Local linear convergence of approximate projections onto regularized sets

The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in question come from inexact measurements that are matched to idealized models. It is unlikely that any such problems in applications will enjoy metrically regular intersection, let alone set intersection. We explore a regularization strategy that generates an intersection with the desired regularity properties. The regularization, however, can lead to a significant increase in computational complexity. In a further refinement, we investigate and prove linear convergence of an approximate alternating projection algorithm. The analysis provides a regularization strategy that fits naturally with many ill-posed inverse problems, and a mathematically sound stopping criterion for extrapolated, approximate algorithms. The theory is demonstrated on the phase retrieval problem with experimental data. The conventional early termination applied in practice to unregularized, consistent problems in diffraction imaging can be justified fully in the framework of this analysis providing, for the first time, proof of convergence of alternating approximate projections for finite dimensional, consistent phase retrieval problems.     

A Perturbation approach for an inverse quadratic programming problem

We consider an inverse quadratic programming (QP) problem in which the parameters in both the objective function and the constraint set of a given QP problem need to be adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a linear complementarity constrained minimization problem with a positive semidefinite cone constraint. With the help of duality theory, we reformulate this problem as a linear complementarity constrained semismoothly differentiable (SC¹) optimization problem with fewer variables than the original one. We propose a perturbation approach to solve the reformulated problem and demonstrate its global convergence. An inexact Newton method is constructed to solve the perturbed problem and its global convergence and local quadratic convergence rate are shown. As the objective function of the problem is a SC¹ function involving the projection operator onto the cone of positively semi-definite symmetric matrices, the analysis requires an implicit function theorem for semismooth functions as well as properties of the projection operator in the symmetric-matrix space. Since an approximate proximal point is required in the inexact Newton method, we also give a Newton method to obtain it. Finally we report our numerical results showing that the proposed approach is quite effective.     

EXPERIMENTAL STUDY OF THE ASYNCHRONOUS MULTISPLITTING RELAXATION METHODS FOR THE LINEAR COMPLEMENTARITY PROBLEMS

We study the numerical behaviours of the relaxed asynchronous multisplitting methods for the linear complementarity problems by solving some typical problems from practical applications on a real multiprocessor system. Numerical results show that the parallel multisplitting relaxation methods always perform much better than the corresponding sequential alternatives, and that the asynchronous multisplitting relaxation methods often outperform their corresponding synchronous counterparts. Moreover, the two-sweep relaxed multisplitting methods have better convergence properties than their corresponding one-sweep relaxed ones in the sense that they have larger convergence domains and faster convergence speeds. Hence, the asynchronous multisplitting unsymmetric relaxation iterations should be the methods of choice for solving the large sparse linear complementarity problems in the parallel computing environments.     

Sequential location of two facilities: comparing random to optimal location of the first facility

We review our recent results in the development of optimal algorithms for the minimization of a strictly convex quadratic function subject to separable convex inequality constraints and/or linear equality constraints. A unique feature of our algorithms is the theoretically supported bound on the rate of convergence in terms of the bounds on the spectrum of the Hessian of the cost function, independent of representation of the constraints. When applied to the class of convex QP or QPQC problems with the spectrum in a given positive interval and a sparse Hessian matrix, the algorithms enjoy optimal complexity, i.e., they can find an approximate solution at the cost that is proportional to the number of unknowns. The algorithms do not assume representation of the linear equality constraints by full rank matrices. The efficiency of our algorithms is demonstrated by the evaluation of the projection of a point to the intersection of the unit cube and unit sphere with hyperplanes.     

Generalized system for relaxed cocoercive variational inequalities in Hilbert spaces

The approximate solvability of a generalized system for relaxed cocoercive nonlinear variational inequality in Hilbert spaces is studied, based on the convergence of projection methods. The results presented in this paper extend and improve the main results of [R.U. Verma, Generalized system for relaxed cocoercive variational inequalities and its projection methods, J. Optim. Theory Appl. 121 (1) (2004) 203–210; R.U. Verma, Generalized class of partial relaxed monotonicity and its connections, Adv. Nonlinear Var. Inequal. 7 (2) (2004) 155–164; R.U. Verma, General convergence analysis for two-step projection methods and applications to variational problems, Appl. Math. Lett. 18 (11) (2005) 1286–1292; N.H. Xiu, J.Z. Zhang, Local convergence analysis of projection type algorithms: Unified approach, J. Optim. Theory Appl. 115 (2002) 211–230; H. Nie, Z. Liu, K.H. Kim, S.M. Kang, A system of nonlinear variational inequalities involving strongly monotone and pseudocontractive mappings, Adv. Nonlinear Var. Inequal. 6 (2) (2003) 91–99].     

A time-domain decomposition iterative method for the solution of distributed linear quadratic optimal control problems

We study a class of time-domain decomposition-based methods for the numerical solution of large-scale linear quadratic optimal control problems. Our methods are based on a multiple shooting reformulation of the linear quadratic optimal control problem as a discrete-time optimal control (DTOC) problem. The optimality conditions for this DTOC problem lead to a linear block tridiagonal system. The diagonal blocks are invertible and are related to the original linear quadratic optimal control problem restricted to smaller time-subintervals. This motivates the application of block Gauss–Seidel (GS)-type methods for the solution of the block tridiagonal systems. Numerical experiments show that the spectral radii of the block GS iteration matrices are larger than one for typical applications, but that the eigenvalues of the iteration matrices decay to zero fast. Hence, while the GS method is not expected to convergence for typical applications, it can be effective as a preconditioner for Krylov-subspace methods. This is confirmed by our numerical tests.A byproduct of this research is the insight that certain instantaneous control techniques can be viewed as the application of one step of the forward block GS method applied to the DTOC optimality system.     

Convergent nonnegative matrices and iterative methods for consistent linear systems

Summary In this paper we study linear stationary iterative methods with nonnegative iteration matrices for solving singular and consistent systems of linear equationsAx=b. The iteration matrices for the schemes are obtained via regular and weak regular splittings of the coefficients matrixA. In certain cases when only some necessary, but not sufficient, conditions for the convergence of the iterations schemes exist, we consider a transformation on the iteration matrices and obtain new iterative schemes which ensure convergence to a solution toAx=b. This transformation is parameter-dependent, and in the case where all the eigenvalues of the iteration matrix are real, we show how to choose this parameter so that the asymptotic convergence rate of the new schemes is optimal. Finally, some applications to the problem of computing the stationary distribution vector for a finite homogeneous ergodic Markov chain are discussed.Research sponsored in part by US Army Research Office     

On generalized successive overrelaxation methods for augmented linear systems

For the augmented system of linear equations, Golub, Wu and Yuan recently studied an SOR-like method (BIT 41(2001)71–85). By further accelerating it with another parameter, in this paper we present a generalized SOR (GSOR) method for the augmented linear system. We prove its convergence under suitable restrictions on the iteration parameters, and determine its optimal iteration parameters and the corresponding optimal convergence factor. Theoretical analyses show that the GSOR method has faster asymptotic convergence rate than the SOR-like method. Also numerical results show that the GSOR method is more effective than the SOR-like method when they are applied to solve the augmented linear system. This GSOR method is further generalized to obtain a framework of the relaxed splitting iterative methods for solving both symmetric and nonsymmetric augmented linear systems by using the techniques of vector extrapolation, matrix relaxation and inexact iteration. Besides, we also demonstrate a complete version about the convergence theory of the SOR-like method. Subsidized by The Special Funds For Major State Basic Research Projects (No. G1999032803) and The National Natural Science Foundation (No. 10471146), P.R. China     

A priori error estimates for optimal control problems with pointwise constraints on the gradient of the state

We analyze a finite element approximation of an elliptic optimal control problem with pointwise bounds on the gradient of the state variable. We derive convergence rates if the control space is discretized implicitly by the state equation. In contrast to prior work we obtain these results directly from classical results for the W ^1,∞-error of the finite element projection, without using adjoint information. If the control space is discretized directly, we first prove a regularity result for the optimal control to control the approximation error, based on which we then obtain analogous convergence rates.     

Convergence rate analysis of iteractive algorithms for solving variational inequality problems

 We present a unified convergence rate analysis of iterative methods for solving the variational inequality problem. Our results are based on certain error bounds; they subsume and extend the linear and sublinear rates of convergence established in several previous studies. We also derive a new error bound for $\gamma$-strictly monotone variational inequalities. The class of algorithms covered by our analysis in fairly broad. It includes some classical methods for variational inequalities, e.g., the extragradient, matrix splitting, and proximal point methods. For these methods, our analysis gives estimates not only for linear convergence (which had been studied extensively), but also sublinear, depending on the properties of the solution. In addition, our framework includes a number of algorithms to which previous studies are not applicable, such as the infeasible projection methods, a separation-projection method, (inexact) hybrid proximal point methods, and some splitting techniques. Finally, our analysis covers certain feasible descent methods of optimization, for which similar convergence rate estimates have been recently obtained by Luo [14]. Received: April 17, 2001 / Accepted: December 10, 2002 Published online: April 10, 2003 RID="⋆" ID="⋆" Research of the author is partially supported by CNPq Grant 200734/95–6, by PRONEX-Optimization, and by FAPERJ. Key Words. Variational inequality – error bound – rate of convergence Mathematics Subject Classification (2000): 90C30, 90C33, 65K05     

On relaxed greedy randomized Kaczmarz methods for solving large sparse linear systems

For solving large sparse systems of linear equations by iteration methods, we further generalize the greedy randomized Kaczmarz method by introducing a relaxation parameter in the involved probability criterion, obtaining a class of relaxed greedy randomized Kaczmarz methods. We prove the convergence of these methods when the linear system is consistent, and show that these methods can be more efficient than the greedy randomized Kaczmarz method if the relaxation parameter is chosen appropriately.     

Accelerated reflection projection algorithm and its application to the LMI problem

We discuss accelerated version of the alternating projection method which can be applied to solve the linear matrix inequality (LMI) problem. The alternating projection method is a well-known algorithm for the convex feasibility problem, and has many generalizations and extensions. Bauschke and Kruk proposed a reflection projection algorithm for computing a point in the intersection of an obtuse cone and a closed convex set. We carry on this research in two directions. First, we present an accelerated version of the reflection projection algorithm, and prove its weak convergence in a Hilbert space; second, we prove the finite termination of an algorithm which is based on the proposed algorithm and provide an explicit upper bound for the required number of iterations under certain assumptions. Numerical experiments for the LMI problem are provided to demonstrate the effectiveness and merits of the proposed algorithms.     

A Modified Relaxation Scheme for Mathematical Programs with Complementarity Constraints

In this paper, we consider a mathematical program with complementarity constraints. We present a modified relaxed program for this problem, which involves less constraints than the relaxation scheme studied by Scholtes (2000). We show that the linear independence constraint qualification holds for the new relaxed problem under some mild conditions. We also consider a limiting behavior of the relaxed problem. We prove that any accumulation point of stationary points of the relaxed problems is C-stationary to the original problem under the MPEC linear independence constraint qualification and, if the Hessian matrices of the Lagrangian functions of the relaxed problems are uniformly bounded below on the corresponding tangent space, it is M-stationary. We also obtain some sufficient conditions of B-stationarity for a feasible point of the original problem. In particular, some conditions described by the eigenvalues of the Hessian matrices mentioned above are new and can be verified easily. This work was supported in part by the Scientific Research Grant-in-Aid from the Ministry of Education, Science, Sports, and Culture of Japan. The authors are grateful to an anonymous referee for critical comments.     

关于Schwarz交替法的收敛因子

§1.Schwarz交替法的收敛因子 我们就二阶自共轭椭圆型方程的Dirichlet问题来讨论.设Ω?R~2为一多边形区域, a(u,v)=(f,v),v∈H_0~1(Ω),f∈H~(-1)(Ω), u∈H_0~1(Ω)是定义在其上的边值问题的变分形式,双线性型时a(·,·)满足     

Unconstrained Optimization Techniques for the Acceleration of Alternating Projection Methods

Alternating projection methods have been extensively used to find the closest point, to a given point, in the intersection of several given sets that belong to a Hilbert space. One of the characteristics of these schemes is the slow convergence that can be observed in practical applications. To overcome this difficulty, several techniques, based on different ideas, have been developed to accelerate their convergence. Recently, a successful acceleration scheme was developed specially for Cimmino's method when applied to the solution of large-scale saddle point problems. This specialized acceleration scheme is based on the use of the well-known conjugate gradient method for minimizing a related convex quadratic map. In this work, we extend and further analyze this optimization approach for several alternating projection methods on different scenarios. In particular, we include a specialized analysis and treatment for the acceleration of von Neumann-Halperin's method and Cimmino's method on subspaces, and Kaczmarz method on linear varieties. For some specific applications we illustrate the advantages of our acceleration schemes with encouraging numerical experiments.