The relaxed inexact projection methods for the split feasibility problem

In this paper we present several relaxed inexact projection methods for the split feasibility problem (SFP). Each iteration of the first proposed algorithm consists of a projection onto a halfspace containing the given closed convex set. The algorithm can be implemented easily and its global convergence to the solution can be established under suitable conditions. Moreover,we present some modifications of the relaxed inexact projection method with constant stepsize by adopting Armijo-like search. We furthermore present a variable-step relaxed inexact projection method which does not require the computation of the matrix inverses and the largest eigenvalue of the matrix A^TA, and the objective function can decrease sufficiently at each iteration. We show convergence of these modified algorithms under mild conditions. Finally, we perform some numerical experiments, which show the behavior of the algorithms proposed.     

Weak convergence theorems of the modified relaxed projection algorithms for the split feasibility problem in Hilbert spaces

In this paper, we introduce a modified relaxed projection algorithm and a modified variable-step relaxed projection algorithm for the split feasibility problem in infinite-dimensional Hilbert spaces. The weak convergence theorems under suitable conditions are proved. Finally, some numerical results are presented, which show the advantage of the proposed algorithms.     

凸可行问题的平行近似次梯度投影算法

对凸可行问题提出了包括上松弛的平行近似次梯度投影算法和加速平行近似次梯度投影算法．与序列近似次梯度投影算法相比, 平行近似次梯度投影算法(每次迭代同时运用多个凸集的近似次梯度超平面上的投影)能够保证迭代序列收敛到离各个凸集最近的点． 上松弛的迭代技术和含有外推因子的加速技术的应用, 减少了数据存储量, 提高了收 敛速度. 最后在较弱的条件下证明了算法的收敛性, 数值实验结果验证了算法的有效性和优越性.     

Relaxed two points projection method for solving the multiple-sets split equality problem

The multiple-sets split equality problem, a generalization and extension of the split feasibility problem, has a variety of specific applications in real world, such as medical care, image reconstruction, and signal processing. It can be a model for many inverse problems where constraints are imposed on the solutions in the domains of two linear operators as well as in the operators’ ranges simultaneously. Although, for the split equality problem, there exist many algorithms, there are but few algorithms for the multiple-sets split equality problem. Hence, in this paper, we present a relaxed two points projection method to solve the problem; under some suitable conditions, we show the weak convergence and give a remark for the strong convergence method in the Hilbert space. The interest of our algorithm is that we transfer the problem to an optimization problem, then, based on the model, we present a modified gradient projection algorithm by selecting two different initial points in different sets for the problem (we call the algorithm as two points algorithm). During the process of iteration, we employ subgradient projections, not use the orthogonal projection, which makes the method implementable. Numerical experiments manifest the algorithm is efficient.     

A predictor-corrector algorithm for general variational inequalities

In this paper, we suggest a new predictor-corrector algorithm for solving general variational inequalities by using the auxiliary principle technique. The convergence of the proposed method only requires the partially relaxed strong monotonicity of the operator, which is weaker than co-coercivity. As special cases, we obtain a number of known and new results for solving various classes of variational inequalities.     

An inertial-like proximal algorithm for equilibrium problems

The paper concerns with an inertial-like algorithm for approximating solutions of equilibrium problems in Hilbert spaces. The algorithm is a combination around the relaxed proximal point method, inertial effect and the Krasnoselski–Mann iteration. The using of the proximal point method with relaxations has allowed us a more flexibility in practical computations. The inertial extrapolation term incorporated in the resulting algorithm is intended to speed up convergence properties. The main convergence result is established under mild conditions imposed on bifunctions and control parameters. Several numerical examples are implemented to support the established convergence result and also to show the computational advantage of our proposed algorithm over other well known algorithms.     

Accelerated reflection projection algorithm and its application to the LMI problem

We discuss accelerated version of the alternating projection method which can be applied to solve the linear matrix inequality (LMI) problem. The alternating projection method is a well-known algorithm for the convex feasibility problem, and has many generalizations and extensions. Bauschke and Kruk proposed a reflection projection algorithm for computing a point in the intersection of an obtuse cone and a closed convex set. We carry on this research in two directions. First, we present an accelerated version of the reflection projection algorithm, and prove its weak convergence in a Hilbert space; second, we prove the finite termination of an algorithm which is based on the proposed algorithm and provide an explicit upper bound for the required number of iterations under certain assumptions. Numerical experiments for the LMI problem are provided to demonstrate the effectiveness and merits of the proposed algorithms.     

A new self-adaptive CQ algorithm with an application to the LASSO problem

In this paper, we introduce a new self-adaptive CQ algorithm for solving split feasibility problems in real Hilbert spaces. The algorithm is designed, such that the stepsizes are directly computed at each iteration. We also consider the corresponding relaxed CQ algorithm for the proposed method. Under certain mild conditions, we establish weak convergence of the proposed algorithm as well as strong convergence of its hybrid-type variant. Finally, numerical examples illustrating the efficiency of our algorithm in solving the LASSO problem are presented.     

A new inertial-type hybrid projection-proximal algorithm for monotone inclusions

This paper investigates an enhanced proximal algorithm with interesting practical features and convergence properties for solving non-smooth convex minimization problems, or approximating zeroes of maximal monotone operators, in Hilbert spaces. The considered algorithm involves a recent inertial-type extrapolation technique, the use of enlargement of operators and also a recently proposed hybrid strategy, which combines inexact computation of the proximal iteration with a projection. Compared to other existing related methods, the resulting algorithm inherits the good convergence properties of the inertial-type extrapolation and the relaxed projection strategy. It also inherits the relative error tolerance of the hybrid proximal-projection method. As a special result, an update of inexact Newton-proximal method is derived and global convergence results are established.     

A Successive Projection Algorithm for Solving the Multiple-Sets Split Feasibility Problem

The multiple-sets split feasibility problem (MSFP) arises in many areas and it can be unified as a model for many inverse problems where the constraints are required on the solutions in the domain of a linear operator as well as in the operator's range. Some existing algorithms, in order to get the suitable step size, need to compute the largest eigenvalue of the related matrix, estimate the Lipschitz constant, or use some step-size search scheme, which usually requires many inner iterations. In this article, we introduce a successive projection algorithm for solving the multiple-sets split feasibility problem. In each iteration of this algorithm, the step size is directly computed, which is not needed to compute the largest eigenvalue of the matrix or estimate the Lipschitz constant. It also does not need any step-size search scheme. Its theoretical convergence results are also given.     

QR algorithm with two-sided Rayleigh quotient shifts

We introduce the two-sided Rayleigh quotient shift to the QR algorithm for non-Hermitian matrices to achieve a cubic local convergence rate. For the singly shifted case, the two-sided Rayleigh quotient iteration is incorporated into the QR iteration. A modified version of the method and its truncated version are developed to improve the efficiency. Based on the observation that the Francis double-shift QR iteration is related to a 2D Grassmann–Rayleigh quotient iteration, A doubly shifted QR algorithm with the two-sided 2D Grassmann–Rayleigh quotient double-shift is proposed. A modified version of the method and its truncated version are also developed. Numerical examples are presented to show the convergence behavior of the proposed algorithms. Numerical examples also show that the truncated versions of the modified methods outperform their counterparts including the standard Rayleigh quotient single-shift and the Francis double-shift.     

A new shrinking gradient-like projection method for equilibrium problems

The paper proposes a new shrinking gradient-like projection method for solving equilibrium problems. The algorithm combines the generalized gradient-like projection method with the monotone hybrid method. Only one optimization program is solved onto the feasible set at each iteration in our algorithm without any extra-step dealing with the feasible set. The absence of an optimization problem in the algorithm is explained by constructing slightly different cutting-halfspace in the monotone hybrid method. Theorem of strong convergence is established under standard assumptions imposed on equilibrium bifunctions. An application of the proposed algorithm to multivalued variational inequality problems (MVIP) is presented. Finally, another algorithm is introduced for MVIPs in which we only use a value of main operator at the current approximation to construct the next approximation. Some preliminary numerical experiments are implemented to illustrate the convergence and computational performance of our algorithms over others.     

Solving the split equality problem without prior knowledge of operator norms

The split equality problem has extraordinary utility and broad applicability in many areas of applied mathematics. Recently, Moudafi proposed an alternating CQ algorithm and its relaxed variant to solve it. However, to employ Moudafi’s algorithms, one needs to know a priori norm (or at least an estimate of the norm) of the bounded linear operators (matrices in the finite-dimensional framework). To estimate the norm of an operator is very difficult, but not an impossible task. It is the purpose of this paper to introduce a projection algorithm with a way of selecting the stepsizes such that the implementation of the algorithm does not need any priori information about the operator norms. We also practise this way of selecting stepsizes for variants of the projection algorithm, including a relaxed projection algorithm where the two closed convex sets are both level sets of convex functions, and a viscosity algorithm. Both weak and strong convergence are investigated.     

Golden ratio algorithms with new stepsize rules for variational inequalities

In this paper, we introduce two golden ratio algorithms with new stepsize rules for solving pseudomonotone and Lipschitz variational inequalities in finite dimensional Hilbert spaces. The presented stepsize rules allow the resulting algorithms to work without the prior knowledge of the Lipschitz constant of operator. The first algorithm uses a sequence of stepsizes that is previously chosen, diminishing, and nonsummable, while the stepsizes in the second one are updated at each iteration and by a simple computation. A special point is that the sequence of stepsizes generated by the second algorithm is separated from zero. The convergence and the convergence rate of the proposed algorithms are established under some standard conditions. Also, we give several numerical results to show the behavior of the algorithms in comparison with other algorithms.     

On generalized successive overrelaxation methods for augmented linear systems

For the augmented system of linear equations, Golub, Wu and Yuan recently studied an SOR-like method (BIT 41(2001)71–85). By further accelerating it with another parameter, in this paper we present a generalized SOR (GSOR) method for the augmented linear system. We prove its convergence under suitable restrictions on the iteration parameters, and determine its optimal iteration parameters and the corresponding optimal convergence factor. Theoretical analyses show that the GSOR method has faster asymptotic convergence rate than the SOR-like method. Also numerical results show that the GSOR method is more effective than the SOR-like method when they are applied to solve the augmented linear system. This GSOR method is further generalized to obtain a framework of the relaxed splitting iterative methods for solving both symmetric and nonsymmetric augmented linear systems by using the techniques of vector extrapolation, matrix relaxation and inexact iteration. Besides, we also demonstrate a complete version about the convergence theory of the SOR-like method. Subsidized by The Special Funds For Major State Basic Research Projects (No. G1999032803) and The National Natural Science Foundation (No. 10471146), P.R. China     

多集分裂等式问题的逐次松弛投影算法

多集分裂等式问题是分裂可行性问题的拓展问题,在图像重建、语言处理、地震探测等实际问题中具有广泛的应用.为了解决这个问题,提出了逐次松弛投影算法,设计了变化的步长,使其充分利用当前迭代点的信息且不需要算子范数的计算,证明了算法的弱收敛性.数值算例验证了算法在迭代次数与运行时间等方面的优越性.     

Generalized Krasnoselskii–Mann-type iterations for nonexpansive mappings in Hilbert spaces

The Krasnoselskii–Mann iteration plays an important role in the approximation of fixed points of nonexpansive operators; it is known to be weakly convergent in the infinite dimensional setting. In this present paper, we provide a new inexact Krasnoselskii–Mann iteration and prove weak convergence under certain accuracy criteria on the error resulting from the inexactness. We also show strong convergence for a modified inexact Krasnoselskii–Mann iteration under suitable assumptions. The convergence results generalize existing ones from the literature. Applications are given to the Douglas–Rachford splitting method, the Fermat–Weber location problem as well as the alternating projection method by John von Neumann.     

Remark on the Successive Projection Algorithm for the Multiple-Sets Split Feasibility Problem

Recently, assuming that the metric projection onto a closed convex set is easily calculated, Liu et al. (Numer. Func. Anal. Opt. 35:1459–1466, 2014) presented a successive projection algorithm for solving the multiple-sets split feasibility problem (MSFP). However, in some cases it is impossible or needs too much work to exactly compute the metric projection. The aim of this remark is to give a modification to the successive projection algorithm. That is, we propose a relaxed successive projection algorithm, in which the metric projections onto closed convex sets are replaced by the metric projections onto halfspaces. Clearly, the metric projection onto a halfspace may be directly calculated. So, the relaxed successive projection algorithm is easy to implement. Its theoretical convergence results are also given.     

A sparsity preserving stochastic gradient methods for sparse regression

We propose a new stochastic first-order algorithm for solving sparse regression problems. In each iteration, our algorithm utilizes a stochastic oracle of the subgradient of the objective function. Our algorithm is based on a stochastic version of the estimate sequence technique introduced by Nesterov (Introductory lectures on convex optimization: a basic course, Kluwer, Amsterdam, 2003). The convergence rate of our algorithm depends continuously on the noise level of the gradient. In particular, in the limiting case of noiseless gradient, the convergence rate of our algorithm is the same as that of optimal deterministic gradient algorithms. We also establish some large deviation properties of our algorithm. Unlike existing stochastic gradient methods with optimal convergence rates, our algorithm has the advantage of readily enforcing sparsity at all iterations, which is a critical property for applications of sparse regressions.     

Parareal Algorithms Applied to Stochastic Differential Equations with Conserved Quantities

In this paper, we couple the parareal algorithm with projection methods of the trajectory on a specific manifold, defined by the preservation of some conserved quantities of stochastic differential equations. First, projection methods are introduced as the coarse and fine propagators. Second, we apply the projection methods for systems with conserved quantities in the correction step of original parareal algorithm. Finally, three numerical experiments are performed by different kinds of algorithms to show the property of convergence in iteration, and preservation in conserved quantities of model systems.