An implementable proximal point algorithmic framework for nuclear norm minimization

The nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints. Such a problem often arises from the convex relaxation of a rank minimization problem with noisy data, and arises in many fields of engineering and science. In this paper, we study inexact proximal point algorithms in the primal, dual and primal-dual forms for solving the nuclear norm minimization with linear equality and second order cone constraints. We design efficient implementations of these algorithms and present comprehensive convergence results. In particular, we investigate the performance of our proposed algorithms in which the inner sub-problems are approximately solved by the gradient projection method or the accelerated proximal gradient method. Our numerical results for solving randomly generated matrix completion problems and real matrix completion problems show that our algorithms perform favorably in comparison to several recently proposed state-of-the-art algorithms. Interestingly, our proposed algorithms are connected with other algorithms that have been studied in the literature.     

有奇异解的无约束最优化问题的一种混合法

本文研究求解含有奇异解的无约束最优化问题算法 .该类问题的一个重要特性是目标函数的Hessian阵可能处处奇异 .我们提出求解该类问题的一种梯度 -正则化牛顿型混合算法 .并在一定的条件下得到了算法的全局收敛性 .而且 ,经一定迭代步后 ,算法还原为正则化 Newton法 .因而 ,算法具有局部二次收敛性 .     

Extension of the Barzilai–Borwein Method for Quadratic Forms in Finite Euclidean Spaces

In this paper, we present an extension of the Barzilai–Borwein method for the minimization of a quadratic form defined in a finite-dimensional real Euclidean space. We present a convergence analysis of the proposed method. We also present some applications in the resolution of linear matrix equations such as the Sylvester equation, which are of great interest in control theory and other engineering disciplines. Our numerical results indicate that the new method competes satisfactorily in the resolution of linear matrix equations with the function lyap of the Toolbox of Control of MATLAB.     

一类不可微二次规划逆问题

本文求解了一类二次规划的逆问题,具体为目标函数是矩阵谱范数与向量无穷范数之和的最小化问题.首先将该问题转化为目标函数可分离变量的凸优化问题,提出用G-ADMM法求解.并结合奇异值阈值算法,Moreau-Yosida正则化算法,matlab优化工具箱的quadprog函数来精确求解相应的子问题.而对于其中一个子问题的精确...     

Fuzzy guaranteed cost controller for trajectory tracking in nonlinear systems

In this paper, we propose a fuzzy logic based guaranteed cost controller for trajectory tracking in nonlinear systems. Takagi–Sugeno (T–S) fuzzy model is used to represent the dynamics of a nonlinear system and the controller design is carried out using this fuzzy model. State feedback law is used for building the fuzzy controller whose performance is evaluated using a quadratic cost function. For designing the fuzzy logic based controller which satisfies guaranteed performance, linear matrix inequality (LMI) approach is used. Sufficient conditions are derived in terms of matrix inequalities for minimizing the performance function of the controller. The performance function minimization problem with polynomial matrix inequalities is then transformed into a problem of minimizing a convex performance function involving standard LMIs. This minimization problem can be solved easily and efficiently using the LMI optimization techniques. Our controller design method also ensures that the closed-loop system is asymptotically stable. Simulation study is carried out on a two-link robotic manipulator tracking a reference trajectory. From the results of the simulation study, it is observed that our proposed controller tracks the reference trajectory closely while maintaining a guaranteed minimum cost.     

A New Approximation of the Matrix Rank Function and Its Application to Matrix Rank Minimization

The matrix rank minimization problem is widely applied in many fields such as control, signal processing and system identification. However, the problem is NP-hard in general and is computationally hard to directly solve in practice. In this paper, we provide a new approximation function of the matrix rank function, and the corresponding approximation problems can be used to approximate the matrix rank minimization problem within any level of accuracy. Furthermore, the successive projected gradient method, which is designed based on the monotonicity and the Fréchet derivative of these new approximation function, can be used to solve the matrix rank minimization this problem by using the projected gradient method to find the stationary points of a series of approximation problems. Finally, the convergence analysis and the preliminary numerical results are given.     

Splitting and linearizing augmented Lagrangian algorithm for subspace recovery from corrupted observations

Given a set of corrupted data drawn from a union of multiple subspace, the subspace recovery problem is to segment the data into their respective subspace and to correct the possible noise simultaneously. Recently, it is discovered that the task can be characterized, both theoretically and numerically, by solving a matrix nuclear-norm and a ?_2,1-mixed norm involved convex minimization problems. The minimization model actually has separable structure in both the objective function and constraint; it thus falls into the framework of the augmented Lagrangian alternating direction approach. In this paper, we propose and investigate an augmented Lagrangian algorithm. We split the augmented Lagrangian function and minimize the subproblems alternatively with one variable by fixing the other one. Moreover, we linearize the subproblem and add a proximal point term to easily derive the closed-form solutions. Global convergence of the proposed algorithm is established under some technical conditions. Extensive experiments on the simulated and the real data verify that the proposed method is very effective and faster than the sate-of-the-art algorithm LRR.     

Low Tucker rank tensor completion using a symmetric block coordinate descent method

Low Tucker rank tensor completion has wide applications in science and engineering. Many existing approaches dealt with the Tucker rank by unfolding matrix rank. However, unfolding a tensor to a matrix would destroy the data's original multi-way structure, resulting in vital information loss and degraded performance. In this article, we establish a relationship between the Tucker ranks and the ranks of the factor matrices in Tucker decomposition. Then, we reformulate the low Tucker rank tensor completion problem as a multilinear low rank matrix completion problem. For the reformulated problem, a symmetric block coordinate descent method is customized. For each matrix rank minimization subproblem, the classical truncated nuclear norm minimization is adopted. Furthermore, temporal characteristics in image and video data are introduced to such a model, which benefits the performance of the method. Numerical simulations illustrate the efficiency of our proposed models and methods.     

New Robust Model Predictive Control for Uncertain Systems with Input Constraints Using Relaxation Matrices

In this paper, we propose a new robust model predictive control (MPC) method for time-varying uncertain systems with input constraints. We formulate the problem as a minimization of the worst-case finite-horizon cost function subject to a new sufficient condition for cost monotonicity. The proposed MPC technique uses relaxation matrices to derive a less conservative terminal inequality condition. The relaxation matrices improve feasibility and system performance. The optimization problem is solved by semidefinite programming involving linear matrix inequalities (LMIs). A numerical example shows the effectiveness of the proposed method. The authors thank the associate editor and two anonymous referees for careful reading and useful suggestions.     

On the low rank solution of the Q‐weighted nearest correlation matrix problem

The low rank solution of the Q‐weighted nearest correlation matrix problem is studied in this paper. Based on the property of Q‐weighted norm and the Gramian representation, we first reformulate the Q‐weighted nearest correlation matrix problem as a minimization problem of the trace function with quadratic constraints and then prove that the solution of the minimization problem the trace function is the stationary point of the original problem if it is rank‐deficient. Finally, the nonmonotone spectral projected gradient method is constructed to solve them. Numerical examples illustrate that the new method is feasible and effective. Copyright © 2015 John Wiley & Sons, Ltd.     

A Riemannian variant of the Fletcher–Reeves conjugate gradient method for stochastic inverse eigenvalue problems with partial eigendata

In this paper, we focus on the stochastic inverse eigenvalue problem with partial eigendata of constructing a stochastic matrix from the prescribed partial eigendata. A Riemannian variant of the Fletcher–Reeves conjugate gradient method is proposed for solving a general unconstrained minimization problem on a Riemannian manifold, and the corresponding global convergence is established under some assumptions. Then, we reformulate the inverse problem as a nonlinear least squares problem over a matrix oblique manifold, and the application of the proposed geometric method to the nonlinear least squares problem is investigated. The proposed geometric method is also applied to the case of prescribed entries and the case of column stochastic matrix. Finally, some numerical tests are reported to illustrate that the proposed geometric method is effective for solving the inverse problem.     

Convergence of Fixed-Point Continuation Algorithms for Matrix Rank Minimization

The matrix rank minimization problem has applications in many fields, such as system identification, optimal control, low-dimensional embedding, etc. As this problem is NP-hard in general, its convex relaxation, the nuclear norm minimization problem, is often solved instead. Recently, Ma, Goldfarb and Chen proposed a fixed-point continuation algorithm for solving the nuclear norm minimization problem (Math. Program., doi:, 2009). By incorporating an approximate singular value decomposition technique in this algorithm, the solution to the matrix rank minimization problem is usually obtained. In this paper, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving affinely constrained matrix rank minimization problems are reported.     

A modified quasi‐Newton diagonal update algorithm for total variation denoising problems and nonlinear monotone equations with applications in compressive sensing

In this paper, we present a new algorithm to accelerate the Chambolle gradient projection method for total variation image restoration. The new proposed method considers an approximation of the Hessian based on the secant equation. Combined with the quasi‐Cauchy equations and diagonal updating, we can obtain a positive definite diagonal matrix. In the proposed minimization method model, we use the positive definite diagonal matrix instead of the constant time stepsize in Chambolle's method. The global convergence of the proposed scheme is proved. Some numerical results illustrate the efficiency of this method. Moreover, we also extend the quasi‐Newton diagonal updating method to solve nonlinear systems of monotone equations. Performance comparisons show that the proposed method is efficient. A practical application of the monotone equations is shown and tested on sparse signal reconstruction in compressed sensing. Copyright © 2015 John Wiley & Sons, Ltd.     

Convergence analysis of projected gradient descent for Schatten-p nonconvex matrix recovery

The matrix rank minimization problem arises in many engineering applications. As this problem is NP-hard, a nonconvex relaxation of matrix rank minimization, called the Schatten-p quasi-norm minimization(0 p 1), has been developed to approximate the rank function closely. We study the performance of projected gradient descent algorithm for solving the Schatten-p quasi-norm minimization(0 p 1) problem.Based on the matrix restricted isometry property(M-RIP), we give the convergence guarantee and error bound for this algorithm and show that the algorithm is robust to noise with an exponential convergence rate.     

An alternating direction method for linear‐constrained matrix nuclear norm minimization

The aim of the nuclear norm minimization problem is to find a matrix that minimizes the sum of its singular values and satisfies some constraints simultaneously. Such a problem has received more attention largely because it is closely related to the affine rank minimization problem, which appears in many control applications including controller design, realization theory, and model reduction. In this paper, we first propose an exact version alternating direction method for solving the nuclear norm minimization problem with linear equality constraints. At each iteration, the method involves a singular value thresholding and linear matrix equations which are solved exactly. Convergence of the proposed algorithm is followed directly. To broaden the capacity of solving larger problems, we solve approximately the subproblem by an iterative method with the Barzilai–Borwein steplength. Some extensions to the noisy problems and nuclear norm regularized least‐square problems are also discussed. Numerical experiments and comparisons with the state‐of‐the‐art method FPCA show that the proposed method is effective and promising. Copyright © 2011 John Wiley & Sons, Ltd.     

Dykstra's Algorithm for a Constrained Least-squares Matrix Problem

We apply Dykstra's alternating projection algorithm to the constrained least-squares matrix problem that arises naturally in statistics and mathematical economics. In particular, we are concerned with the problem of finding the closest symmetric positive definite bounded and patterned matrix, in the Frobenius norm, to a given matrix. In this work, we state the problem as the minimization of a convex function over the intersection of a finite collection of closed and convex sets in the vector space of square matrices. We present iterative schemes that exploit the geometry of the problem, and for which we establish convergence to the unique solution. Finally, we present preliminary numberical results to illustrate the performance of the proposed iterative methods.     

低秩稀疏矩阵恢复的快速非单调交替极小化方法

针对低秩稀疏矩阵恢复问题的一个非凸优化模型,本文提出了一种快速非单调交替极小化方法.主要思想是对低秩矩阵部分采用交替极小化方法,对稀疏矩阵部分采用非单调线搜索技术来分别进行迭代更新.非单调线搜索技术是将单步下降放宽为多步下降,从而提高了计算效率.文中还给出了新算法的收敛性分析.最后,通过数值实验的比较表明,矩阵恢复的非单调交替极小化方法比原单调类方法更有效.     

Nonlinear Proximal Decomposition Method for Convex Programming

In this paper, we propose a new decomposition method for solving convex programming problems with separable structure. The proposed method is based on the decomposition method proposed by Chen and Teboulle and the nonlinear proximal point algorithm using the Bregman function. An advantage of the proposed method is that, by a suitable choice of the Bregman function, each subproblem becomes essentially the unconstrained minimization of a finite-valued convex function. Under appropriate assumptions, the method is globally convergent to a solution of the problem.     

Sparse Steinian Covariance Estimation

We consider a new method for sparse covariance matrix estimation which is motivated by previous results for the so-called Stein-type estimators. Stein proposed a method for regularizing the sample covariance matrix by shrinking together the eigenvalues; the amount of shrinkage is chosen to minimize an unbiased estimate of the risk (UBEOR) under the entropy loss function. The resulting estimator has been shown in simulations to yield significant risk reductions over the maximum likelihood estimator. Our method extends the UBEOR minimization problem by adding an ?₁ penalty on the entries of the estimated covariance matrix, which encourages a sparse estimate. For a multivariate Gaussian distribution, zeros in the covariance matrix correspond to marginal independences between variables. Unlike the ?₁-penalized Gaussian likelihood function, our penalized UBEOR objective is convex and can be minimized via a simple block coordinate descent procedure. We demonstrate via numerical simulations and an analysis of microarray data from breast cancer patients that our proposed method generally outperforms other methods for sparse covariance matrix estimation and can be computed efficiently even in high dimensions.     

Properties of Restricted NCP Functions for Nonlinear Complementarity Problems

In this paper, we study restricted NCP functions which may be used to reformulate the nonlinear complementarity problem as a constrained minimization problem. In particular, we consider three classes of restricted NCP functions, two of them introduced by Solodov and the other proposed in this paper. We give conditions under which a minimization problem based on a restricted NCP function enjoys favorable properties, such as equivalence between a stationary point of the minimization problem and the nonlinear complementarity problem, strict complementarity at a solution of the minimization problem, and boundedness of the level sets of the objective function. We examine these properties for three restricted NCP functions and show that the merit function based on the restricted NCP function proposed in this paper enjoys favorable properties compared with those based on the other restricted NCP functions.