Robust reduced rank regression in a distributed setting

This paper studies the reduced rank regression problem, which assumes a low-rank structure of the coefficient matrix, together with heavy-tailed noises. To address the heavy-tailed noise, we adopt the quantile loss function instead of commonly used squared loss. However, the non-smooth quantile loss brings new challenges to both the computation and development of statistical properties, especially when the data are large in size and distributed across different machines. To this end, we first transform the response variable and reformulate the problem into a trace-norm regularized least-square problem, which greatly facilitates the computation. Based on this formulation, we further develop a distributed algorithm. Theoretically, we establish the convergence rate of the obtained estimator and the theoretical guarantee for rank recovery. The simulation analysis is provided to demonstrate the effectiveness of our method.

     

Proximal alternating linearized minimization for nonconvex and nonsmooth problems

We introduce a proximal alternating linearized minimization (PALM) algorithm for solving a broad class of nonconvex and nonsmooth minimization problems. Building on the powerful Kurdyka–?ojasiewicz property, we derive a self-contained convergence analysis framework and establish that each bounded sequence generated by PALM globally converges to a critical point. Our approach allows to analyze various classes of nonconvex-nonsmooth problems and related nonconvex proximal forward–backward algorithms with semi-algebraic problem’s data, the later property being shared by many functions arising in a wide variety of fundamental applications. A by-product of our framework also shows that our results are new even in the convex setting. As an illustration of the results, we derive a new and simple globally convergent algorithm for solving the sparse nonnegative matrix factorization problem.     

Preconditioner based on the Sherman-Morrison formula for regularized least squares problems

We propose a sparse approximate inverse preconditioner based on the Sherman-Morrison formula for Tikhonov regularized least square problems. Theoretical analysis shows that, the factorization method can take the advantage of the symmetric property of the coefficient matrix and be implemented cheaply. Combined with dropping rules, the incomplete factorization leads to a preconditioner for Krylov iterative methods to solve regularized least squares problems. Numerical experiments show that our preconditioner is competitive compared to existing methods, especially for ill-conditioned and rank deficient least squares problems.     

Range inclusion of operators on non-archimedean Banach space

In this paper, we establish some range inclusion theorems for non-archimedean Banach spaces over general valued fields. These theorems provide close relationship among range inclusion, majorization and factorization for bounded linear operators. It is found that these results depend strongly on a continuous extension property, which is always true in the classical archimedean case, but may fail to hold for the nonarchimedean setting. Several counterexamples are given to show that our results are sharp in some sense.     

Numerical differentiation for the second order derivatives of functions of two variables

A regularized optimization problem for computing numerical differentiation for the second order derivatives of functions with two variables from noisy values at scattered points is discussed in this article. We prove the existence and uniqueness of the solution to this problem, provide a constructive scheme for the solution which is based on bi-harmonic Green's function and give a convergence estimate of the regularized solution to the exact solution for the problem under a simple choice of regularization parameter. The efficiency of the constructive scheme is shown by some numerical examples.     

On a generalized proximal point method for solving equilibrium problems in Banach spaces

We introduce a regularized equilibrium problem in Banach spaces, involving generalized Bregman functions. For this regularized problem, we establish the existence and uniqueness of solutions. These regularizations yield a proximal-like method for solving equilibrium problems in Banach spaces. We prove that the proximal sequence is an asymptotically solving sequence when the dual space is uniformly convex. Moreover, we prove that all weak accumulation points are solutions if the equilibrium function is lower semicontinuous in its first variable. We prove, under additional assumptions, that the proximal sequence converges weakly to a solution.     

Multidimensional butterfly factorization

This paper introduces the multidimensional butterfly factorization as a data-sparse representation of multidimensional kernel matrices that satisfy the complementary low-rank property. This factorization approximates such a kernel matrix of size $N\times N$ with a product of $O(\log ?N)$ sparse matrices, each of which contains $O(N)$ nonzero entries. We also propose efficient algorithms for constructing this factorization when either (i) a fast algorithm for applying the kernel matrix and its adjoint is available or (ii) every entry of the kernel matrix can be evaluated in $O(1)$ operations. For the kernel matrices of multidimensional Fourier integral operators, for which the complementary low-rank property is not satisfied due to a singularity at the origin, we extend this factorization by combining it with either a polar coordinate transformation or a multiscale decomposition of the integration domain to overcome the singularity. Numerical results are provided to demonstrate the efficiency of the proposed algorithms.     

基于混合l_2/l_1范数极小化方法的块稀疏信号重构条件

研究压缩感知中的块稀疏信号重构问题,主要对混合l_2/l_1极小化方法建立了一类改进的可重构条件.具体地说,本文证明若测量矩阵满足条件δ_k+θ_(k,k)1,则混合l_2/l_1极小化方法可精确重构(无噪声情形)或鲁棒重构(有噪声情形)原始块k-稀疏信号.进而表明本文给出的新条件弱于现有文献所给出的条件.     

Building a completely positive factorization

A symmetric matrix of order n is called completely positive if it has a symmetric factorization by means of a rectangular matrix with n columns and no negative entries (a so-called cp factorization), i.e., if it can be interpreted as a Gram matrix of n directions in the positive orthant of another Euclidean space of possibly different dimension. Finding this factor therefore amounts to angle packing and finding an appropriate embedding dimension. Neither the embedding dimension nor the directions may be unique, and so many cp factorizations of the same given matrix may coexist. Using a bordering approach, and building upon an already known cp factorization of a principal block, we establish sufficient conditions under which we can extend this cp factorization to the full matrix. Simulations show that the approach is promising also in higher dimensions.

     

Double k-Class Estimators in Regression Models with Non-spherical Disturbances

In this paper, we consider a family of feasible generalised double k-class estimators in a linear regression model with non-spherical disturbances. We derive the large sample asymptotic distribution of the proposed family of estimators and compare its performance with the feasible generalized least squares and Stein-rule estimators using the mean squared error matrix and risk under quadratic loss criteria. A Monte-Carlo experiment investigates the finite sample behaviour of the proposed family of estimators.     

Convergence of projected Landweber iteration for matrix rank minimization

In this paper, we study the performance of the projected Landweber iteration (PLW) for the general low rank matrix recovery. The PLW was first proposed by Zhang and Chen (2010) [43] based on the sparse recovery algorithm APG (Daubechies et al., 2008) [14] in the matrix completion setting, and numerical experiments have been given to show its efficiency (Zhang and Chen, 2010) [43]. In this paper, we focus on providing a convergence rate analysis of the PLW in the general setting of low rank matrix recovery with the affine transform having the matrix restricted isometry property. We show robustness of the algorithm to noise with a strong geometric convergence rate even for noisy measurements provided that the affine transform satisfies a matrix restricted isometry property condition.     

Strong local nondeterminism and exact modulus of continuity for spherical Gaussian fields

In this paper, we are concerned with sample path properties of isotropic spherical Gaussian fields on ${\mathbb{S}}^2$. In particular, we establish the property of strong local nondeterminism of an isotropic spherical Gaussian field based on the high-frequency behaviour of its angular power spectrum; we then exploit this result to establish an exact uniform modulus of continuity for its sample paths. We also discuss the range of values of the spectral index for which the sample functions exhibit fractal or smooth behaviour.     

Sparse trace norm regularization

We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the \(\ell _1\) -norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method; we also develop efficient algorithms to solve the key components in AG. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.     

������̬��٤���������̬�ֲ��еķ��������Stein��ʧ�µı�Ҷ˹���������

or the variance parameter of the normal distribution with a normal-inverse-gamma prior, we analytically calculate the Bayes posterior estimator with respect to a conjugate normal-inverse-gamma prior distribution under Stein's loss function. This estimator minimizes the Posterior Expected Stein's Loss (PESL). We also analytically calculate the Bayes posterior estimator and the PESL under the squared error loss function. The numerical simulations exemplify our theoretical studies that the PESLs do not depend on the sample, and that the Bayes posterior estimator and the PESL under the squared error loss function are unanimously larger than those under Stein's loss function. Finally, we calculate the Bayes posterior estimators and the PESLs of the monthly simple returns of the SSE Composite Index.     

Truncated regularized Newton method for convex minimizations

Recently, Li et al. (Comput. Optim. Appl. 26:131–147, 2004) proposed a regularized Newton method for convex minimization problems. The method retains local quadratic convergence property without requirement of the singularity of the Hessian. In this paper, we develop a truncated regularized Newton method and show its global convergence. We also establish a local quadratic convergence theorem for the truncated method under the same conditions as those in Li et al. (Comput. Optim. Appl. 26:131–147, 2004). At last, we test the proposed method through numerical experiments and compare its performance with the regularized Newton method. The results show that the truncated method outperforms the regularized Newton method. The work was supported by the 973 project granted 2004CB719402 and the NSF project of China granted 10471036.     

Asymptotic theory of the adaptive Sparse Group Lasso

We study the asymptotic properties of a new version of the Sparse Group Lasso estimator (SGL), called adaptive SGL. This new version includes two distinct regularization parameters, one for the Lasso penalty and one for the Group Lasso penalty, and we consider the adaptive version of this regularization, where both penalties are weighted by preliminary random coefficients. The asymptotic properties are established in a general framework, where the data are dependent and the loss function is convex. We prove that this estimator satisfies the oracle property: the sparsity-based estimator recovers the true underlying sparse model and is asymptotically normally distributed. We also study its asymptotic properties in a double-asymptotic framework, where the number of parameters diverges with the sample size. We show by simulations and on real data that the adaptive SGL outperforms other oracle-like methods in terms of estimation precision and variable selection.

     

Subdifferential of the supremum function: moving back and forth between continuous and non-continuous settings

In this paper we establish general formulas for the subdifferential of the pointwise supremum of convex functions, which cover and unify both the compact continuous and the non-compact non-continuous settings. From the non-continuous to the continuous setting, we proceed by a compactification-based approach which leads us to problems having compact index sets and upper semi-continuously indexed mappings, giving rise to new characterizations of the subdifferential of the supremum by means of upper semicontinuous regularized functions and an enlarged compact index set. In the opposite sense, we rewrite the subdifferential of these new regularized functions by using the original data, also leading us to new results on the subdifferential of the supremum. We give two applications in the last section, the first one concerning the nonconvex Fenchel duality, and the second one establishing Fritz-John and KKT conditions in convex semi-infinite programming.

     

Convex optimization learning of faithful Euclidean distance representations in nonlinear dimensionality reduction

Classical multidimensional scaling only works well when the noisy distances observed in a high dimensional space can be faithfully represented by Euclidean distances in a low dimensional space. Advanced models such as Maximum Variance Unfolding (MVU) and Minimum Volume Embedding (MVE) use Semi-Definite Programming (SDP) to reconstruct such faithful representations. While those SDP models are capable of producing high quality configuration numerically, they suffer two major drawbacks. One is that there exist no theoretically guaranteed bounds on the quality of the configuration. The other is that they are slow in computation when the data points are beyond moderate size. In this paper, we propose a convex optimization model of Euclidean distance matrices. We establish a non-asymptotic error bound for the random graph model with sub-Gaussian noise, and prove that our model produces a matrix estimator of high accuracy when the order of the uniform sample size is roughly the degree of freedom of a low-rank matrix up to a logarithmic factor. Our results partially explain why MVU and MVE often work well. Moreover, the convex optimization model can be efficiently solved by a recently proposed 3-block alternating direction method of multipliers. Numerical experiments show that the model can produce configurations of high quality on large data points that the SDP approach would struggle to cope with.

     

On the variational description of the trajectories of averaging quantum dynamical maps

We study the dynamics of quantum system with degenerated Hamiltonian. To this end we consider the approximating sequence of regularized Hamiltonians and corresponding sequence of dynamical semigroups acting in the space of quantum states. The limit points set of the sequence of regularized semigroups is obtained as the result of averaging by finitely additive measure on the set of regularizing parameters. We establish that the family of averaging dynamical maps does not possess the semigroup property and the injectivity property. We define the functionals on the space of maps of the time interval into the quantum states space such that the maximum points of this functionals coincide with the trajectories of the family of averaging dynamical maps.