半监督度量学习内蕴最速下降算法的收敛性分析

主要研究对称正定矩阵群上的内蕴最速下降算法的收敛性问题.首先针对一个可转化为对称正定矩阵群上无约束优化问题的半监督度量学习模型,提出对称正定矩阵群上一种自适应变步长的内蕴最速下降算法.然后利用李群上的光滑函数在任意一点处带积分余项的泰勒展开式,证明所提算法在对称正定矩阵群上是线性收敛的.最后通过在分类问题中的数值实验说明算法的有效性.     

Proximal point algorithm with Schur decomposition on the cone of symmetric semidefinite positive matrices

In this work, we propose a proximal algorithm for unconstrained optimization on the cone of symmetric semidefinite positive matrices. It appears to be the first in the proximal class on the set of methods that convert a Symmetric Definite Positive Optimization in Nonlinear Optimization. It replaces the main iteration of the conceptual proximal point algorithm by a sequence of nonlinear programming problems on the cone of diagonal definite positive matrices that has the structure of the positive orthant of the Euclidian vector space. We are motivated by results of the classical proximal algorithm extended to Riemannian manifolds with nonpositive sectional curvature. An important example of such a manifold is the space of symmetric definite positive matrices, where the metrics is given by the Hessian of the standard barrier function −lndet(X). Observing the obvious fact that proximal algorithms do not depend on the geodesics, we apply those ideas to develop a proximal point algorithm for convex functions in this Riemannian metric.     

An efficient algorithm for maximum entropy extension of block-circulant covariance matrices

This paper deals with maximum entropy completion of partially specified block-circulant matrices. Since positive definite symmetric circulants happen to be covariance matrices of stationary periodic processes, in particular of stationary reciprocal processes, this problem has applications in signal processing, in particular to image modeling. In fact it is strictly related to maximum likelihood estimation of bilateral AR-type representations of acausal signals subject to certain conditional independence constraints. The maximum entropy completion problem for block-circulant matrices has recently been solved by the authors, although leaving open the problem of an efficient computation of the solution. In this paper, we provide an efficient algorithm for computing its solution which compares very favorably with existing algorithms designed for positive definite matrix extension problems. The proposed algorithm benefits from the analysis of the relationship between our problem and the band-extension problem for block-Toeplitz matrices also developed in this paper.     

Signal classification with a point process distance on the space of persistence diagrams

In this paper, we consider the problem of signal classification. First, the signal is translated into a persistence diagram through the use of delay-embedding and persistent homology. Endowing the data space of persistence diagrams with a metric from point processes, we show that it admits statistical structure in the form of Fréchet means and variances and a classification scheme is established. In contrast with the Wasserstein distance, this metric accounts for changes in small persistence and changes in cardinality. The classification results using this distance are benchmarked on both synthetic data and real acoustic signals and it is demonstrated that this classifier outperforms current signal classification techniques.     

Some geometric properties of matrix means with respect to different metrics

In this paper we study the monotonicity, in-betweenness and in-sphere properties of matrix means with respect to Bures–Wasserstein, Hellinger and log-determinant metrics. More precisely, we show that the matrix power means (Kubo–Ando and non-Kubo–Ando extensions) satisfy the in-betweenness property in the Hellinger metric. We also show that for two positive definite matrices A and B, the curve of weighted Heron means, the geodesic curve of the arithmetic and the geometric mean lie inside the sphere centered at the geometric mean with the radius equal to half of the log-determinant distance between A and B.

     

Geometric means of structured matrices

The geometric mean of positive definite matrices is usually identified with the Karcher mean, which possesses all properties—generalized from the scalar case—a geometric mean is expected to satisfy. Unfortunately, the Karcher mean is typically not structure preserving, and destroys, e.g., Toeplitz and band structures, which emerge in many applications. For this reason, the Karcher mean is not always recommended for modeling averages of structured matrices. In this article a new definition of a geometric mean for structured matrices is introduced, its properties are outlined, algorithms for its computation, and numerical experiments are provided. In the Toeplitz case an existing mean based on the Kähler metric is analyzed for comparison.     

Structure of Hiai-Petz parametrized geometry for positive definite matrices

Recently Hiai-Petz (2009) [10] discussed a parametrized geometry for positive definite matrices with a pull-back metric for a diffeomorphism to the Euclidean space. Though they also showed that the geodesic is a path of operator means, their interest lies mainly in metrics of the geometry. In this paper, we reconstruct their geometry without metrics and then we show their metric for each unitarily invariant norm defines a Finsler one. Also we discuss another type of geometry in Hiai and Petz (2009) [10] which is a generalization of Corach-Porta-Recht’s one [3].     

Means in metric spaces and the center of mass

In this article k-convex metric spaces are considered where a several variable mapping is provided as a limit point of an iteration scheme based on the midpoint map in the metric space itself. This mapping, considered as a mean of its variables, has some properties which relates it to the center of mass of these variables in the metric space. Sufficient conditions are given here for the two points to be identical, as well as upper bounds on their distances from one another. The asymptotic rate of convergence of the iterative process defining the mean is also determined here. The case of the symmetric space on the convex cone of positive definite matrices related to the geometric mean and the special orthogonal group are also studied here as examples of k-convex metric spaces.     

非负定阵元素的扰动和正定完备化

在保持非负定性不变的前提下,本文对矩阵每一元素容许多大的扰动作了进一步的研究, 将本文的结论和C．R．Johnson提出的部分正定阵的正定完备化进行比较,容易发现对已知的正定矩阵求扰动,本文的结论比用C．R.Johnson的正定完备化计算扰动形式上更简单,同时也给出了不同于C．R．Johnson的部分正定阵的正定完备化表示的另外一个公式,推出了这些正定完备化矩阵应具有的若干性质．     

Monotonic properties of the least squares mean

We settle an open problem of several years standing by showing that the least squares mean for positive definite matrices is monotone for the usual (Loewner) order. Indeed we show this is a special case of its appropriate generalization to partially ordered complete metric spaces of nonpositive curvature. Our techniques extend to establish other basic properties of the least squares mean such as continuity and joint concavity. Moreover, we introduce a weighted least squares mean and derive our results in this more general setting.     

最优传输理论在图像处理中的应用

最优传输问题是寻找概率测度间的最优传输变换的一类特殊的优化问题,近年来在众多领域得到了广泛的关注.针对传统最优传输问题存在的计算量过大、正则性缺失等问题,学者们提出了多种改进的最优传输模型和算法,用于处理实际中的各种问题.简述最优传输问题的基本理论和方法,介绍Wasserstein距离的概念及其衍生出的Wasserstein重心,探讨离散化最优传输模型及其在正则化等方面的改进,讨论求解最优传输问题的算法进展,综述Wasserstein距离在图像处理领域的简单应用,并展望有待进一步研究的工作.     

An Interpolating Distance Between Optimal Transport and Fisher–Rao Metrics

This paper defines a new transport metric over the space of nonnegative measures. This metric interpolates between the quadratic Wasserstein and the Fisher–Rao metrics and generalizes optimal transport to measures with different masses. It is defined as a generalization of the dynamical formulation of optimal transport of Benamou and Brenier, by introducing a source term in the continuity equation. The influence of this source term is measured using the Fisher–Rao metric and is averaged with the transportation term. This gives rise to a convex variational problem defining the new metric. Our first contribution is a proof of the existence of geodesics (i.e., solutions to this variational problem). We then show that (generalized) optimal transport and Hellinger metrics are obtained as limiting cases of our metric. Our last theoretical contribution is a proof that geodesics between mixtures of sufficiently close Dirac measures are made of translating mixtures of Dirac masses. Lastly, we propose a numerical scheme making use of first-order proximal splitting methods and we show an application of this new distance to image interpolation.     

The Wasserstein-Fisher-Rao Metric for Waveform Based Earthquake Location

In this paper, we apply the Wasserstein-Fisher-Rao (WFR) metric from the unbalanced optimal transport theory to the earthquake location problem. Compared with the quadratic Wasserstein ($W_2$) metric from the classical optimal transport theory, the advantage of this method is that it retains the important amplitude information as a new constraint, which avoids the problem of the degeneration of the optimization objective function near the real earthquake hypocenter and origin time. As a result, the deviation of the global minimum of the optimization objective function based on the WFR metric from the true solution can be much smaller than the results based on the $W_2$ metric when there exists strong data noise. Thus, we develop an accurate earthquake location method under strong data noise. Many numerical experiments verify our conclusions.     

Intrinsic Data Depth for Hermitian Positive Definite Matrices

Nondegenerate covariance, correlation, and spectral density matrices are necessarily symmetric or Hermitian and positive definite. This article develops statistical data depths for collections of Hermitian positive definite matrices by exploiting the geometric structure of the space as a Riemannian manifold. The depth functions allow one to naturally characterize most central or outlying matrices, but also provide a practical framework for inference in the context of samples of positive definite matrices. First, the desired properties of an intrinsic data depth function acting on the space of Hermitian positive definite matrices are presented. Second, we propose two pointwise and integrated data depth functions that satisfy each of these requirements and investigate several robustness and efficiency aspects. As an application, we construct depth-based confidence regions for the intrinsic mean of a sample of positive definite matrices, which is applied to the exploratory analysis of a collection of covariance matrices in a multicenter clinical trial. Supplementary materials and an accompanying R-package are available online.     

Discrete Approximation Scheme in Distributionally Robust Optimization

Discrete approximation, which has been the prevailing scheme in stochastic programming in the past decade, has been extended to distributionally robust optimization (DRO) recently. In this paper, we conduct rigorous quantitative stability analysis of discrete approximation schemes for DRO, which measures the approximation error in terms of discretization sample size. For the ambiguity set defined through equality and inequality moment conditions, we quantify the discrepancy between the discretized ambiguity sets and the original set with respect to the Wasserstein metric. To establish the quantitative convergence, we develop a Hoffman error bound theory with Hoffman constant calculation criteria in a infinite dimensional space, which can be regarded as a byproduct of independent interest. For the ambiguity set defined by Wasserstein ball and moment conditions combined with Wasserstein ball, we present similar quantitative stability analysis by taking full advantage of the convex property inherently admitted by Wasserstein metric. Efficient numerical methods for specifically solving discrete approximation DRO problems with thousands of samples are also designed. In particular, we reformulate different types of discrete approximation problems into a class of saddle point problems with completely separable structures. The stochastic primal-dual hybrid gradient (PDHG) algorithm where in each iteration we update a random subset of the sampled variables is then amenable as a solution method for the reformulated saddle point problems. Some preliminary numerical tests are reported.     

Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations

We consider stochastic programs where the distribution of the uncertain parameters is only observable through a finite training dataset. Using the Wasserstein metric, we construct a ball in the space of (multivariate and non-discrete) probability distributions centered at the uniform distribution on the training samples, and we seek decisions that perform best in view of the worst-case distribution within this Wasserstein ball. The state-of-the-art methods for solving the resulting distributionally robust optimization problems rely on global optimization techniques, which quickly become computationally excruciating. In this paper we demonstrate that, under mild assumptions, the distributionally robust optimization problems over Wasserstein balls can in fact be reformulated as finite convex programs—in many interesting cases even as tractable linear programs. Leveraging recent measure concentration results, we also show that their solutions enjoy powerful finite-sample performance guarantees. Our theoretical results are exemplified in mean-risk portfolio optimization as well as uncertainty quantification.

     

Positive definite matrix approximation with condition number constraint

Positive definite matrix approximation with a condition number constraint is an optimization problem to find the nearest positive definite matrix whose condition number is smaller than a given constant. We demonstrate that this problem can be converted to a simpler one when we use a unitary similarity invariant norm as a metric. We can especially convert it to a univariate piecewise convex optimization problem when we use the Ky Fan p-k norm. We also present an analytical solution to the problem whose metric is the spectral norm and the trace norm.     

On the Kantorovich–Rubinstein theorem

The Kantorovich–Rubinstein theorem provides a formula for the Wasserstein metric W₁ on the space of regular probability Borel measures on a compact metric space. Dudley and de Acosta generalized the theorem to measures on separable metric spaces. Kellerer, using his own work on Monge–Kantorovich duality, obtained a rapid proof for Radon measures on an arbitrary metric space. The object of the present expository article is to give an account of Kellerer’s generalization of the Kantorovich–Rubinstein theorem, together with related matters. It transpires that a more elementary version of Monge–Kantorovich duality than that used by Kellerer suffices for present purposes. The fundamental relations that provide two characterizations of the Wasserstein metric are obtained directly, without the need for prior demonstration of density or duality theorems. The latter are proved, however, and used in the characterization of optimal measures and functions for the Kantorovich–Rubinstein linear programme. A formula of Dobrushin is proved.     

Stopping criteria for the Ando-Li-Mathias and Bini-Meini-Poloni geometric means

We provide an upper bound for the number of iterations necessary to achieve a desired level of accuracy for the Ando-Li-Mathias [Linear Algebra Appl. 385 (2004) 305-334] and Bini-Meini-Poloni [Math. Comput. 79 (2010) 437-452] symmetrization procedures for computing the geometric mean of n positive definite matrices, where accuracy is measured by the spectral norm and the Thompson metric on the convex cone of positive definite matrices. It is shown that the upper bound for the number of iterations depends only on the diameter of the set of n matrices and the desired convergence tolerance. A striking result is that the upper bound decreases as n increases on any bounded region of positive definite matrices.     

Direct and Inverse Eigenvalue Problems for Diagonal-Plus-Semiseparable Matrices

In this paper we study both direct and inverse eigenvalue problems for diagonal-plus-semiseparable (dpss) matrices. In particular, we show that the computation of the eigenvalues of a symmetric dpss matrix can be reduced by a congruence transformation to solving a generalized symmetric definite tridiagonal eigenproblem. Using this reduction, we devise a set of recurrence relations for evaluating the characteristic polynomial of a dpss matrix in a stable way at a linear time. This in turn allows us to apply divide-and-conquer eigenvalue solvers based on functional iterations directly to dpss matrices without performing any preliminary reduction into a tridiagonal form. In the second part of the paper, we exploit the structural properties of dpss matrices to solve the inverse eigenvalue problem of reconstructing a symmetric dpss matrix from its spectrum and some other informations. Finally, applications of our results to the computation of a QR factorization of a Cauchy matrix with real nodes are provided.