Wasserstein geometry of porous medium equation

We study the porous medium equation with emphasis on q-Gaussian measures, which are generalizations of Gaussian measures by using power-law distribution. On the space of q-Gaussian measures, the porous medium equation is reduced to an ordinary differential equation for covariance matrix. We introduce a set of inequalities among functionals which gauge the difference between pairs of probability measures and are useful in the analysis of the porous medium equation. We show that any q-Gaussian measure provides a nontrivial pair attaining equality in these inequalities.     

Penalized Covariance Matrix Estimation Using a Matrix-Logarithm Transformation

For statistical inferences that involve covariance matrices, it is desirable to obtain an accurate covariance matrix estimate with a well-structured eigen-system. We propose to estimate the covariance matrix through its matrix logarithm based on an approximate log-likelihood function. We develop a generalization of the Leonard and Hsu log-likelihood approximation that no longer requires a nonsingular sample covariance matrix. The matrix log-transformation provides the ability to impose a convex penalty on the transformed likelihood such that the largest and smallest eigenvalues of the covariance matrix estimate can be regularized simultaneously. The proposed method transforms the problem of estimating the covariance matrix into the problem of estimating a symmetric matrix, which can be solved efficiently by an iterative quadratic programming algorithm. The merits of the proposed method are illustrated by a simulation study and two real applications in classification and portfolio optimization. Supplementary materials for this article are available online.     

A conservative bound on the estimation error covariance matrix in the presence of correlated driving noise and correlated discrete measurement noise

In an earlier paper, the conservative and minimal bound to the crosscorrelation terms between estimation error and a random forcing function was presented. That bound was found to be a particular linear combination of the estimation error covariance and the forcing function covariance involving a free scalar parameter. The bound was then substituted for the cross-correlation terms in the differential equation for the estimation error covariance matrix in order to approximate its behavior between discrete measurement times. The time history of the free parameter which minimized a linear combination of the elements of the estimated covariance matrix at the next measurement time was found as the noniterative solution to an optimal control problem with a matrix state.In this paper, necessary and sufficient conditions are presented for the problem of minimizing a linear combination of the elements of the approximated estimation error covariance at the end of an interval in which are linearly incorporated a finite number of discrete vector measurements corrupted by white and/or correlated measurement noise. Although the determination of the optimal trajectory in general requires iteration, a particularly simple algorithm is presented. Numerical results are presented for the case of a satellite in a highly elliptic orbit about a model Earth.     

Sparse Steinian Covariance Estimation

We consider a new method for sparse covariance matrix estimation which is motivated by previous results for the so-called Stein-type estimators. Stein proposed a method for regularizing the sample covariance matrix by shrinking together the eigenvalues; the amount of shrinkage is chosen to minimize an unbiased estimate of the risk (UBEOR) under the entropy loss function. The resulting estimator has been shown in simulations to yield significant risk reductions over the maximum likelihood estimator. Our method extends the UBEOR minimization problem by adding an ?₁ penalty on the entries of the estimated covariance matrix, which encourages a sparse estimate. For a multivariate Gaussian distribution, zeros in the covariance matrix correspond to marginal independences between variables. Unlike the ?₁-penalized Gaussian likelihood function, our penalized UBEOR objective is convex and can be minimized via a simple block coordinate descent procedure. We demonstrate via numerical simulations and an analysis of microarray data from breast cancer patients that our proposed method generally outperforms other methods for sparse covariance matrix estimation and can be computed efficiently even in high dimensions.     

The affine equivariant sign covariance matrix: asymptotic behavior and efficiencies

We consider the affine equivariant sign covariance matrix (SCM) introduced by Visuri et al. (J. Statist. Plann. Inference 91 (2000) 557). The population SCM is shown to be proportional to the inverse of the regular covariance matrix. The eigenvectors and standardized eigenvalues of the covariance matrix can thus be derived from the SCM. We also construct an estimate of the covariance and correlation matrix based on the SCM. The influence functions and limiting distributions of the SCM and its eigenvectors and eigenvalues are found. Limiting efficiencies are given in multivariate normal and t-distribution cases. The estimates are highly efficient in the multivariate normal case and perform better than estimates based on the sample covariance matrix for heavy-tailed distributions. Simulations confirmed these findings for finite-sample efficiencies.     

Bayesian estimation of a covariance matrix with flexible prior specification

Bayesian analysis for a covariance structure has been in use for decades. The commonly adopted Bayesian setup involves the conjugate inverse Wishart prior specification for the covariance matrix. Here we depart from this approach and adopt a novel prior specification by considering a multivariate normal prior for the elements of the matrix logarithm of the covariance structure. This specification allows for a richer class of prior distributions for the covariance structure with respect to strength of beliefs in prior location hyperparameters and the added ability to model potential correlation amongst the covariance structure. We provide three computational methods for calculating the posterior moment of the covariance matrix. The moments of interest are calculated based upon computational results via Importance sampling, Laplacian approximation and Markov Chain Monte Carlo/Metropolis–Hastings techniques. As a particular application of the proposed technique we investigate educational test score data from the project talent data set.     

Statistically and Computationally Efficient Estimating Equations for Large Spatial Datasets

For Gaussian process models, likelihood-based methods are often difficult to use with large irregularly spaced spatial datasets, because exact calculations of the likelihood for n observations require O(n³) operations and O(n²) memory. Various approximation methods have been developed to address the computational difficulties. In this article, we propose new, unbiased estimating equations (EE) based on score equation approximations that are both computationally and statistically efficient. We replace the inverse covariance matrix that appears in the score equations by a sparse matrix to approximate the quadratic forms, then set the resulting quadratic forms equal to their expected values to obtain unbiased EE. The sparse matrix is constructed by a sparse inverse Cholesky approach to approximate the inverse covariance matrix. The statistical efficiency of the resulting unbiased EE is evaluated both in theory and by numerical studies. Our methods are applied to nearly 90,000 satellite-based measurements of water vapor levels over a region in the Southeast Pacific Ocean.     

High Breakdown Estimation for Multiple Populations with Applications to Discriminant Analysis

We consider S-estimators of multivariate location and common dispersion matrix in multiple populations. Instead of averaging the robust estimates of the individual covariance matrices, as used by Todorov, Neykov and Neytchev (1990), the observations are pooled for estimating the common covariance more efficiently. Two such proposals are evaluated by a breakdown point analysis and Monte Carlo simulations. Their applications to the discriminant analysis are also considered.     

Estimation of a Multivariate Normal Covariance Matrix with Staircase Pattern Data

In this paper, we study the problem of estimating a multivariate normal covariance matrix with staircase pattern data. Two kinds of parameterizations in terms of the covariance matrix are used. One is Cholesky decomposition and another is Bartlett decomposition. Based on Cholesky decomposition of the covariance matrix, the closed form of the maximum likelihood estimator (MLE) of the covariance matrix is given. Using Bayesian method, we prove that the best equivariant estimator of the covariance matrix with respect to the special group related to Cholesky decomposition uniquely exists under the Stein loss. Consequently, the MLE of the covariance matrix is inadmissible under the Stein loss. Our method can also be applied to other invariant loss functions like the entropy loss and the symmetric loss. In addition, based on Bartlett decomposition of the covariance matrix, the Jeffreys prior and the reference prior of the covariance matrix with staircase pattern data are also obtained. Our reference prior is different from Berger and Yang’s reference prior. Interestingly, the Jeffreys prior with staircase pattern data is the same as that with complete data. The posterior properties are also investigated. Some simulation results are given for illustration.     

Spatial Localization for Nonlinear Dynamical Stochastic Models for Excitable Media

Nonlinear dynamical stochastic models are ubiquitous in different areas. Their statistical properties are often of great interest, but are also very challenging to compute. Many excitable media models belong to such types of complex systems with large state dimensions and the associated covariance matrices have localized structures. In this article, a mathematical framework to understand the spatial localization for a large class of stochastically coupled nonlinear systems in high dimensions is developed. Rigorous \linebreak mathematical analysis shows that the local effect from the diffusion results in an exponential decay of the components in the covariance matrix as a function of the distance while the global effect due to the mean field interaction synchronizes different components and contributes to a global covariance. The analysis is based on a comparison with an appropriate linear surrogate model, of which the covariance propagation can be computed explicitly. Two important applications of these theoretical results are discussed. They are the spatial averaging strategy for efficiently sampling the covariance matrix and the localization technique in data assimilation. Test examples of a linear model and a stochastically coupled FitzHugh-Nagumo model for excitable media are adopted to validate the theoretical results. The latter is also used for a systematical study of the spatial averaging strategy in efficiently sampling the covariance matrix in different dynamical regimes.     

A moving average Cholesky factor model in joint mean-covariance modeling for longitudinal data

Modeling the mean and covariance simultaneously is a common strategy to efciently estimate the mean parameters when applying generalized estimating equation techniques to longitudinal data.In this article,using generalized estimation equation techniques,we propose a new kind of regression models for parameterizing covariance structures.Using a novel Cholesky factor,the entries in this decomposition have moving average and log innovation interpretation and are modeled as linear functions of covariates.The resulting estimators for the regression coefcients in both the mean and the covariance are shown to be consistent and asymptotically normally distributed.Simulation studies and a real data analysis show that the proposed approach yields highly efcient estimators for the parameters in the mean,and provides parsimonious estimation for the covariance structure.     

Improved entropy decay estimates for the heat equation

Improved entropy decay estimates for the heat equation are obtained by selecting well-parametrized Gaussians. Either by mass centering or by fixing the second moments or the covariance matrix of the solution, relative entropy toward these Gaussians is shown to decay with better constants than classical estimates.     

Diffusion approximation of the two-type Galton-Watson process with mean matrix close to the identity

In this paper a diffusion approximation to the two-type Galton-Watson branching processes with mean matrix close to the identity is given in the form of Berstein stochastic differentials. An associated diffusion equation is found using an extension of the one-dimensional Bernstein technique. Expressions for the mean vector and covariance matrix of the diffusion approximation are derived.     

Covariance estimation under spatial dependence

Correlated multivariate processes have a dependence structure which must be taken into account when estimating the covariance matrix. The natural estimator of the covariance matrix is introduced and is shown that to be biased under the dependence structure. This bias is studied under two different asymptotic models, namely increasing the domain by increasing the number of observations, and increasing the number of observations in the fixed domain. Using the first asymptotic model, we quantify the convergence rate of the bias and of the covariance between the components of the estimated covariance matrix. The second asymptotic model serves to derive a fast and accurate bias correction. As shown, under mild hypotheses, the asymptotic normality of the estimated covariance matrix holds and can be used to test whether the bias is significant, for example, in the sense that the eigenvectors of the estimated and true covariance matrices are significantly different.     

Weak approximations. A Malliavin calculus approach

We introduce a variation of the proof for weak approximations that is suitable for studying the densities of stochastic processes which are evaluations of the flow generated by a stochastic differential equation on a random variable that may be anticipating. Our main assumption is that the process and the initial random variable have to be smooth in the Malliavin sense. Furthermore, if the inverse of the Malliavin covariance matrix associated with the process under consideration is sufficiently integrable, then approximations for densities and distributions can also be achieved. We apply these ideas to the case of stochastic differential equations with boundary conditions and the composition of two diffusions.

     

Asymptotically Normal Explicit Estimation of Parameters in the Michaelis–Menten Equation

Under consideration is the problem of estimating unknown parameters in the Michaelis–Menten equation which is frequent in natural sciences. The authors suggest and study asymptotically normal explicit estimates of unknown parameters which often have a minimal covariance matrix.     

The spectral decomposition of a covariance matrix for the balanced mixed analysis of variance model

We derive the spectral decomposition of a covariance matrix for the balanced mixed analysis of variance model. The derivation is based on determining the distinct eigenvalues of a covariance matrix and then obtaining a principal idempotent matrix for each distinct eigenvalue. Examples are given to illustrate the results.     

Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses

The normal distribution based likelihood ratio (LR) statistic is widely used in structural equation modeling. Under a sequence of local alternative hypotheses, this statistic has been shown to asymptotically follow a noncentral chi-square distribution. In practice, the population mean vector and covariance matrix as well as the model and sample size are always fixed. It is hard to justify the validity of the noncentral chi-square distribution for the resulting LR statistic even when data are normally distributed and sample size is large. By extending results in the literature, this paper develops normal distributions to describe the behavior of the LR statistic for mean and covariance structure analysis. A sequence of local alternative hypotheses is not necessary for the proposed distributions to be asymptotically valid. When the effect size is medium and above or when the model is not trivially misspecified, empirical results indicate that a refined normal distribution describes the behavior of the LR statistic better than the commonly used noncentral chi-square distribution, as measured by the Kolmogorov-Smirnov distance. Quantile-quantile plots are also provided to better understand the different distributions.     

A data driven equivariant approach to constrained Gaussian mixture modeling

Maximum likelihood estimation of Gaussian mixture models with different class-specific covariance matrices is known to be problematic. This is due to the unboundedness of the likelihood, together with the presence of spurious maximizers. Existing methods to bypass this obstacle are based on the fact that unboundedness is avoided if the eigenvalues of the covariance matrices are bounded away from zero. This can be done imposing some constraints on the covariance matrices, i.e. by incorporating a priori information on the covariance structure of the mixture components. The present work introduces a constrained approach, where the class conditional covariance matrices are shrunk towards a pre-specified target matrix \(\varvec{\varPsi }.\) Data-driven choices of the matrix \(\varvec{\varPsi },\) when a priori information is not available, and the optimal amount of shrinkage are investigated. Then, constraints based on a data-driven \(\varvec{\varPsi }\) are shown to be equivariant with respect to linear affine transformations, provided that the method used to select the target matrix be also equivariant. The effectiveness of the proposal is evaluated on the basis of a simulation study and an empirical example.     

Finiteness of small factor analysis models

We consider factor analysis models with one or two factors. Fixing the number of factors, we prove a finiteness result about the covariance matrix parameter space when the size of the covariance matrix increases. According to this result, there exists a distinguished matrix size starting at which one can determine whether a given covariance matrix belongs to the parameter space by determining whether all principal submatrices of the distinguished size belong to the corresponding parameter space. We show that the distinguished matrix size is four in the model with one factor and six with two factors.