Approximating posterior probabilities in a linear model with possibly noninvertible moving average errors

The method of Laplace is used to approximate posterior probabilities for a collection of polynomial regression models when the errors follow a process with a noninvertible moving average component. These results are useful in the problem of period-change analysis of variable stars and in assessing the posterior probability that a time series with trend has been overdifferenced. The nonstandard covariance structure induced by a noninvertible moving average process can invalidate the standard Laplace method. A number of analytical tools is used to produce corrected Laplace approximations. These tools include viewing the covariance matrix of the observations as tending to a differential operator. The use of such an operator and its Green's function provides a convenient and systematic method of asymptotically inverting the covariance matrix.In certain cases there are two different Laplace approximations, and the appropriate one to use depends upon unknown parameters. This problem is dealt with by using a weighted geometric mean of the candidate approximations, where the weights are completely data-based and such that, asymptotically, the correct approximation is used. The new methodology is applied to an analysis of the prototypical long-period variable star known as Mira.     

Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs

We study a Bayesian approach to nonparametric estimation of the periodic drift function of a one-dimensional diffusion from continuous-time data. Rewriting the likelihood in terms of local time of the process, and specifying a Gaussian prior with precision operator of differential form, we show that the posterior is also Gaussian with the precision operator also of differential form. The resulting expressions are explicit and lead to algorithms which are readily implementable. Using new functional limit theorems for the local time of diffusions on the circle, we bound the rate at which the posterior contracts around the true drift function.     

On improved estimation of normal precision matrix and discriminant coefficients

The problem of estimating the precision matrix of a multivariate normal distribution model is considered with respect to a quadratic loss function. A number of covariance estimators originally intended for a variety of loss functions are adapted so as to obtain alternative estimators of the precision matrix. It is shown that the alternative estimators have analytically smaller risks than the unbiased estimator of the precision matrix. Through numerical studies of risk values, it is shown that the new estimators have substantial reduction in risk. In addition, we consider the problem of the estimation of discriminant coefficients, which arises in linear discriminant analysis when Fisher's linear discriminant function is viewed as the posterior log-odds under the assumption that two classes differ in mean but have a common covariance matrix. The above method is also adapted for this problem in order to obtain improved estimators of the discriminant coefficients under the quadratic loss function. Furthermore, a numerical study is undertaken to compare the properties of a collection of alternatives to the “unbiased” estimator of the discriminant coefficients.     

Unitarity of Generalized Fourier–Gauss Transforms

Abstract

A generalized Fourier–Gauss transform is an operator acting in a Boson Fock space and is formulated as a continuous linear operator acting on the space of test white noise functions. It does not admit, in general, a unitary extension with respect to the norm of the Boson Fock space induced from the Gaussian measure with variance 1 but is extended to a unitary isomorphism if the Gaussian measure is replaced with the ones with different covariance operators. As an application, unitarity of a generalized dilation is discussed.     

SPDE with generalized drift and fractional-type noise

We consider a stochastic partial differential equation involving a second order differential operator whose drift is discontinuous. The equation is driven by a Gaussian noise which behaves as a Wiener process in space and the time covariance generates a signed measure. This class includes the Brownian motion, fractional Brownian motion and other related processes. We give a necessary and sufficient condition for the existence of the solution and we study the path regularity of this solution.     

Estimates of MM type for the multivariate linear model

We propose a class of robust estimates for multivariate linear models. Based on the approach of MM-estimation (Yohai 1987, [24]), we estimate the regression coefficients and the covariance matrix of the errors simultaneously. These estimates have both a high breakdown point and high asymptotic efficiency under Gaussian errors. We prove consistency and asymptotic normality assuming errors with an elliptical distribution. We describe an iterative algorithm for the numerical calculation of these estimates. The advantages of the proposed estimates over their competitors are demonstrated through both simulated and real data.     

Well-Posedness and Finite Element Approximations for Elliptic SPDEs with Gaussian Noises

The paper studies the well-posedness and optimal error estimates of spectral finite element approximations for the boundary value problems of semi-linear elliptic SPDEs driven by white or colored Gaussian noises. The noise term is approximated through the spectral projection of the covariance operator, which is not required to be commutative with the Laplacian operator.Through the convergence analysis of SPDEs with the noise terms replaced by the projected noises, the well-posedness of the SPDE is established under certain covariance operator-dependent conditions. These SPDEs with projected noises are then numerically approximated with the finite element method. A general error estimate framework is established for the finite element approximations. Based on this framework, optimal error estimates of finite element approximations for elliptic SPDEs driven by power-law noises are obtained. It is shown that with the proposed approach, convergence order of white noise driven SPDEs is improved by half for one-dimensional problems, and by an infinitesimal factor for higher-dimensional problems.     

Estimation of multivariate normal covariance and precision matrices in a star-shape model with missing data

In this paper, we study the problem of estimating the covariance matrix Σ and the precision matrix Ω (the inverse of the covariance matrix) in a star-shape model with missing data. By considering a type of Cholesky decomposition of the precision matrix Ω=Ψ^′Ψ, where Ψ is a lower triangular matrix with positive diagonal elements, we get the MLEs of the covariance matrix and precision matrix and prove that both of them are biased. Based on the MLEs, unbiased estimators of the covariance matrix and precision matrix are obtained. A special group G, which is a subgroup of the group consisting all lower triangular matrices, is introduced. By choosing the left invariant Haar measure on G as a prior, we obtain the closed forms of the best equivariant estimates of Ω under any of the Stein loss, the entropy loss, and the symmetric loss. Consequently, the MLE of the precision matrix (covariance matrix) is inadmissible under any of the above three loss functions. Some simulation results are given for illustration.     

Inference under functional proportional and common principal component models

In many situations, when dealing with several populations with different covariance operators, equality of the operators is assumed. Usually, if this assumption does not hold, one estimates the covariance operator of each group separately, which leads to a large number of parameters. As in the multivariate setting, this is not satisfactory since the covariance operators may exhibit some common structure. In this paper, we discuss the extension to the functional setting of the common principal component model that has been widely studied when dealing with multivariate observations. Moreover, we also consider a proportional model in which the covariance operators are assumed to be equal up to a multiplicative constant. For both models, we present estimators of the unknown parameters and we obtain their asymptotic distribution. A test for equality against proportionality is also considered.     

The contiguity of probability measures and asymptotic inference in continuous time stationary diffusions and Gaussian processes with known covariance

We establish contiguity of families of probability measures indexed by T, as T → ∞, for classes of continuous time stochastic processes which are either stationary diffusions or Gaussian processes with known covariance. In most cases, and in all the examples we consider in Section 4, the covariance is completely determined by observing the process continuously over any finite interval of time. Many important consequences pertaining to properties of tests and estimators, outlined in Section 5, will then apply.     

Asymptotic nonnull distributions for tests for reality of a covariance matrix

In this paper asymptotic nonnull distributions are derived for two statistics used in testing for the reality of the covariance matrix in a complex Gaussian distribution.     

The efficiency of logistic regression compared to normal discriminant analysis under class-conditional classification noise

In many real world classification problems, class-conditional classification noise (CCC-Noise) frequently deteriorates the performance of a classifier that is naively built by ignoring it. In this paper, we investigate the impact of CCC-Noise on the quality of a popular generative classifier, normal discriminant analysis (NDA), and its corresponding discriminative classifier, logistic regression (LR). We consider the problem of two multivariate normal populations having a common covariance matrix. We compare the asymptotic distribution of the misclassification error rate of these two classifiers under CCC-Noise. We show that when the noise level is low, the asymptotic error rates of both procedures are only slightly affected. We also show that LR is less deteriorated by CCC-Noise compared to NDA. Under CCC-Noise contexts, the Mahalanobis distance between the populations plays a vital role in determining the relative performance of these two procedures. In particular, when this distance is small, LR tends to be more tolerable to CCC-Noise compared to NDA.     

On the limiting spectral distribution of the covariance matrices of time-lagged processes

We consider two continuous-time Gaussian processes, one being partially correlated to a time-lagged version of the other. We first give the limiting spectral distribution for the covariance matrices of the increments of the processes when the span between two observations tends to zero. Then, we derive the limiting distribution of the eigenvalues of the sample covariance matrices. This result is obtained when the number of paths of the processes is asymptotically proportional to the number of observations for each single path. As an application, we use the second moment of this distribution together with auxiliary volatility and correlation estimates to construct an adaptive estimator of the time lag between the two processes. Finally, we provide an asymptotic theory for our estimation procedure.     

Estimation error for blind Gaussian time series prediction

We tackle the issue of the blind prediction of a Gaussian time series. For this, we construct a projection operator built by plugging an empirical covariance estimator into a Schur complement decomposition of the projector. This operator is then used to compute the predictor. Rates of convergence of the estimates are given.     

State space models on special manifolds

This paper concerns modeling time series observations in state space forms considered on the Stiefel and Grassmann manifolds. We develop a state space model relating the time series observations to a sequence of unobserved state or parameter matrices assuming the matrix Langevin noise processes on the Stiefel manifolds. We show a Bayes method for estimating the state matrices by the posterior modes. We consider a further extended state space model where two sequences of unobserved state matrices are involved. A simple state space model on the Grassmann manifolds with matrix Langevin noise processes is also investigated.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

Positivity for Gaussian graphical models

Gaussian graphical models are parametric statistical models for jointly normal random variables whose dependence structure is determined by a graph. In previous work, we introduced trek separation, which gives a necessary and sufficient condition in terms of the graph for when a subdeterminant is zero for all covariance matrices that belong to the Gaussian graphical model. Here we extend this result to give explicit cancellation-free formulas for the expansions of non-zero subdeterminants.     

Statistical approach to some ill-posed problems for linear partial differential equations

For linear partial differential equations, some inverse source problems are treated statistically based on nonparametric estimation ideas. By observing the solution in a small Gaussian white noise, the kernel type of estimators is used to estimate the unknown source function and its partial derivatives.. It is proved that such estimators are consistent as the noise intensity tends to zero. Depending on the principal part of the differential operator, the optimal asymptotic rate of convergence is ascertained within a wide class of risk functions in a minimax sense. Received: 5 May 1997 / Revised version: 18 June 1998     

Exponential mixing for stochastic PDEs: the non-additive case

We establish a general criterion which ensures exponential mixing of parabolic stochastic partial differential equations (SPDE) driven by a non additive noise which is white in time and smooth in space. We apply this criterion on two representative examples: 2D Navier–Stokes (NS) equations and Complex Ginzburg–Landau (CGL) equation with a locally Lipschitz noise. Due to the possible degeneracy of the noise, Doob theorem cannot be applied. Hence, a coupling method is used in the spirit of Kuksin and Shirikyan (J. Math. Pures Appl. 1:567–602, 2002) and Mattingly (Commun. Math. Phys. 230:421–462, 2002). Previous results require assumptions on the covariance of the noise which might seem restrictive and artificial. For instance, for NS and CGL, the covariance operator is supposed to be diagonal in the eigenbasis of the Laplacian and not depending on the high modes of the solutions. The method developed in the present paper gets rid of such assumptions and only requires that the range of the covariance operator contains the low modes.     

Asymptotic minimax risk for sup-norm loss: Solution via optimal recovery

Summary We study the problem of estimating an unknown function on the unit interval (or itsk-th derivative), with supremum norm loss, when the function is observed in Gaussian white noise and the unknown function is known only to obey Lipschitz- smoothness, >k0. We discuss an optimization problem associated with the theory ofoptimal recovery. Although optimal recovery is concerned with deterministic noise chosen by a clever opponent, the solution of this problem furnishes the kernel of the minimax linear estimate for Gaussian white noise. Moreover, this minimax linear estimator is asymptotically minimax among all estimates. We sketch also applications to higher dimensions and to indirect measurement (e.g. deconvolution) problems.Dedicated to R.Z. Khas'minskii for his 60th birthday