基于局地和非局地观测算子的GPS 掩星资料后向映射四维变分同化研究

本文通过在区域暴雨预报模式(AREM) 的后向映射四维变分同化系统(AREM-B4DVar) 中引入GPS 掩星折射率局地和非局地两种观测算子, 使得该系统具备了同化全球定位系统(GPS) 掩星折射率资料的能力, 并采用台湾地区与美国联合执行的气象、电离层和气候星座观测系统计划(COSMIC计划) 探测得到的GPS 掩星折射率资料和常规探空资料, 对2007 年7 月4 日至5 日发生在我国江淮流域的暴雨个例进行了同化预报试验. 结果表明, 在同化系统中采用局地和非局地两种观测算子, 加入GPS 掩星折射率资料后, 均可以提高观测资料附近初值的分析质量, 从而在改进24 小时的降水预报中起到正效果; 基于非局地观测算子的掩星折射率资料同化可以通过大气非局地的约束, 进一步改进基于局地观测算子掩星折射率资料同化的不足; 在常规资料的基础上加入掩星折射率资料, 可以使同化系统进一步改进初值分析质量和24 小时预报效果, 尤其能更好地发挥非局地观测算子的作用.     

L1-regularisation for ill-posed problems in variational data assimilation

We consider four-dimensional variational data assimilation (4DVar) and show that it can be interpreted as Tikhonov or L₂-regularisation, a widely used method for solving ill-posed inverse problems. It is known from image restoration and geophysical problems that an alternative regularisation, namely L₁-norm regularisation, recovers sharp edges better than L₂-norm regularisation. We apply this idea to 4DVar for problems where shocks and model error are present and give two examples which show that L₁-norm regularisation performs much better than the standard L₂-norm regularisation in 4DVar. (© 2010 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Sparse Steinian Covariance Estimation

We consider a new method for sparse covariance matrix estimation which is motivated by previous results for the so-called Stein-type estimators. Stein proposed a method for regularizing the sample covariance matrix by shrinking together the eigenvalues; the amount of shrinkage is chosen to minimize an unbiased estimate of the risk (UBEOR) under the entropy loss function. The resulting estimator has been shown in simulations to yield significant risk reductions over the maximum likelihood estimator. Our method extends the UBEOR minimization problem by adding an ?₁ penalty on the entries of the estimated covariance matrix, which encourages a sparse estimate. For a multivariate Gaussian distribution, zeros in the covariance matrix correspond to marginal independences between variables. Unlike the ?₁-penalized Gaussian likelihood function, our penalized UBEOR objective is convex and can be minimized via a simple block coordinate descent procedure. We demonstrate via numerical simulations and an analysis of microarray data from breast cancer patients that our proposed method generally outperforms other methods for sparse covariance matrix estimation and can be computed efficiently even in high dimensions.     

Some tests for the covariance matrix with fewer observations than the dimension under non-normality

This article analyzes whether some existing tests for the p×p covariance matrix Σ of the N independent identically distributed observation vectors work under non-normality. We focus on three hypotheses testing problems: (1) testing for sphericity, that is, the covariance matrix Σ is proportional to an identity matrix I_p; (2) the covariance matrix Σ is an identity matrix I_p; and (3) the covariance matrix is a diagonal matrix. It is shown that the tests proposed by Srivastava (2005) for the above three problems are robust under the non-normality assumption made in this article irrespective of whether N≤p or N≥p, but (N,p)→∞, and N/p may go to zero or infinity. Results are asymptotic and it may be noted that they may not hold for finite (N,p).     

并行准高斯高阶递归滤波算法研究

三维变分同化系统中一个重要的问题是背景误差协方差矩阵B及其逆的求解.背景误差协方差矩阵的水平变换部分采用递归滤波运算,可以简化矩阵的求解,解决了背景误差协方差矩阵B及其逆难以求解的问题.本文对准高斯高阶递归滤波的算法原理和过程进行了深入研究.因为递归滤波并行的低可扩展性制约了高阶递归滤波算法在三维变分同化系统中的应用,所以本文提出了阶段二维区域剖分并行化方法,实现了并行准高斯高阶递归滤波算法库.数值试验表明,四阶递归滤波1次的效果明显优于一阶4次的滤波效果;并且高阶递归滤波并行算法64核时能达到大约50倍的加速,并行效率高达78%,具有良好的加速效果和较强的可扩展性.     

Asymptotic distributions of the latent roots of the covariance matrix with multiple population roots

An asymptotic expansion for large sample size n is derived by a partial differential equation method, up to and including the term of order n^?2, for the ₀F₀ function with two argument matrices which arise in the joint density function of the latent roots of the covariance matrix, when some of the population latent roots are multiple. Then we derive asymptotic expansions for the joint and marginal distributions of the sample roots in the case of one multiple root.     

The conditioning of least‐squares problems in variational data assimilation

In variational data assimilation a least‐squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least‐squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least‐squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian. We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.     

A computationally efficient Kalman smoother for the evaluation of the CH4 budget in Europe

Two data assimilation methods of the Kalman filtering approach are applied to the evaluation of the methane (CH₄) distribution in the atmosphere of Europe. The long term historical observation data of CH₄ concentration are integrated with the dynamical Eulerian dispersion model (EUROS). In each proposed method a specific algorithm is employed to avoid the heavy computation burden and huge storage requirement of the conventional Kalman filter for large scale systems. Moreover, a smoother algorithm is developed to identify the emission input. The feasibility of proposed data assimilation methods is verified by the application results.     

Updating of moments

Consider a statistical model, given by the distribution of the observation X, conditional on the parameter θ, and the prior distribution of the parameter θ. Let H_x denote the function that maps the prior mean and the prior covariance matrix into the posterior mean and the posterior covariance matrix, when X = x is observed. We prove that if the conditional distribution of X belongs to an exponential family, then the function H_x characterizes the distribution of Xθ.     

Penalized Covariance Matrix Estimation Using a Matrix-Logarithm Transformation

For statistical inferences that involve covariance matrices, it is desirable to obtain an accurate covariance matrix estimate with a well-structured eigen-system. We propose to estimate the covariance matrix through its matrix logarithm based on an approximate log-likelihood function. We develop a generalization of the Leonard and Hsu log-likelihood approximation that no longer requires a nonsingular sample covariance matrix. The matrix log-transformation provides the ability to impose a convex penalty on the transformed likelihood such that the largest and smallest eigenvalues of the covariance matrix estimate can be regularized simultaneously. The proposed method transforms the problem of estimating the covariance matrix into the problem of estimating a symmetric matrix, which can be solved efficiently by an iterative quadratic programming algorithm. The merits of the proposed method are illustrated by a simulation study and two real applications in classification and portfolio optimization. Supplementary materials for this article are available online.     

The asymptotic distribution of a goodness of fit statistic for factorial invariance

Suppose that random factor models with k factors are assumed to hold for m, p-variate populations. A model for factorial invariance has been proposed wherein the covariance or correlation matrices can be written as Σ_i = LC_iL′ + σ_i²I, where C_i is the covariance matrix of factor variables and L is a common factor loading matrix, i = 1,…, m. Also a goodness of fit statistic has been proposed for this model. The asymptotic distribution of this statistic is shown to be that of a quadratic form in normal variables. An approximation to this distribution is given and thus a test for goodness of fit is derived. The problem of dimension is considered and a numerical example is given to illustrate the results.     

Performance Analysis of Local Ensemble Kalman Filter

Ensemble Kalman filter (EnKF) is an important data assimilation method for high-dimensional geophysical systems. Efficient implementation of EnKF in practice often involves the localization technique, which updates each component using only information within a local radius. This paper rigorously analyzes the local EnKF (LEnKF) for linear systems and shows that the filter error can be dominated by the ensemble covariance, as long as (1) the sample size exceeds the logarithmic of state dimension and a constant that depends only on the local radius; (2) the forecast covariance matrix admits a stable localized structure. In particular, this indicates that with small system and observation noises, the filter error will be accurate in long time even if the initialization is not. The analysis also reveals an intrinsic inconsistency caused by the localization technique, and a stable localized structure is necessary to control this inconsistency. While this structure is usually taken for granted for the operation of LEnKF, it can also be rigorously proved for linear systems with sparse local observations and weak local interactions. These theoretical results are also validated by numerical implementation of LEnKF on a simple stochastic turbulence in two dynamical regimes.     

Bayes estimation of number of signals

Bayes estimation of the number of signals, q, based on a binomial prior distribution is studied. It is found that the Bayes estimate depends on the eigenvalues of the sample covariance matrix S for white-noise case and the eigenvalues of the matrix S ₂ (S ₁+A)^–1 for the colored-noise case, where S ₁ is the sample covariance matrix of observations consisting only noise, S ₂ the sample covariance matrix of observations consisting both noise and signals and A is some positive definite matrix. Posterior distributions for both the cases are derived by expanding zonal polynomial in terms of monomial symmetric functions and using some of the important formulae of James (1964, Ann. Math. Statist., 35, 475–501).     

The Factor Analytic Approach to Simultaneous Inference in the General Linear Model

Abstract

Consider the general linear model (GLM) Y = Xβ + ε. Suppose Θ₁,…, Θ_k, a subset of the β's, are of interest; Θ₁,…, Θ_k may be treatment contrasts in an ANOVA setting or regression coefficients in a response surface setting. Existing simultaneous confidence intervals for Θ₁,…, Θ_k are relatively conservative or, in the case of the MEANS option in PROC GLM of SAS, possibly misleading. The difficulty is with the multidimensionality of the integration required to compute exact coverage probability when X does not follow a nice textbook design. Noting that such exact coverage probabilities can be computed if the correlation matrix R of the estimators of Θ₁, …, Θ_k has a one-factor structure in the factor analytic sense, it is proposed that approximate simultaneous confidence intervals be computed by approximating R with the closest one-factor structure correlation matrix. Computer simulations of hundreds of randomly generated designs in the settings of regression, analysis of covariance, and unbalanced block designs show that the coverage probabilities are practically exact, more so than can be anticipated by even second-order probability bounds.     

High-Dimensional Covariance Matrix Estimation Based on Network

A new method for estimating high-dimensional covariance matrix based on network structure with heteroscedasticity of response variables is proposed in this paper. This method greatly reduces the computational complexity by transforming the high-dimensional covariance matrix estimation problem into a low-dimensional linear regression problem. Even if the size of sample is finite, the estimation method is still effective. The error of estimation will decrease with the increase of matrix dimension. In addition, this paper presents a method of identifying influential nodes in network via covariance matrix. This method is very suitable for academic cooperation networks by taking into account both the contribution of the node itself and the impact of the node on other nodes.     

Inexact proximal Newton methods for self-concordant functions

We analyze the proximal Newton method for minimizing a sum of a self-concordant function and a convex function with an inexpensive proximal operator. We present new results on the global and local convergence of the method when inexact search directions are used. The method is illustrated with an application to L1-regularized covariance selection, in which prior constraints on the sparsity pattern of the inverse covariance matrix are imposed. In the numerical experiments the proximal Newton steps are computed by an accelerated proximal gradient method, and multifrontal algorithms for positive definite matrices with chordal sparsity patterns are used to evaluate gradients and matrix-vector products with the Hessian of the smooth component of the objective.     

A sequential method for preventive maintenance and replacement of a repairable single-unit system

This paper is concerned with when to implement preventive maintenance (PM) and replacement for a repairable ‘single-unit’ system in use. Under the main assumption that a ‘single-unit’ system gradually deteriorates with time, a sequential method is proposed to determine an optimal PM and replacement strategy for the system based on minimising expected loss rate. According to this method, PM epochs are determined one after the other, and consequently we can make use of all previous information on the operation process of the system. Also the replacement epoch depends on the effective age of the system. A numerical example shows that the sequential method can be used to solve the PM and replacement problem of a ‘single-unit’ system efficiently. Some properties of the loss functions W(L? _n ,b? _n ) and W? _r (N) with respect to PM and replacement respectively are discussed in the appendix.     

Point estimation for multi-spectral distributed random matrices

In this communication, we consider a p×n random matrix which is normally distributed with mean matrix M and covariance matrix Σ, where the multivariate observation x_i=y_i+?_i with p dimensions on an object consists of two components, the signal y_i with mean vector μ and covariance matrix Σ_s and noise with mean vector zero and covariance matrix Σ_?, then the covariance matrix of x_i and x_j is given by Σ=Cov(x_i,x_j)=Γ⊗(B_|i-j|Σ_s+C_|i-j|Σ_?), where Γ is a correlation matrix; B_|i-j| and C_|i-j| are diagonal constant matrices. The statistical objective is to consider the maximum likelihood estimate of the mean matrix M and various components of the covariance matrix Σ as well as their statistical properties, that is the point estimates of Σ_s,Σ_? and Γ. More importantly, some properties of these estimators are investigated in slightly more general models.     

Wishart distributions associated with matrix quadratic forms

For a normally distributed random matrix Y with mean zero and general covariance matrix Σ_Y and for a symmetric matrix W, necessary and sufficient conditions are derived for the Wishartness of Y′WY.     

Spatial Localization for Nonlinear Dynamical Stochastic Models for Excitable Media

Nonlinear dynamical stochastic models are ubiquitous in different areas. Their statistical properties are often of great interest, but are also very challenging to compute. Many excitable media models belong to such types of complex systems with large state dimensions and the associated covariance matrices have localized structures. In this article, a mathematical framework to understand the spatial localization for a large class of stochastically coupled nonlinear systems in high dimensions is developed. Rigorous \linebreak mathematical analysis shows that the local effect from the diffusion results in an exponential decay of the components in the covariance matrix as a function of the distance while the global effect due to the mean field interaction synchronizes different components and contributes to a global covariance. The analysis is based on a comparison with an appropriate linear surrogate model, of which the covariance propagation can be computed explicitly. Two important applications of these theoretical results are discussed. They are the spatial averaging strategy for efficiently sampling the covariance matrix and the localization technique in data assimilation. Test examples of a linear model and a stochastically coupled FitzHugh-Nagumo model for excitable media are adopted to validate the theoretical results. The latter is also used for a systematical study of the spatial averaging strategy in efficiently sampling the covariance matrix in different dynamical regimes.