Explicit estimators of parameters in the Growth Curve model with linearly structured covariance matrices

Estimation of parameters in the classical Growth Curve model, when the covariance matrix has some specific linear structure, is considered. In our examples maximum likelihood estimators cannot be obtained explicitly and must rely on optimization algorithms. Therefore explicit estimators are obtained as alternatives to the maximum likelihood estimators. From a discussion about residuals, a simple non-iterative estimation procedure is suggested which gives explicit and consistent estimators of both the mean and the linear structured covariance matrix.     

Maximum Likelihood Estimation of Asymmetric Laplace Parameters

Maximum likelihood estimators (MLE's) are presented for the parameters of a univariate asymmetric Laplace distribution for all possible situations related to known or unknown parameters. These estimators admit explicit form in all but two cases. In these exceptions effective algorithms for computing the estimators are provided. Asymptotic distributions of the estimators are given. The asymptotic normality and consistency of the MLE's for the scale and location parameters are derived directly via representations of the relevant random variables rather than from general sufficient conditions for asymptotic normality of the MLE's.     

Second-order accurate inference on eigenvalues of covariance and correlation matrices

Edgeworth expansions and saddlepoint approximations for the distributions of estimators of certain eigenfunctions of covariance and correlation matrices are developed. These expansions depend on second-, third-, and fourth-order moments of the sample covariance matrix. Expressions for and estimators of these moments are obtained. The expansions and moment expressions are used to construct second-order accurate confidence intervals for the eigenfunctions. The expansions are illustrated and the results of a small simulation study that evaluates the finite-sample performance of the confidence intervals are reported.     

Statistical inference for a certain class of bivariate distributions

The paper deals with statistical inference for a certain class of bivariate distributions. The class of marginal distributions is given and is shown to include distributions with only location and scale parameters. A normalizing transformation is applied to the marginal distributions and the parameters are estimated by maximum likelihood. For this class there is a great deal of simplification in the calculations for the asymptotic covariance matrix of the vector of parameter estimators. Statistics for tests of zero correlation are discussed. Also, the analysis is carried out for exponential marginal distributions.     

Improved estimators for the GMANOVA problem with application to Monte Carlo simulation

The problem of finding classes of estimators which improve upon the usual (e.g., ML, LS) estimator of the parameter matrix in the GMANOVA model under (matrix) quadratic loss is considered. Classes of improved estimators are obtained via combining integration-by-parts methods for normal and Wishart distributions. Also considered is the application of control variates to achieve better efficiency in multipopulation multivariate simulation studies.     

Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure

In this paper we study the properties of a kurtosis matrix and propose its eigenvectors as interesting directions to reveal the possible cluster structure of a data set. Under a mixture of elliptical distributions with proportional scatter matrix, it is shown that a subset of the eigenvectors of the fourth-order moment matrix corresponds to Fisher’s linear discriminant subspace. The eigenvectors of the estimated kurtosis matrix are consistent estimators of this subspace and its calculation is easy to implement and computationally efficient, which is particularly favourable when the ratio n/p is large.     

Asymptotic properties of some subset vector autoregressive process estimators

We establish consistency and derive asymptotic distributions for estimators of the coefficients of a subset vector autoregressive (SVAR) process. Using a martingale central limit theorem, we first derive the asymptotic distribution of the subset least squares (LS) estimators. Exploiting the similarity of closed form expressions for the LS and Yule–Walker (YW) estimators, we extend the asymptotics to the latter. Using the fact that the subset Yule–Walker and recently proposed Burg estimators satisfy closely related recursive algorithms, we then extend the asymptotic results to the Burg estimators. All estimators are shown to have the same limiting distribution.     

Extensions of saddlepoint-based bootstrap inference

We propose two substantive extensions to the saddlepoint-based bootstrap (SPBB) methodology, whereby inference in parametric models is made through a monotone quadratic estimating equation (QEE). These are motivated through the first-order moving average model, where SPBB application is complicated by the fact that the usual estimators, method of moments (MOME), least squares, and maximum likelihood (MLE), all have mixed distributions and tend to be roots of high-order polynomials that violate the monotonicity requirement. A unifying perspective is provided by demonstrating that these estimators can all be cast as roots of appropriate QEEs. The first extension consists of two double saddlepoint-based Monte Carlo algorithms for approximating the Jacobian term appearing in the approximated density function of estimators derived from a non-monotone QEE. The second extension considers inference under QEEs from exponential power families. The methods are demonstrated for the MLE under a Gaussian distribution, and the MOME under a joint Laplace distribution for the process.     

Shrinkage estimators for large covariance matrices in multivariate real and complex normal distributions under an invariant quadratic loss

The problem of estimating large covariance matrices of multivariate real normal and complex normal distributions is considered when the dimension of the variables is larger than the number of samples. The Stein–Haff identities and calculus on eigenstructure for singular Wishart matrices are developed for real and complex cases, respectively. By using these techniques, the unbiased risk estimates for certain classes of estimators for the population covariance matrices under invariant quadratic loss functions are obtained for real and complex cases, respectively. Based on the unbiased risk estimates, shrinkage estimators which are counterparts of the estimators due to Haff [L.R. Haff, Empirical Bayes estimation of the multivariate normal covariance matrix, Ann. Statist. 8 (1980) 586–697] are shown to improve upon the best scalar multiple of the empirical covariance matrix under the invariant quadratic loss functions for both real and complex multivariate normal distributions in the situation where the dimension of the variables is larger than the number of samples.     

Estimation of normal covariance matrices parametrized by irreducible symmetric cones under Stein's loss

In this paper the problem of estimating a covariance matrix parametrized by an irreducible symmetric cone in a decision-theoretic set-up is considered. By making use of some results developed in a theory of finite-dimensional Euclidean simple Jordan algebras, Bartlett's decomposition and an unbiased risk estimate formula for a general family of Wishart distributions on the irreducible symmetric cone are derived; these results lead to an extension of Stein's general technique for derivation of minimax estimators for a real normal covariance matrix. Specification of the results to the multivariate normal models with covariances which are parametrized by complex, quaternion, and Lorentz types gives minimax estimators for each model.     

Efficiency of the even-frequency method for estimating the parameters of a three-parameter compound Poisson distribution

Point estimation issues are investigated for the three-parameter family of compound Poisson distributions. Formulas for parameter estimators and their covariance matrix are derived by the even-frequency method; the asymptotic efficiency of the estimators is expressed with an informant in series form. The efficiency of even-frequency estimation is computed and analyzed for the parameter characteristics. __________ Translated from Prikladnaya Matematika i Informatika, No. 25, pp. 99–106, 2007.     

Methods for improvement in estimation of a normal mean matrix

This paper is concerned with the problem of estimating a matrix of means in multivariate normal distributions with an unknown covariance matrix under invariant quadratic loss. It is first shown that the modified Efron-Morris estimator is characterized as a certain empirical Bayes estimator. This estimator modifies the crude Efron-Morris estimator by adding a scalar shrinkage term. It is next shown that the idea of this modification provides a general method for improvement of estimators, which results in the further improvement on several minimax estimators. As a new method for improvement, an adaptive combination of the modified Stein and the James-Stein estimators is also proposed and is shown to be minimax. Through Monte Carlo studies of the risk behaviors, it is numerically shown that the proposed, combined estimator inherits the nice risk properties of both individual estimators and thus it has a very favorable risk behavior in a small sample case. Finally, the application to a two-way layout MANOVA model with interactions is discussed.     

Asymptotic distributions of robust shape matrices and scales

It has been frequently observed in the literature that many multivariate statistical methods require the covariance or dispersion matrix Σ of an elliptical distribution only up to some scaling constant. If the topic of interest is not the scale but only the shape of the elliptical distribution, it is not meaningful to focus on the asymptotic distribution of an estimator for Σ or another matrix Γ∝Σ. In the present work, robust estimators for the shape matrix and the associated scale are investigated. Explicit expressions for their joint asymptotic distributions are derived. It turns out that if the joint asymptotic distribution is normal, the estimators presented are asymptotically independent for one and only one specific choice of the scale function. If it is non-normal (this holds for example if the estimators for the shape matrix and scale are based on the minimum volume ellipsoid estimator) only the scale function presented leads to asymptotically uncorrelated estimators. This is a generalization of a result obtained by Paindaveine [D. Paindaveine, A canonical definition of shape, Statistics and Probability Letters 78 (2008) 2240-2247] in the context of local asymptotic normality theory.     

The bias and risk functions of some Stein-rules in elliptically contoured distributions

In this paper, we derive the bias and risk functions of a class of shrinkage estimators of several mean parameter matrices of matrix-variate elliptically contoured distributions. More specifically, we generalize some recent findings in three ways. First, the class of distributions under consideration is more general than the Gaussian distribution case, which is often studied in literature. Second, the uncertain subspace candidate is more general than that considered in literature. Finally, we generalize some recent identities, which are useful in establishing the risk and the bias of matrix shrinkage estimators.     

MM Algorithms for Some Discrete Multivariate Distributions

The MM (minorization–maximization) principle is a versatile tool for constructing optimization algorithms. Every EM algorithm is an MM algorithm but not vice versa. This article derives MM algorithms for maximum likelihood estimation with discrete multivariate distributions such as the Dirichlet-multinomial and Connor–Mosimann distributions, the Neerchal–Morel distribution, the negative-multinomial distribution, certain distributions on partitions, and zero-truncated and zero-inflated distributions. These MM algorithms increase the likelihood at each iteration and reliably converge to the maximum from well-chosen initial values. Because they involve no matrix inversion, the algorithms are especially pertinent to high-dimensional problems. To illustrate the performance of the MM algorithms, we compare them to Newton’s method on data used to classify handwritten digits.     

On estimation of measurement error models with replication under heavy-tailed distributions

Measurement error (errors-in-variables) models are frequently used in various scientific fields, such as engineering, medicine, chemistry, etc. In this work, we consider a new replicated structural measurement error model in which the replicated observations jointly follow scale mixtures of normal (SMN) distributions. Maximum likelihood estimates are computed via an EM type algorithm method. A closed expression is presented for the asymptotic covariance matrix of those estimators. The SMN measurement error model provides an appealing robust alternative to the usual model based on normal distributions. The results of simulation studies and a real data set analysis confirm the robustness of SMN measurement error model.     

Nonparametric density estimation for linear processes with infinite variance

We consider nonparametric estimation of marginal density functions of linear processes by using kernel density estimators. We assume that the innovation processes are i.i.d. and have infinite-variance. We present the asymptotic distributions of the kernel density estimators with the order of bandwidths fixed as h =  cn ^−1/5, where n is the sample size. The asymptotic distributions depend on both the coefficients of linear processes and the tail behavior of the innovations. In some cases, the kernel estimators have the same asymptotic distributions as for i.i.d. observations. In other cases, the normalized kernel density estimators converge in distribution to stable distributions. A simulation study is also carried out to examine small sample properties.     

Influence functions and efficiencies of the canonical correlation and vector estimates based on scatter and shape matrices

In this paper, the influence functions and limiting distributions of the canonical correlations and coefficients based on affine equivariant scatter matrices are developed for elliptically symmetric distributions. General formulas for limiting variances and covariances of the canonical correlations and canonical vectors based on scatter matrices are obtained. Also the use of the so-called shape matrices in canonical analysis is investigated. The scatter and shape matrices based on the affine equivariant Sign Covariance Matrix as well as the Tyler's shape matrix serve as examples. Their finite sample and limiting efficiencies are compared to those of the Minimum Covariance Determinant estimators and S-estimator through theoretical and simulation studies. The theory is illustrated by an example.     

On improved estimation of normal precision matrix and discriminant coefficients

The problem of estimating the precision matrix of a multivariate normal distribution model is considered with respect to a quadratic loss function. A number of covariance estimators originally intended for a variety of loss functions are adapted so as to obtain alternative estimators of the precision matrix. It is shown that the alternative estimators have analytically smaller risks than the unbiased estimator of the precision matrix. Through numerical studies of risk values, it is shown that the new estimators have substantial reduction in risk. In addition, we consider the problem of the estimation of discriminant coefficients, which arises in linear discriminant analysis when Fisher's linear discriminant function is viewed as the posterior log-odds under the assumption that two classes differ in mean but have a common covariance matrix. The above method is also adapted for this problem in order to obtain improved estimators of the discriminant coefficients under the quadratic loss function. Furthermore, a numerical study is undertaken to compare the properties of a collection of alternatives to the “unbiased” estimator of the discriminant coefficients.     

Efficient parameter estimation via modified Cholesky decomposition for quantile regression with longitudinal data

It is well known that specifying a covariance matrix is difficult in the quantile regression with longitudinal data. This paper develops a two step estimation procedure to improve estimation efficiency based on the modified Cholesky decomposition. Specifically, in the first step, we obtain the initial estimators of regression coefficients by ignoring the possible correlations between repeated measures. Then, we apply the modified Cholesky decomposition to construct the covariance models and obtain the estimator of within-subject covariance matrix. In the second step, we construct unbiased estimating functions to obtain more efficient estimators of regression coefficients. However, the proposed estimating functions are discrete and non-convex. We utilize the induced smoothing method to achieve the fast and accurate estimates of parameters and their asymptotic covariance. Under some regularity conditions, we establish the asymptotically normal distributions for the resulting estimators. Simulation studies and the longitudinal progesterone data analysis show that the proposed approach yields highly efficient estimators.