Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Inference under functional proportional and common principal component models

In many situations, when dealing with several populations with different covariance operators, equality of the operators is assumed. Usually, if this assumption does not hold, one estimates the covariance operator of each group separately, which leads to a large number of parameters. As in the multivariate setting, this is not satisfactory since the covariance operators may exhibit some common structure. In this paper, we discuss the extension to the functional setting of the common principal component model that has been widely studied when dealing with multivariate observations. Moreover, we also consider a proportional model in which the covariance operators are assumed to be equal up to a multiplicative constant. For both models, we present estimators of the unknown parameters and we obtain their asymptotic distribution. A test for equality against proportionality is also considered.     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Empirical likelihood inference in partially linear single-index models for longitudinal data

The empirical likelihood method is especially useful for constructing confidence intervals or regions of parameters of interest. Yet, the technique cannot be directly applied to partially linear single-index models for longitudinal data due to the within-subject correlation. In this paper, a bias-corrected block empirical likelihood (BCBEL) method is suggested to study the models by accounting for the within-subject correlation. BCBEL shares some desired features: unlike any normal approximation based method for confidence region, the estimation of parameters with the iterative algorithm is avoided and a consistent estimator of the asymptotic covariance matrix is not needed. Because of bias correction, the BCBEL ratio is asymptotically chi-squared, and hence it can be directly used to construct confidence regions of the parameters without any extra Monte Carlo approximation that is needed when bias correction is not applied. The proposed method can naturally be applied to deal with pure single-index models and partially linear models for longitudinal data. Some simulation studies are carried out and an example in epidemiology is given for illustration.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.     

Entropy based constrained inference for some HDLSS genomic models: UI tests in a Chen-Stein perspective

For qualitative data models, Gini-Simpson index and Shannon entropy are commonly used for statistical analysis. In the context of high-dimensional low-sample size (HDLSS) categorical models, abundant in genomics and bioinformatics, the Gini-Simpson index, as extended to Hamming distance in a pseudo-marginal setup, facilitates drawing suitable statistical conclusions. Under Lorenz ordering it is shown that Shannon entropy and its multivariate analogues proposed here appear to be more informative than the Gini-Simpson index. The nested subset monotonicity prospect along with subgroup decomposability of some proposed measures are exploited. The usual jackknifing (or bootstrapping) methods may not work out well for HDLSS constrained models. Hence, we consider a permutation method incorporating the union-intersection (UI) principle and Chen-Stein Theorem to formulate suitable statistical hypothesis testing procedures for gene classification. Some applications are included as illustration.     

Statistical inference using a weighted difference-based series approach for partially linear regression models

Partially linear regression models with fixed effects are useful tools for making econometric analyses and normalizing microarray data. Baltagi and Li (2002) [7] proposed a computation friendly difference-based series estimation (DSE) for them. We show that the DSE is not asymptotically efficient in most cases and further propose a weighted difference-based series estimation (WDSE). The weights in it do not involve any unknown parameters. The asymptotic properties of the resulting estimators are established for both balanced and unbalanced cases, and it is shown that they achieve a semiparametric efficient boundary. Additionally, we propose a variable selection procedure for identifying significant covariates in the parametric part of the semiparametric fixed-effects regression model. The method is based on a combination of the nonconcave penalization (Fan and Li, 2001 [13]) and weighted difference-based series estimation techniques. The resulting estimators have the oracle property; that is, they can correctly identify the true model as if the true model (the subset of variables with nonvanishing coefficients) were known in advance. Simulation studies are conducted and an application is given to demonstrate the finite sample performance of the proposed procedures.     

The matrix-t distribution and its applications in predictive inference

The predictive distributions of the future responses and regression matrix under the multivariate elliptically contoured distributions are derived using structural approach. The predictive distributions are obtained as matrix-t which are identical to those obtained under matrix normal and matrix-t distributions. This gives inference robustness with respect to departures from the reference case of independent sampling from the matrix normal or dependent but uncorrelated sampling from matrix-t distributions. Some successful applications of matrix-t distribution in the field of spatial prediction have been addressed.     

On Stein’s lemma, dependent covariates and functional monotonicity in multi-dimensional modeling

Tracking the correct directions of monotonicity in multi-dimensional modeling plays an important role in interpreting functional associations. In the presence of multiple predictors, we provide empirical evidence that the observed monotone directions via parametric, nonparametric or semiparametric fit of commonly used multi-dimensional models may entirely violate the actual directions of monotonicity. This breakdown is caused primarily by the dependence structure of covariates, with negligible influence from the bias of function estimation. To examine the linkage between the dependent covariates and monotone directions, we first generalize Stein’s Lemma for random variables which are mutually independent Gaussian to two important cases: dependent Gaussian, and independent non-Gaussian. We show that in both two cases, there is an explicit one-to-one correspondence between the monotone directions of a multi-dimensional function and the signs of a deterministic surrogate vector. Moreover, we demonstrate that the second case can be extended to accommodate a class of dependent covariates. This generalization further enables us to develop a de-correlation transform for arbitrarily dependent covariates. The transformed covariates preserve modeling interpretability with little loss in modeling efficiency. The simplicity and effectiveness of the proposed method are illustrated via simulation studies and real data application.     

The restricted EM algorithm under inequality restrictions on the parameters

One of the most powerful algorithms for maximum likelihood estimation for many incomplete-data problems is the EM algorithm. The restricted EM algorithm for maximum likelihood estimation under linear restrictions on the parameters has been handled by Kim and Taylor (J. Amer. Statist. Assoc. 430 (1995) 708-716). This paper proposes an EM algorithm for maximum likelihood estimation under inequality restrictions A₀β?0, where β is the parameter vector in a linear model W=Xβ+ε and ε is an error variable distributed normally with mean zero and a known or unknown variance matrix Σ>0. Some convergence properties of the EM sequence are discussed. Furthermore, we consider the consistency of the restricted EM estimator and a related testing problem.     

Nonparametric likelihood based estimation for a multivariate Lipschitz density

We consider a problem of nonparametric density estimation under shape restrictions. We deal with the case where the density belongs to a class of Lipschitz functions. Devroye [L. Devroye, A Course in Density Estimation, in: Progress in Probability and Statistics, vol. 14, Birkhäuser Boston Inc., Boston, MA, 1987] considered these classes of estimates as tailor-made estimates, in contrast in some way to universally consistent estimates. In our framework we get the existence and uniqueness of the maximum likelihood estimate as well as strong consistency. This NPMLE can be easily characterized but it is not easy to compute. Some simpler approximations are also considered.     

Nonparametric inference for extrinsic means on size-and-(reflection)-shape manifolds with applications in medical imaging

For all p>2,k>p, a size-and-reflection-shape space of k-ads in general position in R^p, invariant under translation, rotation and reflection, is shown to be a smooth manifold and is equivariantly embedded in a space of symmetric matrices, allowing a nonparametric statistical analysis based on extrinsic means. Equivariant embeddings are also given for the reflection-shape-manifold , a space of orbits of scaled k-ads in general position under the group of isometries of R^p, providing a methodology for statistical analysis of three-dimensional images and a resolution of the mathematical problems inherent in the use of the Kendall shape spaces in p-dimensions, p>2. The Veronese embedding of the planar Kendall shape manifold is extended to an equivariant embedding of the size-and-shape manifold , which is useful in the analysis of size-and-shape. Four medical imaging applications are provided to illustrate the theory.     

On weighting of bivariate margins in pairwise likelihood

Composite and pairwise likelihood methods have recently been increasingly used. For clustered data with varying cluster sizes, we study asymptotic relative efficiencies for various weighted pairwise likelihoods, with weight being a function of cluster size. For longitudinal data, we also study weighted pairwise likelihoods with weights that can depend on lag. Good choice of weights are needed to avoid the undesirable behavior of estimators with low efficiency. Some analytic results are obtained using the multivariate normal distribution. For clustered data, a practically good choice of weight is obtained after study of relative efficiencies for an exchangeable multivariate normal model; they are different from weights that had previously been suggested. For longitudinal data, there are advantages to only include bivariate margins of adjacent or nearly adjacent pairs in the weighted pairwise likelihood.     

Signed-rank tests for location in the symmetric independent component model

The so-called independent component (IC) model states that the observed p-vector X is generated via X=ΛZ+μ, where μ is a p-vector, Λ is a full-rank matrix, and the centered random vector Z has independent marginals. We consider the problem of testing the null hypothesis H₀:μ=0 on the basis of i.i.d. observations X₁,…,X_n generated by the symmetric version of the IC model above (for which all ICs have a symmetric distribution about the origin). In the spirit of [M. Hallin, D. Paindaveine, Optimal tests for multivariate location based on interdirections and pseudo-Mahalanobis ranks, Annals of Statistics, 30 (2002), 1103-1133], we develop nonparametric (signed-rank) tests, which are valid without any moment assumption and are, for adequately chosen scores, locally and asymptotically optimal (in the Le Cam sense) at given densities. Our tests are measurable with respect to the marginal signed ranks computed in the collection of null residuals , where is a suitable estimate of Λ. Provided that is affine-equivariant, the proposed tests, unlike the standard marginal signed-rank tests developed in [M.L. Puri, P.K. Sen, Nonparametric Methods in Multivariate Analysis, Wiley & Sons, New York, 1971] or any of their obvious generalizations, are affine-invariant. Local powers and asymptotic relative efficiencies (AREs) with respect to Hotelling’s T² test are derived. Quite remarkably, when Gaussian scores are used, these AREs are always greater than or equal to one, with equality in the multinormal model only. Finite-sample efficiencies and robustness properties are investigated through a Monte Carlo study.     

Sequential order statistics with an order statistics prior

In the model of sequential order statistics, prior distributions are considered for the model parameters, which, for example, describe increasing load put on remaining components. Gamma priors are examined as well as priors out of a class of extended truncated Erlang distributions (ETED), which is introduced along with some properties. The choice of independent priors in both set-ups leads to respective independent, conjugate posterior distributions for the model parameters of sequential order statistics. Since, in practical applications, the model parameters will often be increasingly ordered, a multivariate prior is applied being the joint distribution of common ETED-order statistics. Whatever baseline distribution of the sequential order statistics is chosen, the joint posterior distribution turns out to be a Weinman multivariate exponential distribution. Posterior moments are given explicitly, and HPD credible sets for the model parameters are stated.     

Finite sample tail behavior of multivariate location estimators

A finite sample performance measure of multivariate location estimators is introduced based on “tail behavior”. The tail performance of multivariate “monotone” location estimators and the halfspace depth based “non-monotone” location estimators including the Tukey halfspace median and multivariate L-estimators is investigated. The connections among the finite sample performance measure, the finite sample breakdown point, and the halfspace depth are revealed. It turns out that estimators with high breakdown point or halfspace depth have “appealing” tail performance. The tail performance of the halfspace median is very appealing and also robust against underlying population distributions, while the tail performance of the sample mean is very sensitive to underlying population distributions. These findings provide new insights into the notions of the halfspace depth and breakdown point and identify the important role of tail behavior as a quantitative measure of robustness in the multivariate location setting.     

Point prediction of future order statistics from an exponential distribution

Two-sample point prediction is considered for a two-parameter exponential distribution. Several point predictors such as the best unbiased predictor, best invariant predictor and maximum likelihood predictor are obtained for future order statistics on the basis of observed record values in two cases: where the location parameter is known and unknown. These predictors are compared in the sense of their mean squared prediction errors. Finally, some numerical results are given to illustrate the proposed procedures.     

Fiducial inference on the largest mean of a multivariate normal distribution

Inference on the largest mean of a multivariate normal distribution is a surprisingly difficult and unexplored topic. Difficulties arise when two or more of the means are simultaneously the largest mean. Our proposed solution is based on an extension of R.A. Fisher’s fiducial inference methods termed generalized fiducial inference. We use a model selection technique along with the generalized fiducial distribution to allow for equal largest means and alleviate the overestimation that commonly occurs. Our proposed confidence intervals for the largest mean have asymptotically correct frequentist coverage and simulation results suggest that they possess promising small sample empirical properties. In addition to the theoretical calculations and simulations we also applied this approach to the air quality index of the four largest cities in the northeastern United States (Baltimore, Boston, New York, and Philadelphia).