PLS regression: A directional signal-to-noise ratio approach

We present a new approach to univariate partial least squares regression (PLSR) based on directional signal-to-noise ratios (SNRs). We show how PLSR, unlike principal components regression, takes into account the actual value and not only the variance of the ordinary least squares (OLS) estimator. We find an orthogonal sequence of directions associated with decreasing SNR. Then, we state partial least squares estimators as least squares estimators constrained to be null on the last directions. We also give another procedure that shows how PLSR rebuilds the OLS estimator iteratively by seeking at each step the direction with the largest difference of signals over the noise. The latter approach does not involve any arbitrary scale or orthogonality constraints.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

On the conic section fitting problem

Adjusted least squares (ALS) estimators for the conic section problem are considered. Consistency of the translation invariant version of ALS estimator is proved. The similarity invariance of the ALS estimator with estimated noise variance is shown. The conditions for consistency of the ALS estimator are relaxed compared with the ones of the paper Kukush et al. [Consistent estimation in an implicit quadratic measurement error model, Comput. Statist. Data Anal. 47(1) (2004) 123-147].     

A comparison of minimax and least squares estimators in linear regression with polyhedral prior information

We consider the linear regression model where prior information in the form of linear inequalities restricts the parameter space to a polyhedron. Since the linear minimax estimator has, in general, to be determined numerically, it was proposed to minimize an upper bound of the maximum risk instead. The resulting so-called quasiminimax estimator can be easily calculated in closed form. Unfortunately, both minimax estimators may violate the prior information. Therefore, we consider projection estimators which are obtained by projecting the estimate in an optional second step. The performance of these estimators is investigated in a Monte Carlo study together with several least squares estimators, including the inequality restricted least squares estimator. It turns out that both the projected and the unprojected quasiminimax estimators have the best average performance.     

Biased estimation and hypothesis testing in linear regression

We consider a test of the simple hypothesis =₀ based on some biased estimator. Under a certain condition the corresponding test statistic coincides with the usualF-statistic based on the least squares estimator. Surprisingly, this condition is met by several well-known biased estimators.     

A class of proper priors for Bayesian simultaneous prediction of independent Poisson observables

Simultaneous prediction and parameter inference for the independent Poisson observables model are considered. A class of proper prior distributions for Poisson means is introduced. Bayesian predictive densities and estimators based on priors in the introduced class dominate the Bayesian predictive density and estimator based on the Jeffreys prior under Kullback-Leibler loss.     

On an asymptotically more efficient estimation of the single-index model

In this note, we revisit the single-index model with heteroscedastic error, and recommend an estimating equation method in terms of transferring restricted least squares to unrestricted least squares: the estimator of the index parameter is asymptotically more efficient than existing estimators in the literature in the sense that it is of a smaller limiting variance.     

A family of estimators for multivariate kurtosis in a nonnormal linear regression model

In this paper, we propose a new estimator for a kurtosis in a multivariate nonnormal linear regression model. Usually, an estimator is constructed from an arithmetic mean of the second power of the squared sample Mahalanobis distances between observations and their estimated values. The estimator gives an underestimation and has a large bias, even if the sample size is not small. We replace this squared distance with a transformed squared norm of the Studentized residual using a monotonic increasing function. Our proposed estimator is defined by an arithmetic mean of the second power of these squared transformed squared norms with a correction term and a tuning parameter. The correction term adjusts our estimator to an unbiased estimator under normality, and the tuning parameter controls the sizes of the squared norms of the residuals. The family of our estimators includes estimators based on ordinary least squares and predicted residuals. We verify that the bias of our new estimator is smaller than usual by constructing numerical experiments.     

Hypothesis testing using affine linear estimators

The general linear hypothesis is usually tested by means of anF-statistic dependent on the least squares estimator. In this paper, a class of linear estimators is identified which can also serve as a basis for such anF-statistic. Conditions are derived under which thisF-statistic coincides with the usual one. This opens the possibility of constructing minimax-estimators which dominate LS with respect to risk, yielding the same test results.Support by Deutsche Forschungsgemeinschaft, Grant No. Tr 253/1-2 is gratefully acknowledged.     

Automatic model selection for partially linear models

We propose and study a unified procedure for variable selection in partially linear models. A new type of double-penalized least squares is formulated, using the smoothing spline to estimate the nonparametric part and applying a shrinkage penalty on parametric components to achieve model parsimony. Theoretically we show that, with proper choices of the smoothing and regularization parameters, the proposed procedure can be as efficient as the oracle estimator [J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of American Statistical Association 96 (2001) 1348–1360]. We also study the asymptotic properties of the estimator when the number of parametric effects diverges with the sample size. Frequentist and Bayesian estimates of the covariance and confidence intervals are derived for the estimators. One great advantage of this procedure is its linear mixed model (LMM) representation, which greatly facilitates its implementation by using standard statistical software. Furthermore, the LMM framework enables one to treat the smoothing parameter as a variance component and hence conveniently estimate it together with other regression coefficients. Extensive numerical studies are conducted to demonstrate the effective performance of the proposed procedure.     

STRONG CONVERGENCE RATES OF SEVERAL ESTIMATORS IN SEMIPARAMETRIC VARYING-COEFFICIENT PARTIALLY LINEAR MODELS

This article is concerned with the estimating problem of semiparametric varyingcoefficient partially linear regression models. By combining the local polynomial and least squares procedures Fan and Huang （2005） proposed a profile least squares estimator for the parametric component and established its asymptotic normality. We further show that the profile least squares estimator can achieve the law of iterated logarithm. Moreover, we study the estimators of the functions characterizing the non-linear part as well as the error variance. The strong convergence rate and the law of iterated logarithm are derived for them, respectively.     

Frequentist standard errors of Bayes estimators

Frequentist standard errors are a measure of uncertainty of an estimator, and the basis for statistical inferences. Frequestist standard errors can also be derived for Bayes estimators. However, except in special cases, the computation of the standard error of Bayesian estimators requires bootstrapping, which in combination with Markov chain Monte Carlo can be highly time consuming. We discuss an alternative approach for computing frequentist standard errors of Bayesian estimators, including importance sampling. Through several numerical examples we show that our approach can be much more computationally efficient than the standard bootstrap.     

Asymptotic properties of conditional maximum likelihood estimator in a certain exponential model

The conditional maximum likelihood estimator is suggested as an alternative to the maximum likelihood estimator and is favorable for an estimator of a dispersion parameter in the normal distribution, the inverse-Gaussian distribution, and so on. However, it is not clear whether the conditional maximum likelihood estimator is asymptotically efficient in general. Consider the case where it is asymptotically efficient and its asymptotic covariance depends only on an objective parameter in an exponential model. This remand implies that the exponential model possesses a certain parallel foliation. In this situation, this paper investigates asymptotic properties of the conditional maximum likelihood estimator and compares the conditional maximum likelihood estimator with the maximum likelihood estimator. We see that the bias of the former is more robust than that of the latter and that two estimators are very close, especially in the sense of bias-corrected version. The mean Pythagorean relation is also discussed.     

A subclass of bayes linear estimators that are minimax

In the linear regression model with ellipsoidal parameter constraints, the problem of estimating the unknown parameter vector is studied. A well-described subclass of Bayes linear estimators is proposed in the paper. It is shown that for each member of this subclass, a generalized quadratic risk function exists so that the estimator is minimax. Moreover, some of the proposed Bayes linear estimators are admissible with respect to all possible generalized quadratic risks. Also, a necessary and sufficient condition is given to ensure that the considered Bayes linear estimator improves the least squares estimator over the whole ellipsoid whatever generalized risk function is chosen.     

Least median of squares estimation by optimization heuristics with an application to the CAPM and a multi-factor model

For estimating the parameters of models for financial market data, the use of robust techniques is of particular interest. Conditional forecasts, based on the capital asset pricing model, and a factor model are considered. It is proposed to consider least median of squares estimators as one possible alternative to ordinary least squares. Given the complexity of the objective function for the least median of squares estimator, the estimates are obtained by means of optimization heuristics. The performance of two heuristics is compared, namely differential evolution and threshold accepting. It is shown that these methods are well suited to obtain least median of squares estimators for real world problems. Furthermore, it is analyzed to what extent parameter estimates and conditional forecasts differ between the two estimators. The empirical analysis considers daily and monthly data on some stocks from the Dow Jones Industrial Average Index.     

Estimators of scale parameters in linear regression

This note discusses the asymptotic distribution of two scale and location invariant estimators of two scale parameters in the multiple linear regression model. Both of these estimators need an initial estimator of the regression parameter vector. The asymptotic distribution of one of these estimators does not depend on this initial estimator. Both of these estimators are useful in the computation of scale and translation invariant adaptive estimators and M-estimators of the regression parameter vector.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Improving on the maximum likelihood estimators of the means in Poisson decomposable graphical models

In this article we study the simultaneous estimation of the means in Poisson decomposable graphical models. We derive some classes of estimators which improve on the maximum likelihood estimator under the normalized squared losses. Our estimators are based on the argument in Chou [Simultaneous estimation in discrete multivariate exponential families, Ann. Statist. 19 (1991) 314-328.] and shrink the maximum likelihood estimator depending on the marginal frequencies of variables forming a complete subgraph of the conditional independence graph.     

Asymptotic expansions in sequential estimation for the first-order random coefficient autoregressive model: Regenerative approach

Edgeworth expansions for the distribution of a sequential least squares estimator in the random coefficient autoregressive (RCA) model are derived. The regenerative approach to second-order asymptotic analysis of Markov-type statistical models is developed.