An adjusted maximum likelihood method for solving small area estimation problems

For the well-known Fay-Herriot small area model, standard variance component estimation methods frequently produce zero estimates of the strictly positive model variance. As a consequence, an empirical best linear unbiased predictor of a small area mean, commonly used in small area estimation, could reduce to a simple regression estimator, which typically has an overshrinking problem. We propose an adjusted maximum likelihood estimator of the model variance that maximizes an adjusted likelihood defined as a product of the model variance and a standard likelihood (e.g., a profile or residual likelihood) function. The adjustment factor was suggested earlier by Carl Morris in the context of approximating a hierarchical Bayes solution where the hyperparameters, including the model variance, are assumed to follow a prior distribution. Interestingly, the proposed adjustment does not affect the mean squared error property of the model variance estimator or the corresponding empirical best linear unbiased predictors of the small area means in a higher order asymptotic sense. However, as demonstrated in our simulation study, the proposed adjustment has a considerable advantage in small sample inference, especially in estimating the shrinkage parameters and in constructing the parametric bootstrap prediction intervals of the small area means, which require the use of a strictly positive consistent model variance estimate.     

Stochastic response restrictions

This paper considers the implementation of prior stochastic information on unknown outcomes of the response variables into estimation and forecasting of systems of linear regression equations in the context of time series, cross sections, pooled and longitudinal data models. The established approach proves particularly useful when only aggregated information on the response variables is available, as is frequently the case in applied statistics. We address the combination of prior stochastic and sample information as an extension of standard Gauss-Markov theory. Prior stochastic information could be given in the form of experts' expectations, or from estimations and/or projections of other models. A classical (i.e. non-Bayesian) regression framework for the incorporation of prior knowledge in generalized least-squares estimation and prediction is developed.     

Robust Gaussian graphical modeling

A new Gaussian graphical modeling that is robustified against possible outliers is proposed. The likelihood function is weighted according to how the observation is deviated, where the deviation of the observation is measured based on its likelihood. Test statistics associated with the robustified estimators are developed. These include statistics for goodness of fit of a model. An outlying score, similar to but more robust than the Mahalanobis distance, is also proposed. The new scores make it easier to identify outlying observations. A Monte Carlo simulation and an analysis of a real data set show that the proposed method works better than ordinary Gaussian graphical modeling and some other robustified multivariate estimators.     

Testing model assumptions in functional regression models

In the functional regression model where the responses are curves, new tests for the functional form of the regression and the variance function are proposed, which are based on a stochastic process estimating L²-distances. Our approach avoids the explicit estimation of the functional regression and it is shown that normalized versions of the proposed test statistics converge weakly. The finite sample properties of the tests are illustrated by means of a small simulation study. It is also demonstrated that for small samples, bootstrap versions of the tests improve the quality of the approximation of the nominal level.     

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara-Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.     

Multivariate generalized S-estimators

In this paper we introduce generalized S-estimators for the multivariate regression model. This class of estimators combines high robustness and high efficiency. They are defined by minimizing the determinant of a robust estimator of the scatter matrix of differences of residuals. In the special case of a multivariate location model, the generalized S-estimator has the important independence property, and can be used for high breakdown estimation in independent component analysis. Robustness properties of the estimators are investigated by deriving their breakdown point and the influence function. We also study the efficiency of the estimators, both asymptotically and at finite samples. To obtain inference for the regression parameters, we discuss the fast and robust bootstrap for multivariate generalized S-estimators. The method is illustrated on a real data example.     

Partially linear single-index beta regression model and score test

An important model in handling the multivariate data is the partially linear single-index regression model with a very flexible distribution—beta distribution, which is commonly used to model data restricted to some open intervals on the line. In this paper, the score test is extended to the partially linear single-index beta regression model. The penalized likelihood estimation based on P-spline is proposed. Based on the estimation, the score test statistics about varying dispersion parameter is given. Its asymptotical property is investigated. Both simulated examples are used to illustrate our proposed methods.     

Uncertainty under a multivariate nested-error regression model with logarithmic transformation

This work aims to predict exponentials of mixed effects under a multivariate linear regression model with one random factor. Such quantities are of particular interest in prediction problems where the dependent variable is the logarithm of the variable that is the object of inference. Bias-corrected empirical predictors of the target quantities are defined. A second-order approximation for the mean crossed product error of two of these predictors is obtained, where the mean squared error is a particular case. An estimator of the mean crossed product error with second-order bias is proposed. Finally, results are illustrated through an application related to small area estimation.     

On prediction rate in partial functional linear regression

We consider a prediction of a scalar variable based on both a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, it is inevitable to impose some kind of regularization. We consider two different approaches, which are shown to achieve the same convergence rate of the mean squared prediction error under respective assumptions. One is based on functional principal components regression (FPCR) and the alternative is functional ridge regression (FRR) based on Tikhonov regularization. Also, numerical studies are carried out for a simulation data and a real data.     

Restricted estimation in multivariate measurement error regression model

We study a multivariate ultrastructural measurement error (MUME) model with more than one response variable. This model is a synthesis of multivariate functional and structural models. Three consistent estimators of regression coefficients, satisfying the exact linear restrictions have been proposed. Their asymptotic distributions are derived under the assumption of a non-normal measurement error and random error components. A simulation study is carried out to investigate the small sample properties of the estimators. The effect of departure from normality of the measurement errors on the estimators is assessed.     

Estimation in generalized linear models for functional data via penalized likelihood

We analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation. The L² rate of convergence of this estimator is given under smoothness assumption on the functional coefficient. Heuristic arguments show how these rates may be improved for some particular frameworks.     

Bayesian prediction for the two-parameter exponential distribution based on type II doubly censoring

The two-parameter exponential distribution is proposed to be an underlying model, and prediction bounds for future observations are obtained by using Bayesian approach. Prediction intervals are derived for unobserved lifetimes in one-sample prediction and two-sample prediction based on type II doubly censored samples. A numerical example is given to illustrate the procedures, prediction intervals are investigated via Monte Carlo method, and the accuracy of prediction intervals is presented. Supported by the National Natural Science Foundation of China (79970022) and Aviation Fund (02J53079).     

Low dimensional semiparametric estimation in a censored regression model

A new estimation procedure for a partial linear additive model with censored responses is proposed. To this aim, ideas of Lewbel and Linton [A. Lewbel, O. Linton, Nonparametric censored and truncated regression, Econometrica 70 (2002) 765-779] on censored model regression are combined with those of Kim et al. [W. Kim, O. Linton, N.W. Hengartner, A computationally efficient estimator for additive nonparametric regression with bootstrap confidence intervals, Journal of Computational and Graphical Statistics, 8 (1999) 278-297] on marginal integration and those on average derivatives. This allows for dimension reduction, interpretability and — depending on the context — for weights yielding computationally attractive estimates. Asymptotic behavior is provided for all proposed estimators.     

Linear prediction in functional data analysis

In this paper we introduce a new perspective of linear prediction in the functional data context that predicts a scalar response by observing a functional predictor. This perspective broadens the scope of functional linear prediction currently in the literature, which is exclusively focused on the functional linear regression model. It also provides a natural link to the classical linear prediction theory. Based on this formulation, we derive the convergence rate of the optimal mean squared predictor.     

Wavelet-Based Weighted LASSO and Screening Approaches in Functional Linear Regression

One useful approach for fitting linear models with scalar outcomes and functional predictors involves transforming the functional data to wavelet domain and converting the data-fitting problem to a variable selection problem. Applying the LASSO procedure in this situation has been shown to be efficient and powerful. In this article, we explore two potential directions for improvements to this method: techniques for prescreening and methods for weighting the LASSO-type penalty. We consider several strategies for each of these directions which have never been investigated, either numerically or theoretically, in a functional linear regression context. We compare the finite-sample performance of the proposed methods through both simulations and real-data applications with both 1D signals and 2D image predictors. We also discuss asymptotic aspects. We show that applying these procedures can lead to improved estimation and prediction as well as better stability. Supplementary materials for this article are available online.     

Minimax estimation for singular linear multivariate models with mixed uncertainty

The problem of minimax estimation is examined for the linear multivariate statistically indeterminate observation model with mixed uncertainty. The a priori information on the distributions of model parameters is formulated in terms of second-order moment characteristics. It is shown that in the regular case the minimax estimate is defined explicitly via the solution of the dual optimization problem. For singular models, the method of dual optimization is developed by means of using the Tikhonov regularization techniques. Several particular cases which are widely used in practice are also considered.     

Special variance structures in the growth curve model

Standard and extended growth curve model (multivariate linear model) with practically important variance structures are considered and a method for parameters estimation is proposed.     

Bayes estimation in linear models: A coordinate-free approach

The unified theory of Bayes estimation in linear models is presented, using a coordinate-free approach. The results are applied to the problem of linear and quadratic estimation in linear regression model.     

Autoregressive process modeling via the Lasso procedure

The Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double asymptotic framework where the maximal lag may increase with the sample size. We derive theoretical results establishing various types of consistency. In particular, we derive conditions under which the Lasso estimator for the autoregressive coefficients is model selection consistent, estimation consistent and prediction consistent. Simulation study results are reported.     

Theory and inference for regression models with missing responses and covariates

In this paper, we carry out an in-depth theoretical investigation for inference with missing response and covariate data for general regression models. We assume that the missing data are missing at random (MAR) or missing completely at random (MCAR) throughout. Previous theoretical investigations in the literature have focused only on missing covariates or missing responses, but not both. Here, we consider theoretical properties of the estimates under three different estimation settings: complete case (CC) analysis, a complete response (CR) analysis that involves an analysis of those subjects with only completely observed responses, and the all case (AC) analysis, which is an analysis based on all of the cases. Under each scenario, we derive general expressions for the likelihood and devise estimation schemes based on the EM algorithm. We carry out a theoretical investigation of the three estimation methods in the normal linear model and analytically characterize the loss of information for each method, as well as derive and compare the asymptotic variances for each method assuming the missing data are MAR or MCAR. In addition, a theoretical investigation of bias for the CC method is also carried out. A simulation study and real dataset are given to illustrate the methodology.