Asymptotic expansions and higher order properties of semi-parametric estimators in a system of simultaneous equations

Asymptotic expansions are made for the distributions of the Maximum Empirical Likelihood (MEL) estimator and the Estimating Equation (EE) estimator (or the Generalized Method of Moments (GMM) in econometrics) for the coefficients of a single structural equation in a system of linear simultaneous equations, which corresponds to a reduced rank regression model. The expansions in terms of the sample size, when the non-centrality parameters increase proportionally, are carried out to O(n⁻¹). Comparisons of the distributions of the MEL and GMM estimators are made. Also, we relate the asymptotic expansions of the distributions of the MEL and GMM estimators to the corresponding expansions for the Limited Information Maximum Likelihood (LIML) and the Two-Stage Least Squares (TSLS) estimators. We give useful information on the higher order properties of alternative estimators including the semi-parametric inefficiency factor under the homoscedasticity assumption.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Risk bounds for model selection via penalization

Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term motivated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function. We show that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. This accuracy index quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size. If we choose a list of models which exhibit good approximation properties with respect to different classes of smoothness, the estimator can be simultaneously minimax rate optimal in each of those classes. This is what is usually called adaptation. The type of classes of smoothness in which one gets adaptation depends heavily on the list of models. If too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously, it may happen that the estimator is only approximately adaptive (typically up to a slowly varying function of the sample size). We shall provide various illustrations of our method such as penalized maximum likelihood, projection or least squares estimation. The models will involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets and related nonlinear expansions defined by superposition of ridge functions. Received: 7 July 1995 / Revised version: 1 November 1997     

Adaptive varying-coefficient linear quantile model: a profiled estimating equations approach

We consider an estimating equations approach to parameter estimation in adaptive varying-coefficient linear quantile model. We propose estimating equations for the index vector of the model in which the unknown nonparametric functions are estimated by minimizing the check loss function, resulting in a profiled approach. The estimating equations have a bias-corrected form that makes undersmoothing of the nonparametric part unnecessary. The estimating equations approach makes it possible to obtain the estimates using a simple fixed-point algorithm. We establish asymptotic properties of the estimator using empirical process theory, with additional complication due to the nuisance nonparametric part. The finite sample performance of the new model is illustrated using simulation studies and a forest fire dataset.     

The Maximum Lq-Likelihood Method: An Application to Extreme Quantile Estimation in Finance

Estimating financial risk is a critical issue for banks and insurance companies. Recently, quantile estimation based on extreme value theory (EVT) has found a successful domain of application in such a context, outperforming other methods. Given a parametric model provided by EVT, a natural approach is maximum likelihood estimation. Although the resulting estimator is asymptotically efficient, often the number of observations available to estimate the parameters of the EVT models is too small to make the large sample property trustworthy. In this paper, we study a new estimator of the parameters, the maximum Lq-likelihood estimator (MLqE), introduced by Ferrari and Yang (Estimation of tail probability via the maximum Lq-likelihood method, Technical Report 659, School of Statistics, University of Minnesota, 2007 ). We show that the MLqE outperforms the standard MLE, when estimating tail probabilities and quantiles of the generalized extreme value (GEV) and the generalized Pareto (GP) distributions. First, we assess the relative efficiency between the MLqE and the MLE for various sample sizes, using Monte Carlo simulations. Second, we analyze the performance of the MLqE for extreme quantile estimation using real-world financial data. The MLqE is characterized by a distortion parameter q and extends the traditional log-likelihood maximization procedure. When q→1, the new estimator approaches the traditional maximum likelihood estimator (MLE), recovering its desirable asymptotic properties; when q ≠ 1 and the sample size is moderate or small, the MLqE successfully trades bias for variance, resulting in an overall gain in terms of accuracy (mean squared error).      

Penalized empirical likelihood estimation of semiparametric models

We propose an empirical likelihood-based estimation method for conditional estimating equations containing unknown functions, which can be applied for various semiparametric models. The proposed method is based on the methods of conditional empirical likelihood and penalization. Thus, our estimator is called the penalized empirical likelihood (PEL) estimator. For the whole parameter including infinite-dimensional unknown functions, we derive the consistency and a convergence rate of the PEL estimator. Furthermore, for the finite-dimensional parametric component, we show the asymptotic normality and efficiency of the PEL estimator. We illustrate the theory by three examples. Simulation results show reasonable finite sample properties of our estimator.     

Estimation in semiparametric models with missing data

This paper considers the problem of parameter estimation in a general class of semiparametric models when observations are subject to missingness at random. The semiparametric models allow for estimating functions that are non-smooth with respect to the parameter. We propose a nonparametric imputation method for the missing values, which then leads to imputed estimating equations for the finite dimensional parameter of interest. The asymptotic normality of the parameter estimator is proved in a general setting, and is investigated in detail for a number of specific semiparametric models. Finally, we study the small sample performance of the proposed estimator via simulations.     

A third order optimum property of the ML estimator in a linear functional relationship model and simultaneous equation system in econometrics

Summary The maximum likelihood (ML) estimator and its modification in the linear functional relationship model with incidental parameters are shown to be third-order asymptotically efficient among a class of almost median-unbiased and almost mean-unbiased estimators, respectively, in the large sample sense. This means that the limited information maximum likelihood (LIML) estimator in the simultaneous equation system is third-order asymptotically efficient when the number of excluded exogenous variables in a particular structural equation is growing along with the sample size. It implies that the LIML estimator has an optimum property when the system of structural equations is large. The research was partly supported by National Science Foundation Grant SES 79-13976 at the Institute for Mathematical Studies in the Social Sciences, Stanford University and Grant-in-Aid 60301081 of the Ministry of Education, Science and Culture at the Faculty of Economics, University of Tokyo. This paper was originally written as a part of the author's Ph.D. dissertation submitted to Stanford University in August, 1981. Some details of the paper were deleted at the suggestion of the associate editor of this journal.     

Asymptotic theory of the adaptive Sparse Group Lasso

We study the asymptotic properties of a new version of the Sparse Group Lasso estimator (SGL), called adaptive SGL. This new version includes two distinct regularization parameters, one for the Lasso penalty and one for the Group Lasso penalty, and we consider the adaptive version of this regularization, where both penalties are weighted by preliminary random coefficients. The asymptotic properties are established in a general framework, where the data are dependent and the loss function is convex. We prove that this estimator satisfies the oracle property: the sparsity-based estimator recovers the true underlying sparse model and is asymptotically normally distributed. We also study its asymptotic properties in a double-asymptotic framework, where the number of parameters diverges with the sample size. We show by simulations and on real data that the adaptive SGL outperforms other oracle-like methods in terms of estimation precision and variable selection.

     

Estimation of some functional of the population distribution based on a stratified random sample

Summary In the preceding papers ([7], [8] and [9]), one of the authors discussed about the estimation of variances, covariances and correlation coefficients of the population based on a stratified random sample. In this paper we consider more general problem; estimating some functional θ(F) of the population distributionF based on a stratified random sample, which include our previous papers as special cases. We propose an unbiased estimator of θ(F) based on a stratified random sample and give an asymptotic expression of the gain in precision due to stratification in the case of proportional allocation. Furthermore, we present the general form of the optimum stratification in the proportional allocation for the estimation of θ(F).     

Mode estimation for discrete distributions

The problem of estimating the mode of a discrete distribution is considered. New characterizations of discrete unimodal and multi-modal distributions are obtained. The proposed mode estimator is essentially the sample mode, modulo appropriate modifications when the sample mode is not well defined. In the case of i.i.d. observations coming from a unimodal discrete distribution, our proposed mode estimator is shown to possess a number of strong asymptotic properties. Many of these results extend to the case of multi-modal discrete distributions as well. Our method also applies — and we have similar asymptotic results — to the problem of mode estimation based on finitely many observations on a Markov chain whose equilibrium distribution is the underlying unimodal distribution. For unimodal discrete distributions, we also propose a consistent large sample test of mode based on the proposed statistic. Applications of mode estimation problem in Monte-Carlo optimization problem using the Hastings Metropolis chain and in prediction problem using binary response variable, specially in the context of dose-response experiments, are also illustrated.     

Reducing asymptotic bias of weak instrumental estimation using independently repeated cross-sectional information

In this paper, we consider the instrumental variable estimation (the two-stage least squares estimator and the limited information maximum likelihood estimator) using weak instruments in a repeated measurements or a panel data model. We show that independently repeated cross-sectional data can reduce the asymptotic bias of the instrumental variable estimation when instruments are weakly correlated with endogenous variables. When the number of repeated measurements tends to infinity, we can achieve consistent instrumental variable estimation with weak instruments.     

Stability and Spectral Properties of General Tree-Shaped Wave Networks with Variable Coefficients

The stability of general tree-shaped wave networks with variable coefficients under boundary feedback controls is considered. Making full use of the tree-shaped structures, we present a detailed asymptotic spectral analysis of the networks. By proposing the from-root-to-leaf calculating technique, we deduce an explicit recursive expression for the asymptotic characteristic equation and the spectral properties are further obtained. We show that the spectrum-determined-growth (SDG) condition holds. Thus the stability analysis of the closed-loop system can be completely converted to the infimum estimation of the asymptotic characteristic equation. Especially, we further show that the infimum is positive so as to obtain the exponential stability by estimating the recursive expression in from-leaf-to-root order. Some numerical simulations are presented to illustrate and support the theoretical results.

     

纵向数据下广义估计方程估计

广义估计方程方法是一种最一般的参数估计方法,广泛地应用于生物统计、经济计量、医疗保险等领域.在纵向数据下,由于组间数据是相关的,为了提高估计的效率,广义估计方程方法一般需要考虑个体组内相关性.因此,大多数文献对个体组内的协方差矩阵进行参数假设,但假设的合理性及协方差矩阵估计的好坏对参数估计效率产生很大影响,同时参数假设也可能导致模型误判.针对纵向数据下广义估计方程,本文提出了改进的GMM方法和经验似然方法,并对给出的估计量建立了大样本性质.其中分块的思想,避免了对个体组内相关性结构进行假设,从这种意义上说,这种方法具有一定的稳健性.我们还通过两个模拟的例子,考察了文中提出估计量的有限样本性质.     

Semiparametric estimation in regression with missing covariates using single-index models

We investigate semiparametric estimation of regression coefficients through generalized estimating equations with single-index models when some covariates are missing at random. Existing popular semiparametric estimators may run into difficulties when some selection probabilities are small or the dimension of the covariates is not low. We propose a new simple parameter estimator using a kernel-assisted estimator for the augmentation by a single-index model without using the inverse of selection probabilities. We show that under certain conditions the proposed estimator is as efficient as the existing methods based on standard kernel smoothing, which are often practically infeasible in the case of multiple covariates. A simulation study and a real data example are presented to illustrate the proposed method. The numerical results show that the proposed estimator avoids some numerical issues caused by estimated small selection probabilities that are needed in other estimators.

     

Delete-group Jackknife Estimate in Partially Linear Regression Models with Heteroscedasticity

Abstract Consider a partially linear regression model with an unknown vector parameter β,an unknownfunction g(.),and unknown heteroscedastic error variances.Chen,You proposed a semiparametric generalizedleast squares estimator(SGLSE)for β,which takes the heteroscedasticity into account to increase efficiency.Forinference based on this SGLSE,it is necessary to construct a consistent estimator for its asymptotic covariancematrix.However,when there exists within-group correlation, the traditional delta method and the delete-1jackknife estimation fail to offer such a consistent estimator.In this paper, by deleting grouped partial residualsa delete-group jackknife method is examined.It is shown that the delete-group jackknife method indeed canprovide a consistent estimator for the asymptotic covariance matrix in the presence of within-group correlations.This result is an extension of that in[21].     

One-step estimation of spatial dependence parameters: Properties and extensions of the APLE statistic

We consider one-step estimation of parameters that represent the strength of spatial dependence in a geostatistical or lattice spatial model. While the maximum likelihood estimators (MLE) of spatial dependence parameters are known to have various desirable properties, they do not have closed-form expressions. Therefore, we consider a one-step alternative to maximum likelihood estimation based on solving an approximate (i.e., one-step) profile likelihood estimating equation. The resulting approximate profile likelihood estimator (APLE) has a closed-form representation, making it a suitable alternative to the widely used Moran’s I statistic. Since the finite-sample and asymptotic properties of one-step estimators of covariance-function parameters have not been studied rigorously, we explore these properties for the APLE of the spatial dependence parameter in the simultaneous autoregressive (SAR) model. Motivated by the APLE statistic’s closed from, we develop exploratory spatial data analysis tools that capture regions of local clustering or the extent to which the strength of spatial dependence varies across space. We illustrate these exploratory tools using both simulated data and observed crime rates in Columbus, OH.     

Quasi-likelihood analysis for the stochastic differential equation with jumps

In this paper, we consider a multidimensional diffusion process X with jumps whose jump term is driven by a compound Poisson process, and discuss its parametric estimation. We present asymptotic normality and convergence of moments of any order for a quasi-maximum likelihood estimator and a Bayes type estimator by assuming an exponential mixing property of X. To show these properties, we use the polynomial type large deviation theory.     

Asymptotic efficiency of ridge estimator in linear and semiparametric linear models

The linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this article, we consider the commonly used ridge estimator in linear models. We propose analyzing the ridge estimator for a finite sample size n and a growing dimension p. The existence and asymptotic normality of the ridge estimator are established under some regularity conditions when p→∞. It also occurs that a strictly linear model is inadequate when some of the relations are believed to be of certain linear form while others are not easily parameterized, and thus a semiparametric partial linear model is considered. For these semiparametric partial linear models with p>n, we develop a procedure to estimate the linear coefficients as if the nonparametric part is not present. The asymptotic efficiency of the proposed estimator for the linear component is studied for p→∞. It is shown that the proposed estimator of the linear component asymptotically performs very well.     

Testing for serial correlation of unknown form in cointegrated time series models

Portmanteau test statistics are useful for checking the adequacy of many time series models. Here we generalized the omnibus procedure proposed by Duchesne and Roy (2004,Journal of Multivariate Analysis,89, 148–180) for multivariate stationary autoregressive models with exogenous variables (VARX) to the case of cointegrated (or partially nonstationary) VARX models. We show that for cointegrated VARX time series, the test statistic obtained by comparing the spectral density of the errors under the null hypothesis of non-correlation with a kernel-based spectral density estimator, is asymptotically standard normal. The parameters of the model can be estimated by conditional maximum likelihood or by asymptotically equivalent estimation procedures. The procedure relies on a truncation point or a smoothing parameter. We state conditions under which the asymptotic distribution of the test statistic is unaffected by a data-dependent method. The finite sample properties of the test statistics are studied via a small simulation study.