Empirical likelihood for LAD estimators in infinite variance ARMA models

In this paper, we use an empirical likelihood method to construct confidence regions for the stationary ARMA(p,q) models with infinite variance. An empirical log-likelihood ratio is derived by the estimating equation of the self-weighted LAD estimator. It is proved that the proposed statistic has an asymptotic standard chi-squared distribution. Simulation studies show that in a small sample case, the performance of empirical likelihood method is better than that of normal approximation of the LAD estimator in terms of the coverage accuracy.     

Empirical likelihood for the smoothed LAD estimator in infinite variance autoregressive models

This paper proposes an empirical likelihood method to estimate the parameters of infinite variance autoregressive (IVAR) models and to construct confidence regions for the parameters. Simulation studies suggest that in small sample case, the empirical likelihood confidence regions may be more accurate than the confidence regions constructed by the normal approximation based on the self-weighted LAD estimator proposed by Ling (2005).     

Minimax Hierarchical Empirical Bayes Estimation in Multivariate Regression

The multivariate normal regression model, in which a vector y of responses is to be predicted by a vector x of explanatory variables, is considered. A hierarchical framework is used to express prior information on both x and y. An empirical Bayes estimator is developed which shrinks the maximum likelihood estimator of the matrix of regression coefficients across rows and columns to nontrivial subspaces which reflect both types of prior information. The estimator is shown to be minimax and is applied to a set of chemometrics data for which it reduces the cross-validated predicted mean squared error of the maximum likelihood estimator by 38%.     

Full likelihood inferences in the Cox model: an empirical likelihood approach

For the regression parameter β ₀ in the Cox model, there have been several estimators constructed based on various types of approximated likelihood, but none of them has demonstrated small-sample advantage over Cox’s partial likelihood estimator. In this article, we derive the full likelihood function for (β ₀, F ₀), where F ₀ is the baseline distribution in the Cox model. Using the empirical likelihood parameterization, we explicitly profile out nuisance parameter F ₀ to obtain the full-profile likelihood function for β ₀ and the maximum likelihood estimator (MLE) for (β ₀, F ₀). The relation between the MLE and Cox’s partial likelihood estimator for β ₀ is made clear by showing that Taylor’s expansion gives Cox’s partial likelihood estimating function as the leading term of the full-profile likelihood estimating function. We show that the log full-likelihood ratio has an asymptotic chi-squared distribution, while the simulation studies indicate that for small or moderate sample sizes, the MLE performs favorably over Cox’s partial likelihood estimator. In a real dataset example, our full likelihood ratio test and Cox’s partial likelihood ratio test lead to statistically different conclusions.     

Local Whittle likelihood estimators and tests for non-Gaussian stationary processes

In this paper, we propose a local Whittle likelihood estimator for spectral densities of non-Gaussian processes and a local Whittle likelihood ratio test statistic for the problem of testing whether the spectral density of a non-Gaussian stationary process belongs to a parametric family or not. Introducing a local Whittle likelihood of a spectral density f _θ (λ) around λ, we propose a local estimator [^(q)] = [^(q)] (l){\hat{\theta } = \hat{\theta } (\lambda ) } of θ which maximizes the local Whittle likelihood around λ, and use f_{[^(q)] (l)} (l){f_{\hat{\theta } (\lambda )} (\lambda )} as an estimator of the true spectral density. For the testing problem, we use a local Whittle likelihood ratio test statistic based on the local Whittle likelihood estimator. The asymptotics of these statistics are elucidated. It is shown that their asymptotic distributions do not depend on non-Gaussianity of the processes. Because our models include nonlinear stationary time series models, we can apply the results to stationary GARCH processes. Advantage of the proposed estimator is demonstrated by a few simulated numerical examples.     

On the minimal penalty for Markov order estimation

We show that large-scale typicality of Markov sample paths implies that the likelihood ratio statistic satisfies a law of iterated logarithm uniformly to the same scale. As a consequence, the penalized likelihood Markov order estimator is strongly consistent for penalties growing as slowly as log log n when an upper bound is imposed on the order which may grow as rapidly as log n. Our method of proof, using techniques from empirical process theory, does not rely on the explicit expression for the maximum likelihood estimator in the Markov case and could therefore be applicable in other settings.     

Maximum Kernel Likelihood Estimation

We introduce an estimator for the population mean based on maximizing likelihoods formed by parameterizing a kernel density estimate. Due to these origins, we have dubbed the estimator the maximum kernel likelihood estimate (MKLE). A speedy computational method to compute the MKLE based on binning is implemented in a simulation study which shows that the MKLE at an optimal bandwidth is decidedly superior in terms of efficiency to the sample mean and other measures of location for heavy tailed symmetric distributions. An empirical rule and a computational method to estimate this optimal bandwidth are developed and used to construct bootstrap confidence intervals for the population mean. We show that the intervals have approximately nominal coverage and have significantly smaller average width than the standard t and z intervals. Finally, we develop some mathematical properties for a very close approximation to the MKLE called the kernel mean. In particular, we demonstrate that the kernel mean is indeed unbiased for the population mean for symmetric distributions.     

Estimation of a Normal Covariance Matrix with Incomplete Data under Stein′s Loss

Suppose that we have (n − a) independent observations from N_p(0, Σ) and that, in addition, we have a independent observations available on the last (p − c) coordinates. Assuming that both observations are independent, we consider the problem of estimating Σ under the Stein′s loss function, and show that some estimators invariant under the permutation of the last (p − c) coordinates as well as under those of the first c coordinates are better than the minimax estimators of Eaten. The estimators considered outperform the maximum likelihood estimator (MLE) under the Stein′s loss function as well. The method involved here is computation of an unbiased estimate of the risk of an invariant estimator considered in this article. In addition we discuss its application to the problem of estimating a covariance matrix in a GMANOVA model since the estimation problem of the covariance matrix with extra data can be regarded as its canonical form.     

Empirical likelihood inference for estimating equation with missing data

In this article, empirical likelihood inference for estimating equation with missing data is considered. Based on the weighted-corrected estimating function, an empirical log-likelihood ratio is proved to be a standard chi-square distribution asymptotically under some suitable conditions. This result is different from those derived before. So it is convenient to construct confidence regions for the parameters of interest. We also prove that our proposed maximum empirical likelihood estimator θ is asymptotically normal and attains the semiparametric efficiency bound of missing data. Some simulations indicate that the proposed method performs the best.     

Nonparametric Estimation of Regression Functions in Point Process Models

We prove that the empirical L ²-risk minimizing estimator over some general type of sieve classes is universally, strongly consistent for the regression function in a class of point process models of Poissonian type (random sampling processes). The universal consistency result needs weak assumptions on the underlying distributions and regression functions. It applies in particular to neural net classes and to radial basis function nets. For the estimation of the intensity functions of a Poisson process a similar technique yields consistency of the sieved maximum likelihood estimator for some general sieve classes. This revised version was published online in August 2006 with corrections to the Cover Date.     

A penalty method for nonparametric estimation of the logarithmic derivative of a density function

Summary Given a random sample of sizen from a densityf ₀ on the real line satisfying certain regularity conditions, we propose a nonparametric estimator forψ ₀=−f ₀ ^′ /f⁰. The estimate is the minimizer of a quadratic functional of the formλJ(ψ)+∫[ψ ²−2ψ′]dF_n where λ>0 is a smoothing parameter,J(·) is a roughness penalty, andF _n is the empirical c.d.f. of the sample. A characterization of the estimate (useful for computational purposes) is given which is related to spline functions. A more complete study of the caseJ(ψ)=∫[d ²ψ/dx²]² is given, since it has the desirable property of giving the maximum likelihood normal estimate in the infinite smoothness limit (λ→∞). Asymptotics under somewhat restrictive assumptions (periodicity) indicate that the estimator is asymptotically consistent and achieves the optimal rate of convergence. This type of estimator looks promising because the minimization problem is simple in comparison with the analogous penalized likelihood estimators. This research was supported by the Office of Naval Research under Grant Number N00014-82-C-0062.     

Stein shrinkage and second-order efficiency for semiparametric estimation of the shift

The problem of estimating the shift (or, equivalently, the center of symmetry) of an unknown symmetric and periodic function f observed in Gaussian white noise is considered. Using the blockwise Stein method, a penalized profile likelihood with a data-driven penalization is introduced so that the estimator of the center of symmetry is defined as the maximizer of the penalized profile likelihood. This estimator has the advantage of being independent of the functional class to which the signal f is assumed to belong and, furthermore, is shown to be semiparametrically adaptive and efficient. Moreover, the second-order term of the risk expansion of the proposed estimator is proved to behave at least as well as the second-order term of the risk of the best possible estimator using monotone smoothing filter. Under mild assumptions, this estimator is shown to be second-order minimax sharp adaptive over the whole scale of Sobolev balls with smoothness β > 1. Thus, these results extend those of [10], where second-order asymptotic minimaxity is proved for an estimator depending on the functional class containing f and β ≥ 2 is required.      

Vitesse de convergence presque sûre de l'estimateur à noyau du maximum de vraisemblance local

We consider the local maximum likelihood estimation of $\theta (x)$, unknown parameter of the conditional distribution of Y given $X=x$. The aim of this Note is the study of strong uniform consistency rates of the local maximum likelihood kernel estimator. Under suitable regularity conditions, we establish a uniform law of the logarithm for the maximal deviation of this estimator. The method of proof is based upon functional limit laws derived by modern empirical process theory. To cite this article: D. Blondin, C. R. Acad. Sci. Paris, Ser. I 342 (2006).     

Partial penalized empirical likelihood ratio test under sparse case

A consistent test via the partial penalized empirical likelihood approach for the parametric hypothesis testing under the sparse case, called the partial penalized empirical likelihood ratio (PPELR) test, is proposed in this paper. Our results are demonstrated for the mean vector in multivariate analysis and regression coefficients in linear models, respectively. And we establish its asymptotic distributions under the null hypothesis and the local alternatives of order n^?1/2 under regularity conditions. Meanwhile, the oracle property of the partial penalized empirical likelihood estimator also holds. The proposed PPELR test statistic performs as well as the ordinary empirical likelihood ratio test statistic and outperforms the full penalized empirical likelihood ratio test statistic in term of size and power when the null parameter is zero. Moreover, the proposed method obtains the variable selection as well as the p-values of testing. Numerical simulations and an analysis of Prostate Cancer data confirm our theoretical findings and demonstrate the promising performance of the proposed method in hypothesis testing and variable selection.     

Generalized time-dependent conditional linear models under left truncation and right censoring

Consider the model f(S(z|X)){\phi(S(z|X))} = \pmbb(z) [(X)\vec]{\pmb{\beta}(z) {\vec{X}}}, where f{\phi} is a known link function, S(·|X) is the survival function of a response Y given a covariate X, [(X)\vec]{\vec{X}} = (1, X, X ² , . . . , X ^p ) and \pmbb(z){\pmb{\beta}(z)} is an unknown vector of time-dependent regression coefficients. The response is subject to left truncation and right censoring. Under this model, which reduces for special choices of f{\phi} to e.g. Cox proportional hazards model or the additive hazards model with time dependent coefficients, we study the estimation of the vector \pmbb(z){\pmb{\beta}(z)} . A least squares approach is proposed and the asymptotic properties of the proposed estimator are established. The estimator is also compared with a competing maximum likelihood based estimator by means of simulations. Finally, the method is applied to a larynx cancer data set.     

Combining empirical likelihood and generalized method of moments estimators: Asymptotics and higher order bias

This paper proposes an estimator combining empirical likelihood (EL) and the generalized method of moments (GMM) by allowing the sample average moment vector to deviate from zero and the sample weights to deviate from n⁻¹. The new estimator may be adjusted through free parameter δ∈(0,1) with GMM behavior attained as δ?0 and EL as δ?1. When the sample size is small and the number of moment conditions is large, the parameter space under which the EL estimator is defined may be restricted at or near the population parameter value. The support of the parameter space for the new estimator may be adjusted through δ. The new estimator performs well in Monte Carlo simulations.     

Empirical likelihood for mixed-effects error-in-variables model

This paper mainly introduces the method of empirical likelihood and its applications on two different models. We discuss the empirical likelihood inference on fixed-effect parameter in mixed-effects model with error-in-variables. We first consider a linear mixed-effects model with measurement errors in both fixed and random effects. We construct the empirical likelihood confidence regions for the fixed-effects parameters and the mean parameters of random-effects. The limiting distribution of the empirical log likelihood ratio at the true parameter is X2p＋q, where p, q are dimension of fixed and random effects respectively. Then we discuss empirical likelihood inference in a semi-linear error-in-variable mixed-effects model. Under certain conditions, it is shown that the empirical log likelihood ratio at the true parameter also converges to X2p＋q. Simulations illustrate that the proposed confidence region has a coverage probability more closer to the nominal level than normal approximation based confidence region.     

Likelihood Based Confidence Intervals for the Tail Index

For the estimation of the tail index of a heavy tailed distribution, one of the well-known estimators is the Hill estimator (Hill, 1975). One obvious way to construct a confidence interval for the tail index is via the normal approximation of the Hill estimator. In this paper we apply both the empirical likelihood method and the parametric likelihood method to obtaining confidence intervals for the tail index. Our limited simulation study indicates that the normal approximation method is worse than the other two methods in terms of coverage probability, and the empirical likelihood method and the parametric likelihood method are comparable.     

Estimators for the long-memory parameter in LARCH models, and fractional Brownian motion

This paper investigates several strategies for consistently estimating the so-called Hurst parameter H responsible for the long-memory correlations in a linear class of ARCH time series, known as LARCH(∞) models, as well as in the continuous-time Gaussian stochastic process known as fractional Brownian motion (fBm). A LARCH model’s parameter is estimated using a conditional maximum likelihood method, which is proved to have good stability properties. A local Whittle estimator is also discussed. The article further proposes a specially designed conditional maximum likelihood method for estimating the H which is closer in spirit to one based on discrete observations of fBm. In keeping with the popular financial interpretation of ARCH models, all estimators are based only on observation of the “returns” of the model, not on their “volatilities”.     

Penalized empirical likelihood estimation of semiparametric models

We propose an empirical likelihood-based estimation method for conditional estimating equations containing unknown functions, which can be applied for various semiparametric models. The proposed method is based on the methods of conditional empirical likelihood and penalization. Thus, our estimator is called the penalized empirical likelihood (PEL) estimator. For the whole parameter including infinite-dimensional unknown functions, we derive the consistency and a convergence rate of the PEL estimator. Furthermore, for the finite-dimensional parametric component, we show the asymptotic normality and efficiency of the PEL estimator. We illustrate the theory by three examples. Simulation results show reasonable finite sample properties of our estimator.