Maximum likelihood estimation of a change-point in the distribution of independent random variables: General multiparameter case

In a sequence ofn independent random variables the pdf changes fromf(x, 0) tof(x, 0 + δv_n⁻¹) after the firstnλ variables. The problem is to estimateλ (0, 1 ), where 0 and δ are unknownd-dim parameters andv_n → ∞ slower thann^1/2. Let_n denote the maximum likelihood estimator (mle) ofλ. Analyzing the local behavior of the likelihood function near the true parameter values it is shown under regularity conditions that ifn_n²(− λ) is bounded in probability asn → ∞, then it converges in law to the timeT_(δjδ)^1/2 at which a two-sided Brownian motion (B.M.) with drift1/2(δ′Jδ)^1/2ton(−∞, ∞) attains its a.s. unique minimum, whereJ denotes the Fisher-information matrix. This generalizes the result for small change in mean of univariate normal random variables obtained by Bhattacharya and Brockwell (1976,Z. Warsch. Verw. Gebiete37, 51–75) who also derived the distribution ofT_μ forμ > 0. For the general case an alternative estimator is constructed by a three-step procedure which is shown to have the above asymptotic distribution. In the important case of multiparameter exponential families, the construction of this estimator is considerably simplified.     

Discrete time parametric models with long memory and infinite variance

We study a large class of infinite variance time series that display long memory. They can be represented as linear processes (infinite order moving averages) with coefficients that decay slowly to zero and with innovations that are in the domain of attraction of a stable distribution with index 1 < α < 2 (stable fractional ARIMA is a particular example). Assume that the coefficients of the linear process depend on an unknown parameter vector β which is to be estimated from a series of length n. We show that a Whittle-type estimator β_n for β is consistent (β_n converges to the true value β₀ in probability as n → ∞), and, under some additional conditions, we characterize the limiting distribution of the rescaled differences (n/logn)^1/ga(β_n − β₀).     

Likelihood-Based Local Polynomial Fitting for Single-Index Models

The parametric generalized linear model assumes that the conditional distribution of a response Y given a d-dimensional covariate X belongs to an exponential family and that a known transformation of the regression function is linear in X. In this paper we relax the latter assumption by considering a nonparametric function of the linear combination β^TX, say η₀(β^TX). To estimate the coefficient vector β and the nonparametric component η₀ we consider local polynomial fits based on kernel weighted conditional likelihoods. We then obtain an estimator of the regression function by simply replacing β and η₀ in η₀(β^TX) by these estimators. We derive the asymptotic distributions of these estimators and give the results of some numerical experiments.     

r-k Class estimation in regression model with concomitant variables

We treat with the r-k class estimation in a regression model, which includes the ordinary least squares estimator, the ordinary ridge regression estimator and the principal component regression estimator as special cases of the r-k class estimator. Many papers compared total mean square error of these estimators. Sarkar (1989, Ann. Inst. Statist. Math., 41, 717–724) asserts that the results of this comparison are still valid in a misspecified linear model. We point out some confusions of Sarkar and show additional conditions under which his assertion holds.     

Adaptive Semiparametric Estimation of the Memory Parameter

In Giraitis, Robinson, and Samarov (1997), we have shown that the optimal rate for memory parameter estimators in semiparametric long memory models with degree of “local smoothness” β is n^−r(β), r(β)=β/(2β+1), and that a log-periodogram regression estimator (a modified Geweke and Porter-Hudak (1983) estimator) with maximum frequency m=m(β)n^2r(β) is rate optimal. The question which we address in this paper is what is the best obtainable rate when β is unknown, so that estimators cannot depend on β. We obtain a lower bound for the asymptotic quadratic risk of any such adaptive estimator, which turns out to be larger than the optimal nonadaptive rate n^−r(β) by a logarithmic factor. We then consider a modified log-periodogram regression estimator based on tapered data and with a data-dependent maximum frequency m=m(β), which depends on an adaptively chosen estimator β of β, and show, using methods proposed by Lepskii (1990) in another context, that this estimator attains the lower bound up to a logarithmic factor. On one hand, this means that this estimator has nearly optimal rate among all adaptive (free from β) estimators, and, on the other hand, it shows near optimality of our data-dependent choice of the rate of the maximum frequency for the modified log-periodogram regression estimator. The proofs contain results which are also of independent interest: one result shows that data tapering gives a significant improvement in asymptotic properties of covariances of discrete Fourier transforms of long memory time series, while another gives an exponential inequality for the modified log-periodogram regression estimator.     

On uniform asymptotic normality of sequential least squares estimators for the parameters in a stable AR(p)

For a stable autoregressive process of order p with unknown vector parameter θ, it is shown that under a sequential sampling scheme with the stopping time defined by the trace of the observed Fisher information matrix, the least-squares estimator of θ is asymptotically normally distributed uniformly in θ belonging to any compact set in the parameter region.     

Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process

We consider an estimation problem with observations from a Gaussian process. The problem arises from a stochastic process modeling of computer experiments proposed recently by Sacks, Schiller, and Welch. By establishing various representations and approximations to the corresponding log-likelihood function, we show that the maximum likelihood estimator of the identifiable parameter θσ² is strongly consistent and converges weakly (when normalized by √n) to a normal random variable, whose variance does not depend on the selection of sample points. Some extensions to regression models are also obtained.     

On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

Iterative Estimation of the Extreme Value Index

Let {X_n, n ≥ 1} be a sequence of independent random variables with common continuous distribution function F having finite and unknown upper endpoint. A new iterative estimation procedure for the extreme value index γ is proposed and one implemented iterative estimator is investigated in detail, which is asymptotically as good as the uniform minimum varianced unbiased estimator in an ideal model. Moreover, the superiority of the iterative estimator over its non iterated counterpart in the non asymptotic case is shown in a simulation study.AMS 2000 Subject Classification: 62G32Supported by Swiss National Science foundation.     

Asymptotic expansions of the distributions of some test statistics for Gaussian ARMA processes

Let {X_t} be a Gaussian ARMA process with spectral density f_θ(λ), where θ is an unknown parameter. The problem considered is that of testing a simple hypothesis H:θ = θ₀ against the alternative A:θ ≠ θ₀. For this problem we propose a class of tests , which contains the likelihood ratio (LR), Wald (W), modified Wald (MW) and Rao (R) tests as special cases. Then we derive the χ² type asymptotic expansion of the distribution of T up to order n⁻¹, where n is the sample size. Also we derive the χ² type asymptotic expansion of the distribution of T under the sequence of alternatives A_n: θ = θ₀ + /√n, ε > 0. Then we compare the local powers of the LR, W, MW, and R tests on the basis of their asymptotic expansions.     

Minimax estimation of means of multivariate normal mixtures

Assume X = (X₁, …, X_p)′ is a normal mixture distribution with density w.r.t. Lebesgue measure, , where Σ is a known positive definite matrix and F is any known c.d.f. on (0, ∞). Estimation of the mean vector under an arbitrary known quadratic loss function _Q(θ, a) = (a − θ)′ Q(a − θ), Q a positive definite matrix, is considered. An unbiased estimator of risk is obatined for an arbitrary estimator, and a sufficient condition for estimators to be minimax is then achieved. The result is applied to modifying all the Stein estimators for the means of independent normal random variables to be minimax estimators for the problem considered here. In particular the results apply to the Stein class of limited translation estimators.     

Nonparametric Approach for Non-Gaussian Vector Stationary Processes

Suppose that {z(t)} is a non-Gaussian vector stationary process with spectral density matrixf(λ). In this paper we consider the testing problemH: ∫^π_−π K{f(λ)} dλ=cagainstA: ∫^π_−π K{f(λ)} dλ≠c, whereK{·} is an appropriate function andcis a given constant. For this problem we propose a testT_nbased on ∫^π_−π K{f(λ)} dλ=c, wheref(λ) is a nonparametric spectral estimator off(λ), and we define an efficacy ofT_nunder a sequence of nonparametric contiguous alternatives. The efficacy usually depnds on the fourth-order cumulant spectraf₄^Zofz(t). If it does not depend onf₄^Z, we say thatT_nis non-Gaussian robust. We will give sufficient conditions forT_nto be non-Gaussian robust. Since our test setting is very wide we can apply the result to many problems in time series. We discuss interrelation analysis of the components of {z(t)} and eigenvalue analysis off(λ). The essential point of our approach is that we do not assume the parametric form off(λ). Also some numerical studies are given and they confirm the theoretical results.     

Admissibility and complete class results for the multinomial estimation problem with entropy and squared error loss

Let X ≡ (X₁, …, X_t) have a multinomial distribution based on N trials with unknown vector of cell probabilities p ≡ (p₁, …, p_t). This paper derives admissibility and complete class results for the problem of simultaneously estimating p under entropy loss (EL) and squared error loss (SEL). Let and f(x¦p) denote the (t − 1)-dimensional simplex, the support of X and the probability mass function of X, respectively. First it is shown that δ is Bayes w.r.t. EL for prior P if and only if δ is Bayes w.r.t. SEL for P. The admissible rules under EL are proved to be Bayes, a result known for the case of SEL. Let Q denote the class of subsets of of the form T = _j=1^kF_j where k ≥ 1 and each F_j is a facet of which satisfies: F a facet of such that F naF_jF ncT. The minimal complete class of rules w.r.t. EL when N ≥ t − 1 is characterized as the class of Bayes rules with respect to priors P which satisfy P( ⁰) = 1, ξ(x) ≡ ∫ f(x¦p) P(dp) > 0 for all x in {x : sup ^₀ f(x¦p) > 0} for some ⁰ in Q containing all the vertices of . As an application, the maximum likelihood estimator is proved to be admissible w.r.t. EL when the estimation problem has parameter space Θ = but it is shown to be inadmissible for the problem with parameter space Θ = ( minus its vertices). This is a severe form of “tyranny of boundary.” Finally it is shown that when N ≥ t − 1 any estimator δ which satisfies δ(x) > 0 x is admissible under EL if and only if it is admissible under SEL. Examples are given of nonpositive estimators which are admissible under SEL but not under EL and vice versa.     

Shrinkage Estimators under Spherical Symmetry for the General Linear Model

This paper is primarily concerned with extending the results of Brandwein and Strawderman in the usual canonical setting of a general linear model when sampling from a spherically symmetric distribution. When the location parameter belongs to a proper linear subspace of the sampling space, we give an unbiased estimator of the difference of the risks between the least squares estimator φ₀ and a general shrinkage estimator φ = φ₀ − X − φ₀ ² · g φ₀. We obtain a general condition of domination for φ over φ₀ which is weaker than that of Brandwein and Strawderman. We do not need any superharmonicity condition on g. Our results are valid for general quadratic loss.     

A stochastic restricted ridge regression estimator

Groß [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64] proposed a restricted ridge regression estimator when exact restrictions are assumed to hold. When there are stochastic linear restrictions on the parameter vector, we introduce a new estimator by combining ideas underlying the mixed and the ridge regression estimators under the assumption that the errors are not independent and identically distributed. Apart from [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64], we call this new estimator as the stochastic restricted ridge regression (SRRR) estimator. The performance of the SRRR estimator over the mixed estimator in respect of the variance and the mean square error matrices is examined. We also illustrate our findings with a numerical example. The shrinkage generalized least squares (GLS) and the stochastic restricted shrinkage GLS estimators are proposed.     

Asymptotic Behaviour of Trajectory Fitting Estimators for Certain Non-ergodic SDE

Suppose one observes a path of a stochastic processX = (X_t)_t≥0 driven by the equation dX_t=θ a(X_t)dt + dW_t, t≥0, θ ≥ 0 with a(x) = x or a(x) = |x|^α for some α ∈ [0,1) and given initial condition X ₀. If the true but unknown parameter θ₀ is positive then X is non-ergodic. It is shown that in this situation a trajectory fitting estimator for θ₀ is strongly consistent and has the same limiting distribution as the maximum likelihood estimator, but converges of minor order. This revised version was published online in August 2006 with corrections to the Cover Date.     

Estimating Functions for Nonlinear Time Series Models

This paper discusses the problem of estimation for two classes of nonlinear models, namely random coefficient autoregressive (RCA) and autoregressive conditional heteroskedasticity (ARCH) models. For the RCA model, first assuming that the nuisance parameters are known we construct an estimator for parameters of interest based on Godambe's asymptotically optimal estimating function. Then, using the conditional least squares (CLS) estimator given by Tjøstheim (1986, Stochastic Process. Appl., 21, 251–273) and classical moment estimators for the nuisance parameters, we propose an estimated version of this estimator. These results are extended to the case of vector parameter. Next, we turn to discuss the problem of estimating the ARCH model with unknown parameter vector. We construct an estimator for parameters of interest based on Godambe's optimal estimator allowing that a part of the estimator depends on unknown parameters. Then, substituting the CLS estimators for the unknown parameters, the estimated version is proposed. Comparisons between the CLS and estimated optimal estimator of the RCA model and between the CLS and estimated version of the ARCH model are given via simulation studies.     

Model selection for regression on a fixed design

We deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points. For this end, we consider some collection of finite dimensional linear spaces (models) and the least-squares estimator built on a data driven selected model among this collection. This data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on Mallows' C _p . We provide non asymptotic risk bounds for the so-defined estimator from which we deduce adaptivity properties. Our results hold under mild moment conditions on the errors. The statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach. Received: 2 July 1997 / Revised version: 20 September 1999 / Published online: 6 July 2000     

On Distribution of Error of Linear Wavelet Estimator of Probability Density

We establish the asymptotic normality of the squared L ²-norm of the approximation error of a linear wavelet estimator of the density of a distribution. The calculations are based on the smallness of correlations between the coefficients of the high-frequency part of the multiresolution expansion of the estimator.Supported by the FCT Foundation (Portugal) in the framework of the project Probability and Statistics (2000–2002), Centro de Matematica, Universidade da Beira Interior.__________Translated from Lietuvos Matematikos Rinkinys, Vol. 45, No. 2, pp. 184–207, April–June, 2005.     

Admissibility of the natural estimator of the mean of a Gaussian process

Let {X(t): t [a, b]} be a Gaussian process with mean μ L₂[a, b] and continuous covariance K(s, t). When estimating μ under the loss ∫_a^b ( (t)−μ(t))² dt the natural estimator X is admissible if K is unknown. If K is known, X is minimax with risk ∫_a^b K(t, t) dt and admissible if and only if the three by three matrix whose entries are K(t_i, t_j) has a determinant which vanishes identically in t_i [a, b], i = 1, 2, 3.