Asymptotic efficiency of ridge estimator in linear and semiparametric linear models

The linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this article, we consider the commonly used ridge estimator in linear models. We propose analyzing the ridge estimator for a finite sample size n and a growing dimension p. The existence and asymptotic normality of the ridge estimator are established under some regularity conditions when p→∞. It also occurs that a strictly linear model is inadequate when some of the relations are believed to be of certain linear form while others are not easily parameterized, and thus a semiparametric partial linear model is considered. For these semiparametric partial linear models with p>n, we develop a procedure to estimate the linear coefficients as if the nonparametric part is not present. The asymptotic efficiency of the proposed estimator for the linear component is studied for p→∞. It is shown that the proposed estimator of the linear component asymptotically performs very well.     

Inverse portfolio problem with mean-deviation model

A Markowitz-type portfolio selection problem is to minimize a deviation measure of portfolio rate of return subject to constraints on portfolio budget and on desired expected return. In this context, the inverse portfolio problem is finding a deviation measure by observing the optimal mean-deviation portfolio that an investor holds. Necessary and sufficient conditions for the existence of such a deviation measure are established. It is shown that if the deviation measure exists, it can be chosen in the form of a mixed CVaR-deviation, and in the case of n risky assets available for investment (to form a portfolio), it is determined by a combination of (n + 1) CVaR-deviations. In the later case, an algorithm for constructing the deviation measure is presented, and if the number of CVaR-deviations is constrained, an approximate mixed CVaR-deviation is offered as well. The solution of the inverse portfolio problem may not be unique, and the investor can opt for the most conservative one, which has a simple closed-form representation.     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

Simultaneous estimation of Poisson means

In the simultaneous estimation of means from independent Poisson distributions, an estimator is developed which incorporates a prior mean and variance for each Poisson mean estimated. This estimator possesses substantially smaller risk than the usual estimator in a region of the parameter space and seems superior to other estimators proposed to estimate p Poisson means. It is indicated through two asymptotic results that, unlike the conjugate Bayes estimator, the risk of the estimator does not greatly exceed the risk of the usual estimator outside of the region of risk improvement.     

Minmax regret location–allocation problem on a network under uncertainty

We consider a robust location–allocation problem with uncertainty in demand coefficients. Specifically, for each demand point, only an interval estimate of its demand is known and we consider the problem of determining where to locate a new service when a given fraction of these demand points must be served by the utility. The optimal solution of this problem is determined by the “minimax regret” location, i.e., the point that minimizes the worst-case loss in the objective function that may occur because a decision is made without knowing which state of nature will take place. For the case where the demand points are vertices of a network we show that the robust location–allocation problem can be solved in O(min{p, n − p}n³m) time, where n is the number of demand points, p (p < n) is the fixed number of demand points that must be served by the new service and m is the number of edges of the network.     

Optimal variance estimation based on lagged second-order difference in nonparametric regression

Differenced estimators of variance bypass the estimation of regression function and thus are simple to calculate. However, there exist two problems: most differenced estimators do not achieve the asymptotic optimal rate for the mean square error; for finite samples the estimation bias is also important and not further considered. In this paper, we estimate the variance as the intercept in a linear regression with the lagged Gasser-type variance estimator as dependent variable. For the equidistant design, our estimator is not only \(n^{1/2}\)-consistent and asymptotically normal, but also achieves the optimal bound in terms of estimation variance with less asymptotic bias. Simulation studies show that our estimator has less mean square error than some existing differenced estimators, especially in the cases of immense oscillation of regression function and small-sized sample.     

Empirical Bayes estimation in a multiple linear regression model

Summary Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n ⁻¹) are constructed. Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.     

A penalty method for nonparametric estimation of the logarithmic derivative of a density function

Summary Given a random sample of sizen from a densityf ₀ on the real line satisfying certain regularity conditions, we propose a nonparametric estimator forψ ₀=−f ₀ ^′ /f⁰. The estimate is the minimizer of a quadratic functional of the formλJ(ψ)+∫[ψ ²−2ψ′]dF_n where λ>0 is a smoothing parameter,J(·) is a roughness penalty, andF _n is the empirical c.d.f. of the sample. A characterization of the estimate (useful for computational purposes) is given which is related to spline functions. A more complete study of the caseJ(ψ)=∫[d ²ψ/dx²]² is given, since it has the desirable property of giving the maximum likelihood normal estimate in the infinite smoothness limit (λ→∞). Asymptotics under somewhat restrictive assumptions (periodicity) indicate that the estimator is asymptotically consistent and achieves the optimal rate of convergence. This type of estimator looks promising because the minimization problem is simple in comparison with the analogous penalized likelihood estimators. This research was supported by the Office of Naval Research under Grant Number N00014-82-C-0062.     

An Isotonic Regression Problem for Infinite-Dimensional Parameters

We consider an infinite-dimensional isotonic regression problem which is an extension of the suitably revised classical isotonic regression problem. Given p-summable data, for p finite and at least one, there exists an optimal estimator to our problem. For p greater than one, this estimator is unique and is the limit in the p-norm of the sequence of unique estimators in canonical finite-dimensional truncations of our problem. However, for p equal to one, our problem, as well as the finite-dimensional truncations, admit multiple optimal estimators in general. In this case, the sequence of optimal estimator sets to the truncations converges to the optimal estimator set of the infinite problem in the sense of Kuratowski. Moreover, the selection of natural best optimal estimators to the truncations converges in the 1-norm to an optimal estimator of the infinite problem.     

Superefficient estimation of the marginals by exploiting knowledge on the copula

We consider the problem of estimating the marginals in the case where there is knowledge on the copula. If the copula is smooth, it is known that it is possible to improve on the empirical distribution functions: optimal estimators still have a rate of convergence n^−1/2, but a smaller asymptotic variance. In this paper we show that for non-smooth copulas it is sometimes possible to construct superefficient estimators of the marginals: we construct both a copula and, exploiting the information our copula provides, estimators of the marginals with the rate of convergence logn/n.     

Admissible and minimax multiparameter estimation in exponential families

Consider p independent distributions each belonging to the one parameter exponential family with distribution functions absolutely continuous with respect to Lebesgue measure. For estimating the natural parameter vector with p ≥ p₀ (p₀ is typically 2 or 3), a general class of estimators dominating the minimum variance unbiased estimator (MVUE) or an estimator which is a known constant multiple of the MVUE is produced under different weighted squared error losses. Included as special cases are some results of Hudson [13] and Berger [5]. Also, for a subfamily of the general exponential family, a class of estimators dominating the MVUE of the mean vector or an estimator which is a known constant multiple of the MVUE is produced. The major tool is to obtain a general solution to a basic differential inequality.     

On the estimation of a restricted normal mean

Let X ∼ N(θ,1), where θ ϵ [−m, m], for some m > 0, and consider the problem of estimating θ with quadratic loss. We show that the Bayes estimator δ_m, corresponding to the uniform prior on [−m, m], dominates δ⁰ (x) = x on [−m, m] and it also dominates the MLE over a large part of the parameter interval. We further offer numerical evidence to suggest that δ_m has quite satisfactory risk performance when compared with the minimax estimators proposed by Casella and Strawderman (1981) and the estimators proposed by Bickel (1981).     

Error analysis of large-time approximations for decay chains

Consider a radioactive decay chain X₁ → ? → X_n→ and let N_n(t) be the amount of X_n at time t. This paper establishes error bounds for large-time approximations to N_n(t) that include and generalize the transient equilibrium approximations and other known approximations. The error bounds allow one to find the range of t for which these approximations can be used with a given degree of precision.     

Semi-parametric estimation of partially linear single-index models

One of the most difficult problems in applications of semi-parametric partially linear single-index models (PLSIM) is the choice of pilot estimators and complexity parameters which may result in radically different estimators. Pilot estimators are often assumed to be root-n consistent, although they are not given in a constructible way. Complexity parameters, such as a smoothing bandwidth are constrained to a certain speed, which is rarely determinable in practical situations.In this paper, efficient, constructible and practicable estimators of PLSIMs are designed with applications to time series. The proposed technique answers two questions from Carroll et al. [Generalized partially linear single-index models, J. Amer. Statist. Assoc. 92 (1997) 477-489]: no root-n pilot estimator for the single-index part of the model is needed and complexity parameters can be selected at the optimal smoothing rate. The asymptotic distribution is derived and the corresponding algorithm is easily implemented. Examples from real data sets (credit-scoring and environmental statistics) illustrate the technique and the proposed methodology of minimum average variance estimation (MAVE).     

Difference schemes of exponential type for singularly perturbed Volterra integro-differential problems

Extended one-step schemes of exponential type are introduced for solving singularly perturbed Volterra integro-differential problems. These schemes are of order (m + 1), m = 0, 1, 2, …, when the perturbation parameter, ε, is fixed. These schemes have the property that if ε is of order h they reduced to first order of accuracy and optimal when ε → 0. Stability analysis of these schemes are presented. Numerical results and comparisons with other schemes are presented.     

Smooth estimation of survival and density functions for a stationary associated process using Poisson weights

Let {X_n,n≥1} be a sequence of stationary non-negative associated random variables with common marginal density f(x). Here we use the empirical survival function as studied in Bagai and Prakasa Rao (1991) and apply the smoothing technique proposed by Gawronski (1980) (see also Chaubey and Sen, 1996) in proposing a smooth estimator of the density function f and that of the corresponding survival function. Some asymptotic properties of the resulting estimators, similar to those obtained in Chaubey and Sen (1996) for the i.i.d. case, are derived. A simulation study has been carried out to compare the new estimator to the kernel estimator of a density function given in Bagai and Prakasa Rao (1996) and the estimator in Buch-Larsen et al. (2005).     

Trimmed estimates in simultaneous estimation of parameters in exponential families

Let X₁,…, X_p be p (≥ 3) independent random variables, where each X_i has a distribution belonging to the one-parameter exponential family of distributions. The problem is to estimate the unknown parameters simultaneously in the presence of extreme observations. C. Stein (Ann. Statist.9 (1981), 1135–1151) proposed a method of estimating the mean vector of a multinormal distribution, based on order statistics corresponding to the |X_i|'s, which permitted improvement over the usual maximum likelihood estimator, for long-tailed empirical distribution functions. In this paper, the ideas of Stein are extended to the general discrete and absolutely continuous exponential families of distributions. Adaptive versions of the estimators are also discussed.     

Local generalized empirical estimation of regression

Let f(x) be the density of a design variable X and m(x) = E[Y\X = x] the regression function. Then m(x) - G(x)/f(x), where G(x) = m(x)f(x). The Dirac δ-function is used to define a generalized empirical function Gn (x) for G(x) whose expectation equals G(x). This generalized empirical function exists only in the space of Schwartz distributions, so we introduce a local polynomial of order p approximation to Gn(.) which provides estimators of the function G(x) and its derivatives. The density f(x) can be estimated in a similar manner. The resulting local generalized empirical estimator (LGE) of m(x) is exactly the Nadaraya-Watson estimator at interior points when p = 1, but on the boundary the estimator automatically corrects the boundary effect. Asymptotic normality of the estimator is established. Asymptotic expressions for the mean squared errors are obtained and used in bandwidth selection. Boundary behavior of the estimators is investigated in details. We use Monte Carlo simulations to show that the     

Explicit Nordsieck methods with extended stability regions

We describe the construction of explicit Nordsieck methods of order p and stage order q = p with large regions of absolute stability. We also discuss error propagation and estimation of local discretization errors. The error estimators are derived for examples of general linear methods constructed in this paper. Some numerical experiments are presented which illustrate the effectiveness of proposed methods.