On the role of the shrinkage parameter in local linear smoothing

Summary. It has been shown that local linear smoothing possesses a variety of very attractive properties, not least being its mean square performance. However, such results typically refer only to asymptotic mean squared error, meaning the mean squared error of the asymptotic distribution, and in fact, the actual mean squared error is often infinite. See Seifert and Gasser (1996). This difficulty may be overcome by shrinking the local linear estimator towards another estimator with bounded mean square. However, that approach requires information about the size of the shrinkage parameter. From at least a theoretical viewpoint, very little is known about the effects of shrinkage. In particular, it is not clear how small the shrinkage parameter may be chosen without affecting first-order properties, or whether infinitely supported kernels such as the Gaussian require shrinkage in order to achieve first-order optimal performance. In the present paper we provide concise and definitive answers to such questions, in the context of general ridged and shrunken local linear estimators. We produce necessary and sufficient conditions on the size of the shrinkage parameter that ensure the traditional mean squared error formula. We show that a wide variety of infinitely-supported kernels, with tails even lighter than those of the Gaussian kernel, do not require any shrinkage at all in order to achieve traditional first-order optimal mean square performance. Received: 22 May 1995 / In revised form: 23 January 1997     

利用对偶树复数小波与全变差模型实现图像去噪的新方法

本文首先研究了一种三层小波系数相关萎缩的概念与性质,利用对偶树复数小波与全变差模型相结合,提出了一种新的图像去噪方法。实验结果表明,与现有的图像去噪方法相比,本文方法无论是在视觉还是在均方误差等方面均有更好的效果。     

相对损失函数下的倍度保费

传统的倍度保费公式利用均方损失函数估计特定保人的风险. 然而, 索取保费与真实保费之间的比例比它们差的绝对值更适合于衡量保费的公平性. 基于这一点, 我们提出了两种计算保费的损失函数: 均方相对损失函数和熵相对损失函数, 并且给出了倍度因子的估计公式及它们的性质.     

Hazard function estimation with cause-of-death data missing at random

Hazard function estimation is an important part of survival analysis. Interest often centers on estimating the hazard function associated with a particular cause of death. We propose three nonparametric kernel estimators for the hazard function, all of which are appropriate when death times are subject to random censorship and censoring indicators can be missing at random. Specifically, we present a regression surrogate estimator, an imputation estimator, and an inverse probability weighted estimator. All three estimators are uniformly strongly consistent and asymptotically normal. We derive asymptotic representations of the mean squared error and the mean integrated squared error for these estimators and we discuss a data-driven bandwidth selection method. A simulation study, conducted to assess finite sample behavior, demonstrates that the proposed hazard estimators perform relatively well. We illustrate our methods with an analysis of some vascular disease data.     

An examiniation of criticisms of ridge regression simulation methodology

Page1 (1981) claimed that there is a serious design flaw in some of the recent simulation studies of ridge estimators, in particular those in Hoerl, Kennard and Baldwin (1976) and Lawless and Wang (1976). Farebrother (1983) argued that the major criticism in Page1 (1981) is unsubstantiated. In this paper we obtain a series expansion for the mean squared error of the ordinary ridge estimator, use it to prove that Pagel's claim is incorrect and reinforce Farebrother's comments.     

Targeted smoothing parameter selection for estimating average causal effects

The non-parametric estimation of average causal effects in observational studies often relies on controlling for confounding covariates through smoothing regression methods such as kernel, splines or local polynomial regression. Such regression methods are tuned via smoothing parameters which regulates the amount of degrees of freedom used in the fit. In this paper we propose data-driven methods for selecting smoothing parameters when the targeted parameter is an average causal effect. For this purpose, we propose to estimate the exact expression of the mean squared error of the estimators. Asymptotic approximations indicate that the smoothing parameters minimizing this mean squared error converges to zero faster than the optimal smoothing parameter for the estimation of the regression functions. In a simulation study we show that the proposed data-driven methods for selecting the smoothing parameters yield lower empirical mean squared error than other methods available such as, e.g., cross-validation.     

IMPROVED ESTIMATES OF THE COVARIANCE MATRIX IN GENERAL LINEAR MIXED MODELS

In this article, the problem of estimating the covariance matrix in general linear mixed models is considered. Two new classes of estimators obtained by shrinking the eigenvalues towards the origin and the arithmetic mean, respectively, are proposed. It is shown that these new estimators dominate the unbiased estimator under the squared error loss function. Finally, some simulation results to compare the performance of the proposed estimators with that of the unbiased estimator are reported. The simulation results indicate that these new shrinkage estimators provide a substantial improvement in risk under most situations.     

Improving Penalized Least Squares through Adaptive Selection of Penalty and Shrinkage

Estimation of the mean function in nonparametric regression is usefully separated into estimating the means at the observed factor levels—a one-way layout problem—and interpolation between the estimated means at adjacent factor levels. Candidate penalized least squares (PLS) estimators for the mean vector of a one-way layout are expressed as shrinkage estimators relative to an orthogonal regression basis determined by the penalty matrix. The shrinkage representation of PLS suggests a larger class of candidate monotone shrinkage (MS) estimators. Adaptive PLS and MS estimators choose the shrinkage vector and penalty matrix to minimize estimated risk. The actual risks of shrinkage-adaptive estimators depend strongly upon the economy of the penalty basis in representing the unknown mean vector. Local annihilators of polynomials, among them difference operators, generate penalty bases that are economical in a range of examples. Diagnostic techniques for adaptive PLS or MS estimators include basis-economy plots and estimates of loss or risk.     

Operational Variants of the Minimum Mean Squared Error Estimator in Linear Regression Models with Non-Spherical Disturbances

There is a good deal of literature that investigates the properties of various operational variants of Theil's (1971, Principles of Econometrics, Wiley, New York) minimum mean squared error estimator. It is interesting that virtually all of the existing analysis to date is based on the premise that the model's disturbances are i.i.d., an assumption which is not satisfied in many practical situations. In this paper, we consider a model with non-spherical errors and derive the asymptotic distribution, bias and mean squared error of a general class of feasible minimum mean squared error estimators. A Monte-Carlo experiment is conducted to examine the performance of this class of estimators in finite samples.     

Prediction with measurement errors in finite populations

We address the problem of selecting the best linear unbiased predictor (BLUP) of the latent value (e.g., serum glucose fasting level) of sample subjects with heteroskedastic measurement errors. Using a simple example, we compare the usual mixed model BLUP to a similar predictor based on a mixed model framed in a finite population (FPMM) setup with two sources of variability, the first of which corresponds to simple random sampling and the second, to heteroskedastic measurement errors. Under this last approach, we show that when measurement errors are subject-specific, the BLUP shrinkage constants are based on a pooled measurement error variance as opposed to the individual ones generally considered for the usual mixed model BLUP. In contrast, when the heteroskedastic measurement errors are measurement condition-specific, the FPMM BLUP involves different shrinkage constants. We also show that in this setup, when measurement errors are subject-specific, the usual mixed model predictor is biased but has a smaller mean squared error than the FPMM BLUP which points to some difficulties in the interpretation of such predictors.     

Integrated squared error of kernel-type estimator of distribution function

Let X ₁ ,...,X _n be a random sample drawn from distribution function F(x) with density function f(x) and suppose we want to estimate X(x). It is already shown that kernel estimator of F(x) is better than usual empirical distribution function in the sense of mean integrated squared error. In this paper we derive integrated squared error of kernel estimator and compare the error with that of the empirical distribution function. It is shown that the superiority of kernel estimators is not necessarily true in the sense of integrated squared error.     

Gradient Estimation Using Lagrange Interpolation Polynomials

We use Lagrange interpolation polynomials to obtain good gradient estimations. This is e.g. important for nonlinear programming solvers. As an error criterion, we take the mean squared error, which can be split up into a deterministic error and a stochastic error. We analyze these errors using N-times replicated Lagrange interpolation polynomials. We show that the mean squared error is of order if we replicate the Lagrange estimation procedure N times and use 2d evaluations in each replicate. As a result, the order of the mean squared error converges to N ⁻¹ if the number of evaluation points increases to infinity. Moreover, we show that our approach is also useful for deterministic functions in which numerical errors are involved. We provide also an optimal division between the number of gridpoints and replicates in case the number of evaluations is fixed. Further, it is shown that the estimation of the derivatives is more robust when the number of evaluation points is increased. Finally, test results show the practical use of the proposed method. We thank Jack Kleijnen, Gül Gürkan, and Peter Glynn for useful remarks on an earlier version of this paper. We thank Henk Norde for the proof of Lemma 2.2.     

Uncertainty under a multivariate nested-error regression model with logarithmic transformation

This work aims to predict exponentials of mixed effects under a multivariate linear regression model with one random factor. Such quantities are of particular interest in prediction problems where the dependent variable is the logarithm of the variable that is the object of inference. Bias-corrected empirical predictors of the target quantities are defined. A second-order approximation for the mean crossed product error of two of these predictors is obtained, where the mean squared error is a particular case. An estimator of the mean crossed product error with second-order bias is proposed. Finally, results are illustrated through an application related to small area estimation.     

MSE's of prediction in growth curve model with covariance structures

We consider the growth curve model with covariance structures: positive-definite, uniform covariance structure and serial covariance structure. Two types of prediction problems are studied in this paper. One is called the conditional prediction problem and the other is called the extended prediction problem. For both types of prediction problems, the mean squared error for a serial covariance structure is obtained for the estimates based on the conditional expectation: the mean squared error for an unrestricted covariance structure is compared with the mean squared error for a uniform covariance structure or a serial covariance structure. These results are exemplified by two sets of real data.This research was supported in part by Grant-in-Aid for general Scientific Research, The Ministry of Education, Science and Culture under Contract Number 03640239.     

Improved estimation of a covariance matrix in an elliptically contoured matrix distribution

In this paper, the problem of estimating the covariance matrix of the elliptically contoured distribution (ECD) is considered. A new class of estimators which shrink the eigenvalues towards their arithmetic mean is proposed. It is shown that this new estimator dominates the unbiased estimator under the squared error loss function. Two special classes of ECD, namely, the multivariate-elliptical t distribution and the ε-contaminated normal distribution are considered. A simulation study is carried out and indicates that this new shrinkage estimator provides a substantial improvement in risk under most situations.     

Inequalities for mean squared error of multidimensional kernel density estimations

The upper limits for the integral mean squared error of multidimensional kernel density estimations are obtained. In particular, it is showed that under certain conditions of regularity, the real errors are always smaller than the asymptotic.     

一类Lorentz曲线的本质属性与拟合研究

给出曲线族L(p)=p~α(1-(1-p)~β)~γ,0β1是Lorentz曲线的充分必要条件为:α≥0,γ≥0且α+γ≥1.结果是已有相关结论的推广.在此基础上,在一类Lorentz曲线族中选择了一条最佳曲线作为衡量国家收入分配的指标.用2013全国研究生数学建模竞赛E题数据,得到了相当精确的结果(均方误差精确到10~(-6).     

Bandwidth choice for local polynomial estimation of smooth boundaries

Local polynomial methods hold considerable promise for boundary estimation, where they offer unmatched flexibility and adaptivity. Most rival techniques provide only a single order of approximation; local polynomial approaches allow any order desired. Their more conventional rivals, for example high-order kernel methods in the context of regression, do not have attractive versions in the case of boundary estimation. However, the adoption of local polynomial methods for boundary estimation is inhibited by lack of knowledge about their properties, in particular about the manner in which they are influenced by bandwidth; and by the absence of techniques for empirical bandwidth choice. In the present paper we detail the way in which bandwidth selection determines mean squared error of local polynomial boundary estimators, showing that it is substantially more complex than in regression settings. For example, asymptotic formulae for bias and variance contributions to mean squared error no longer decompose into monotone functions of bandwidth. Nevertheless, once these properties are understood, relatively simple empirical bandwidth selection methods can be developed. We suggest a new approach to both local and global bandwidth choice, and describe its properties.     

A Loss function approach to identifying environmental exceedances

A frequent problem in environmental science is the prediction of extrema and exceedances. It is well known that Bayesian and empirical-Bayesian predictors based on integrated squared error loss (ISEL) tend to overshrink predictions of extrema toward the mean. In this paper, we consider a geostatistical extension of the weighted rank squared error loss function (WRSEL) of Wright et al. (2003), which we call the integrated weighted quantile squared error loss (IWQSEL), as the basis for prediction of exceedances and their spatial location. The loss function is based on an ordering of the underlying spatial process using a spatially averaged cumulative distribution function. We illustrate this methodology with a Bayesian analysis of surface-nitrogen concentrations in the Chesapeake Bay and compare the new IWQSEL predictor with a standard ISEL predictor. We also give a comparison to predicted extrema obtained from a “plug-in” goestatistical analysis. AMS 2000 Subject Classification Primary—62M30; Secondary—62H11     

<Emphasis Type="Italic">L</Emphasis><Subscript>1</Subscript> linear interpolator for missing values in time series

We propose a minimum mean absolute error linear interpolator (MMAELI), based on theL ₁ approach. A linear functional of the observed time series due to non-normal innovations is derived. The solution equation for the coefficients of this linear functional is established in terms of the innovation series. It is found that information implied in the innovation series is useful for the interpolation of missing values. The MMAELIs of the AR(1) model with innovations following mixed normal andt distributions are studied in detail. The MMAELI also approximates the minimum mean squared error linear interpolator (MMSELI) well in mean squared error but outperforms the MMSELI in mean absolute error. An application to a real series is presented. Extensions to the general ARMA model and other time series models are discussed. This research was supported by a CityU Research Grant and Natural Science Foundation of China.