A stochastic restricted ridge regression estimator

Groß [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64] proposed a restricted ridge regression estimator when exact restrictions are assumed to hold. When there are stochastic linear restrictions on the parameter vector, we introduce a new estimator by combining ideas underlying the mixed and the ridge regression estimators under the assumption that the errors are not independent and identically distributed. Apart from [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64], we call this new estimator as the stochastic restricted ridge regression (SRRR) estimator. The performance of the SRRR estimator over the mixed estimator in respect of the variance and the mean square error matrices is examined. We also illustrate our findings with a numerical example. The shrinkage generalized least squares (GLS) and the stochastic restricted shrinkage GLS estimators are proposed.     

Asymptotic efficiency of the arithmetic-mean estimator of the unknown mean of a homogeneous random field

We consider the estimation of the unknown mean of a homogeneous random field from observations on a system of homothetically expanding regions. We examine the asymptotic behavior of the variance of the arithmetic-mean estimator. The arithmetic-mean estimator is shown to be asymptotically efficient in the class of linear estimators.Translated from Vychislitel'naya i Prikladnaya Matematika, No. 66, pp. 106–111, 1988.     

Probability Density Function Estimation Using Gamma Kernels

We consider estimating density functions which have support on [0, ) using some gamma probability densities as kernels to replace the fixed and symmetric kernel used in the standard kernel density estimator. The gamma kernels are non-negative and have naturally varying shape. The gamma kernel estimators are free of boundary bias, non-negative and achieve the optimal rate of convergence for the mean integrated squared error. The variance of the gamma kernel estimators at a distance x away from the origin is O(n ^–4/5 x ^–1/2) indicating a smaller variance as x increases. Finite sample comparisons with other boundary bias free kernel estimators are made via simulation to evaluate the performance of the gamma kernel estimators.     

Stein's idea and minimax admissible estimation of a multivariate normal mean

We consider estimation of a multivariate normal mean vector under sum of squared error loss.We propose a new class of minimax admissible estimator which are generalized Bayes with respect to a prior distribution which is a mixture of a point prior at the origin and a continuous hierarchical type prior. We also study conditions under which these generalized Bayes minimax estimators improve on the James–Stein estimator and on the positive-part James–Stein estimator.     

混合模型中方差分量估计的容许性及非负估计

对含有两个方差分量的线性混合模型, 本文构造了方差分量的一个线性估计类, 它包含许多常见的方差分量估计. 在这个类中我们建立了容许性的必要条件, 据此得到了两个新的改进估计. 最后我们讨论了方差分量的非负估计, 得到了优于方差分析估计和Tatsuya估计的正估计.     

Estimation of the entropy of a multivariate normal distribution

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster–Zidek-type estimators dominating it. The Brewster–Zidek-type estimator is shown to be generalized Bayes.     

Convergence rate of the best-r-point-average estimator for the maximizer of a nonparametric regression function

The best-r-point-average (BRPA) estimator of the maximizer of a regression function, proposed in Changchien (in: M.T. Chao, P.E. Cheng (Eds.), Proceedings of the 1990 Taipei Symposium in Statistics, June 28–30, 1990, pp. 63–78) has certain merits over the estimators derived through the estimation of the regression function. Some of the properties of the BRPA estimator have been studied in Chen et al. (J. Multivariate Anal. 57 (1996) 191) and Bai and Huang (Sankhya: Indian J. Statist. Ser. A. 61 (Pt. 2) (1999) 208–217). In this article, we further study the properties of the BRPA estimator and give its convergence rate under some quite general conditions. Simulation results are presented for the illustration of the convergence rate. Some comparisons with existing estimators such as the Müller estimator are provided.     

A sequence of improvements over the James-Stein estimator

In this article, we consider the problem of estimating a p-variate (p ≥ 3) normal mean vector in a decision-theoretic setup. Using a simple property of the noncentral chi-square distribution, we have produced a sequence of smooth estimators dominating the James-Stein estimator and each improved estimator is better than the previous one. It is also shown by using a technique of [5]. J. Multivariate Anal.36 121–126) that our smooth estimators can be dominated by non-smooth estimators.     

方差分量的改进估计

本文研究一类方差分量模型中方差分量的改进估计问题,对单向分类随机模型的对应于随机效应的方差分量,我们研究了一个不变估计类,它包含了一些常用重要估计。证明了在均方误差准则下,在该估计类中不存在一致最优不变估计,且方差分析估计是不容许估计。在一个重要子估计类中,找到了一致最优估计。对于较一般的含两个方差分量的混合模型,我们研究了一个非负估计类的性质,给出了它们的分布,并建立了它们优于方差分析估计的充分     

方差分量的二次不变估计的非负改进

研究一类方差分量模型中的方差分量的估计改进问题,首先在含两个方差分量模型中给出σ21二次型估计类,并且此估计类还具有无偏性和不变性.考虑二次损失(δ-θ)2,在此估计类基础上放弃无偏性进行非负改进,不仅得到优于二次不变无偏估计类的σ21的非负二次不变估计类,而且还说明了它优于方差分析估计和最小均方误差估计,文献[5]中给出s>2时的非负改进,但是非负改进存在是有条件的,本文克服了这个缺陷.最后给出了非负改进存在的充分必要条件.     

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n^−1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.     

Optimal variance estimation based on lagged second-order difference in nonparametric regression

Differenced estimators of variance bypass the estimation of regression function and thus are simple to calculate. However, there exist two problems: most differenced estimators do not achieve the asymptotic optimal rate for the mean square error; for finite samples the estimation bias is also important and not further considered. In this paper, we estimate the variance as the intercept in a linear regression with the lagged Gasser-type variance estimator as dependent variable. For the equidistant design, our estimator is not only \(n^{1/2}\)-consistent and asymptotically normal, but also achieves the optimal bound in terms of estimation variance with less asymptotic bias. Simulation studies show that our estimator has less mean square error than some existing differenced estimators, especially in the cases of immense oscillation of regression function and small-sized sample.     

Asymptotic Comparisons of Several Variance Estimators and their Effects for Studentizations

In this paper we obtain asymptotic representations of several variance estimators of U-statistics and study their effects for studentizations via Edgeworth expansions. Jackknife, unbiased and Sen's variance estimators are investigated up to the order o^p(n^-1). Substituting these estimators to studentized U-statistics, the Edgeworth expansions with remainder term o(n^-1) are established and inverting the expansions, the effects on confidence intervals are discussed theoretically. We also show that Hinkley's corrected jackknife variance estimator is asymptotically equivalent to the unbiased variance estimator up to the order o^p(n^-1).     

Optimal importance sampling for Lévy processes

We develop importance sampling estimators for Monte Carlo pricing of European and path-dependent options in models driven by Lévy processes. Using results from the theory of large deviations for processes with independent increments, we compute an explicit asymptotic approximation for the variance of the pay-off under a time-dependent Esscher-style change of measure. Minimizing this asymptotic variance using convex duality, we then obtain an importance sampling estimator of the option price. We show that our estimator is logarithmically optimal among all importance sampling estimators. Numerical tests in the variance gamma model show consistent variance reduction with a small computational overhead.     

Estimating a Distribution Function for Censored Time Series Data

Consider a long term study, where a series of dependent and possibly censored failure times is observed. Suppose that the failure times have a common marginal distribution function, but they exhibit a mode of time series structure such as α-mixing. The inference on the marginal distribution function is of interest to us. The main results of this article show that, under some regularity conditions, the Kaplan–Meier estimator enjoys uniform consistency with rates, and a stochastic process generated by the Kaplan–Meier estimator converges weakly to a certain Gaussian process with a specified covariance structure. Finally, an estimator of the limiting variance of the Kaplan–Meier estimator is proposed and its consistency is established.     

Safe density ratio modeling

An important problem in logistic regression modeling is the existence of the maximum likelihood estimators. In particular, when the sample size is small, the maximum likelihood estimator of the regression parameters does not exist if the data are completely, or quasicompletely separated. Recognizing that this phenomenon has a serious impact on the fitting of the density ratio model–which is a semiparametric model whose profile empirical log-likelihood has the logistic form because of the equivalence between prospective and retrospective sampling–we suggest a linear programming methodology for examining whether the maximum likelihood estimators of the finite dimensional parameter vector of the model exist. It is shown that the methodology can be effectively utilized in the analysis of case–control gene expression data by identifying cases where the density ratio model cannot be applied. It is demonstrated that naive application of the density ratio model yields erroneous conclusions.     

A general ratio estimator and its application in model based inference

A general ratio estimator of a population total is proposed as an approximation to the estimator introduced by Srivastava (1985,Bull. Internat. Statist. Inst.,51(10.3), 1–16). This estimator incorporates additional information gathered during the survey in a new way. Statistical properties of the general ratio estimator are given and its relationship to the estimator proposed by Srivastava is explored. A special kind of general ratio estimator is suggested and it turns out to be very efficient in a simulation study when compared to several other commonly used estimators.The work of this author was supported by AFOSR grant #830080.     

Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient

We theoretically compare variances between the Infinitesimal Perturbation Analysis (IPA) estimator and the Likelihood Ratio (LR) estimator to Monte Carlo gradient for stochastic systems. The results presented in Cui et al. (2020) [2] on variance comparison between these two estimators are substantially improved. We also prove a practically interesting result that the IPA estimators to European vanilla and arithmetic Asian options' Delta, respectively, have smaller variance when the underlying asset's return process is independent with the initial price and square integrable.     

Admissible and minimax multiparameter estimation in exponential families

Consider p independent distributions each belonging to the one parameter exponential family with distribution functions absolutely continuous with respect to Lebesgue measure. For estimating the natural parameter vector with p ≥ p₀ (p₀ is typically 2 or 3), a general class of estimators dominating the minimum variance unbiased estimator (MVUE) or an estimator which is a known constant multiple of the MVUE is produced under different weighted squared error losses. Included as special cases are some results of Hudson [13] and Berger [5]. Also, for a subfamily of the general exponential family, a class of estimators dominating the MVUE of the mean vector or an estimator which is a known constant multiple of the MVUE is produced. The major tool is to obtain a general solution to a basic differential inequality.     

Generalized variance estimators in the multivariate gamma models

It is already known that the uniformly minimum variance unbiased (UMVU) estimator of the generalized variance always exists for any natural exponential family. However, in practice, this estimator is often difficult to obtain. This paper provides explicit forms of the UMVU estimators for the bivariate and symmetric multivariate gamma models, which are diagonal quadratic exponential families. For the non-independent multivariate gamma models, it is shown that the UMVU and the maximum likelihood estimators are not proportional.