Second derivative estimation using harmonic analysis

Simulation sensitivity analysis is an important problem for simulation practitioners analyzing complex systems. The significance of this problem has resulted in the development of various gradient estimators that can be used to address this issue. Although higher derivative estimators have been discussed concurrently, less attention has been given to assess the efficiency and feasibility of computing such estimators. In this paper, two second derivative estimators are presented. The first estimators, called the HFD estimators, combine harmonic gradient estimators with finite differences second derivative estimators. The resulting hybrid estimators requireO(p) fewer simulation runs to implement compared to the straightforward finite differences approach, wherep is the number of input parameters in the simulation model. The second estimators, called the HA estimators, incorporate harmonic analysis directly, requiring one or two simulation runs to implement, depending on whether a control variate simulation run is made. Expressions for the bias and the variance of the HFD and the HA estimators (with and without variance reduction techniques) are derived. Optimal mean squared error convergence rates are also discussed. In particular, the convergence rates for both these estimators are shown to be the same, though the computational performance of the HFD estimators is better than that for the HA estimators on anM/M/1 queue simulation model. Computational results for the HFD estimators on an (s, S) inventory system simulation model are also included.     

Asymptotic properties of some subset vector autoregressive process estimators

We establish consistency and derive asymptotic distributions for estimators of the coefficients of a subset vector autoregressive (SVAR) process. Using a martingale central limit theorem, we first derive the asymptotic distribution of the subset least squares (LS) estimators. Exploiting the similarity of closed form expressions for the LS and Yule–Walker (YW) estimators, we extend the asymptotics to the latter. Using the fact that the subset Yule–Walker and recently proposed Burg estimators satisfy closely related recursive algorithms, we then extend the asymptotic results to the Burg estimators. All estimators are shown to have the same limiting distribution.     

Folded overlapping variance estimators for simulation

We propose and analyze a new class of estimators for the variance parameter of a steady-state simulation output process. The new estimators are computed by averaging individual estimators from “folded” standardized time series based on overlapping batches composed of consecutive observations. The folding transformation on each batch can be applied more than once to produce an entire set of estimators. We establish the limiting distributions of the proposed estimators as the sample size tends to infinity while the ratio of the sample size to the batch size remains constant. We give analytical and Monte Carlo results showing that, compared to their counterparts computed from nonoverlapping batches, the new estimators have roughly the same bias but smaller variance. In addition, these estimators can be computed with order-of-sample-size work.     

Improvement in estimating the population mean in simple random sampling

This paper proposes some estimators for the population mean using the ratio estimators presented in [C. Kadilar, H. Cingi, Ratio estimators in simple random sampling, Applied Mathematics and Computation 151 (2004) 893–902] and shows that all proposed estimators are always more efficient than the ratio estimators. This result is also supported by a numerical example.     

Minimum f-divergence estimators and quasi-likelihood functions

Maximum quasi-likelihood estimators have several nice asymptotic properties. We show that, in many situations, a family of estimators, called the minimum f-divergence estimators, can be defined such that each estimator has the same asymptotic properties as the maximum quasi-likelihood estimator. The family of minimum f-divergence estimators include the maximum quasi-likelihood estimators as a special case. When a quasi-likelihood is the log likelihood from some exponential family, Amari's dual geometries can be used to study the maximum likelihood estimator. A dual geometric structure can also be defined for more general quasi-likelihood functions as well as for the larger family of minimum f-divergence estimators. The relationship between the f-divergence and the quasi-likelihood function and the relationship between the f-divergence and the power divergence is discussed.This work was supported by National Science Foundation grant DMS 88-03584.     

Rates of convergence for minimum contrast estimators

Summary We shall present here a general study of minimum contrast estimators in a nonparametric setting (although our results are also valid in the classical parametric case) for independent observations. These estimators include many of the most popular estimators in various situations such as maximum likelihood estimators, least squares and other estimators of the regression function, estimators for mixture models or deconvolution... The main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters. Optimal rates depending on entropy conditions are already known, at least for some of the models involved, and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large. But, under some circumstances (large entropies or changes in the entropy structure due to local perturbations), the resulting the rates are only suboptimal. Counterexamples are constructed which show that the phenomenon is real for non-parametric maximum likelihood or regression. This proves that, under purely metric assumptions, our theorem is optimal and that minimum contrast estimators happen to be suboptimal.     

Malliavin Greeks without Malliavin calculus

We derive and analyze Monte Carlo estimators of price sensitivities (“Greeks”) for contingent claims priced in a diffusion model. There have traditionally been two categories of methods for estimating sensitivities: methods that differentiate paths and methods that differentiate densities. A more recent line of work derives estimators through Malliavin calculus. The purpose of this article is to investigate connections between Malliavin estimators and the more traditional and elementary pathwise method and likelihood ratio method. Malliavin estimators have been derived directly for diffusion processes, but implementation typically requires simulation of a discrete-time approximation. This raises the question of whether one should discretize first and then differentiate, or differentiate first and then discretize. We show that in several important cases the first route leads to the same estimators as are found through Malliavin calculus, but using only elementary techniques. Time-averaging of multiple estimators emerges as a key feature in achieving convergence to the continuous-time limit.     

Robustness of Stein-type estimators under a non-scalar error covariance structure

The Stein-rule (SR) and positive-part Stein-rule (PSR) estimators are two popular shrinkage techniques used in linear regression, yet very little is known about the robustness of these estimators to the disturbances’ deviation from the white noise assumption. Recent studies have shown that the OLS estimator is quite robust, but whether this is so for the SR and PSR estimators is less clear as these estimators also depend on the F statistic which is highly susceptible to covariance misspecification. This study attempts to evaluate the effects of misspecifying the disturbances as white noise on the SR and PSR estimators by a sensitivity analysis. Sensitivity statistics of the SR and PSR estimators are derived and their properties are analyzed. We find that the sensitivity statistics of these estimators exhibit very similar properties and both estimators are extremely robust to MA(1) disturbances and reasonably robust to AR(1) disturbances except for the cases of severe autocorrelation. The results are useful in light of the rising interest of the SR and PSR techniques in the applied literature.     

Bayesian inference using record values from Rayleigh model with application

In this article, based on a set of upper record values from a Rayleigh distribution, Bayesian and non-Bayesian approaches have been used to obtain the estimators of the parameter, and some lifetime parameters such as the reliability and hazard functions. Bayes estimators have been developed under symmetric (squared error) and asymmetric (LINEX and general entropy (GE)) loss functions. These estimators are derived using the informative and non-informative prior distributions for σ. We compare the performance of the presented Bayes estimators with known, non-Bayesian, estimators such as the maximum likelihood (ML) and the best linear unbiased (BLU) estimators. We show that Bayes estimators under the asymmetric loss functions are superior to both the ML and BLU estimators. The highest posterior density (HPD) intervals for the Rayleigh parameter and its reliability and hazard functions are presented. Also, Bayesian prediction intervals of the future record values are obtained and discussed. Finally, practical examples using real record values are given to illustrate the application of the results.     

Improving Penalized Least Squares through Adaptive Selection of Penalty and Shrinkage

Estimation of the mean function in nonparametric regression is usefully separated into estimating the means at the observed factor levels—a one-way layout problem—and interpolation between the estimated means at adjacent factor levels. Candidate penalized least squares (PLS) estimators for the mean vector of a one-way layout are expressed as shrinkage estimators relative to an orthogonal regression basis determined by the penalty matrix. The shrinkage representation of PLS suggests a larger class of candidate monotone shrinkage (MS) estimators. Adaptive PLS and MS estimators choose the shrinkage vector and penalty matrix to minimize estimated risk. The actual risks of shrinkage-adaptive estimators depend strongly upon the economy of the penalty basis in representing the unknown mean vector. Local annihilators of polynomials, among them difference operators, generate penalty bases that are economical in a range of examples. Diagnostic techniques for adaptive PLS or MS estimators include basis-economy plots and estimates of loss or risk.     

Statistical estimation in partial linear models with covariate data missing at random

In this paper, we consider the partial linear model with the covariables missing at random. A model calibration approach and a weighting approach are developed to define the estimators of the parametric and nonparametric parts in the partial linear model, respectively. It is shown that the estimators for the parametric part are asymptotically normal and the estimators of g(·) converge to g(·) with an optimal convergent rate. Also, a comparison between the proposed estimators and the complete case estimator is made. A simulation study is conducted to compare the finite sample behaviors of these estimators based on bias and standard error.     

Nonparametric estimation of the spectral measure,and associated dependence measures,for multivariate extreme values using a limiting conditional representation

The traditional approach to multivariate extreme values has been through the multivariate extreme value distribution G, characterised by its spectral measure H and associated Pickands’ dependence function A. More generally, for all asymptotically dependent variables, H determines the probability of all multivariate extreme events. When the variables are asymptotically dependent and under the assumption of unit Fréchet margins, several methods exist for the estimation of G, H and A which use variables with radial component exceeding some high threshold. For each of these characteristics, we propose new asymptotically consistent nonparametric estimators which arise from Heffernan and Tawn’s approach to multivariate extremes that conditions on variables with marginal values exceeding some high marginal threshold. The proposed estimators improve on existing estimators in three ways. First, under asymptotic dependence, they give self-consistent estimators of G, H and A; existing estimators are not self-consistent. Second, these existing estimators focus on the bivariate case, whereas our estimators extend easily to describe dependence in the multivariate case. Finally, for asymptotically independent cases, our estimators can model the level of asymptotic independence; whereas existing estimators for the spectral measure treat the variables as either being independent, or asymptotically dependent. For asymptotically dependent bivariate random variables, the new estimators are found to compare favourably with existing estimators, particularly for weak dependence. The method is illustrated with an application to finance data.     

Surface tensor estimation from linear sections

From Crofton's formula for Minkowski tensors we derive stereological estimators of translation invariant surface tensors of convex bodies in the n‐dimensional Euclidean space. The estimators are based on one‐dimensional linear sections. In a design based setting we suggest three types of estimators. These are based on isotropic uniform random lines, vertical sections, and non‐isotropic random lines, respectively. Further, we derive estimators of the specific surface tensors associated with a stationary process of convex particles in the model based setting.     

More Higher-Order Efficiency: Concentration Probability

Based on concentration probability of estimators about a true parameter, third-order asymptotic efficiency of the first-order bias-adjusted MLE within the class of first-order bias-adjusted estimators has been well established in a variety of probability models. In this paper we consider the class of second-order bias-adjusted Fisher consistent estimators of a structural parameter vector on the basis of an i.i.d. sample drawn from a curved exponential-type distribution, and study the asymptotic concentration probability, about a true parameter vector, of these estimators up to the fifth-order. In particular, (i) we show that third-order efficient estimators are always fourth-order efficient; (ii) a necessary and sufficient condition for fifth-order efficiency is provided; and finally (iii) the MLE is shown to be fifth-order efficient.     

An experimental survey of a posteriori Courant finite element error control for the Poisson equation

This comparison of some a posteriori error estimators aims at empirical evidence for a ranking of their performance for a Poisson model problem with conforming lowest order finite element discretizations. Modified residual-based error estimates compete with averaging techniques and two estimators based on local problem solving. Multiplicative constants are involved to achieve guaranteed upper and lower energy error bounds up to higher order terms. The optimal strategy combines various estimators.     

Multivariate numerical differentiation

We present an innovative method for multivariate numerical differentiation i.e. the estimation of partial derivatives of multidimensional noisy signals. Starting from a local model of the signal consisting of a truncated Taylor expansion, we express, through adequate differential algebraic manipulations, the desired partial derivative as a function of iterated integrals of the noisy signal. Iterated integrals provide noise filtering. The presented method leads to a family of estimators for each partial derivative of any order. We present a detailed study of some structural properties given in terms of recurrence relations between elements of a same family. These properties are next used to study the performance of the estimators. We show that some differential algebraic manipulations corresponding to a particular family of estimators lead implicitly to an orthogonal projection of the desired derivative in a Jacobi polynomial basis functions, yielding an interpretation in terms of the popular least squares. This interpretation allows one to (1) explain the presence of a spatial delay inherent to the estimators and (2) derive an explicit formula for the delay. We also show how one can devise, by a proper combination of different elementary estimators of a given order derivative, an estimator giving a delay of any prescribed value. The simulation results show that delay-free estimators are sensitive to noise. Robustness with respect to noise can be highly increased by utilizing voluntary-delayed estimators. A numerical implementation scheme is given in the form of finite impulse response digital filters. The effectiveness of our derivative estimators is attested by several numerical simulations.     

Estimation of Time Varying Linear Systems

Based on kernel and wavelet estimators of the evolutionary spectrum and cross-spectrum we propose nonlinear wavelet estimators of the time varying coefficients of a linear system, whose input and output are locally stationary processes, in the sense of Dahlhaus (1997). We obtain large sample properties of these estimators, present some simulated examples and derive results on the L ₂-risk for the wavelet threshold estimators, assuming that the coefficients belong to some smoothness class. This revised version was published online in June 2006 with corrections to the Cover Date.     

Improving estimation in speckled imagery

We propose an analytical bias correction for the maximum likelihood estimators of theG ₁ ⁰ distribution. This distribution is a very powerful tool for speckled imagery analysis, since it is capable of describing a wide range of target roughness. We compare the performance of the corrected estimators with the corresponding original version using Monte Carlo simulation. This second-order bias correction leads to estimators which are better from both the bias and mean square error criteria.     

Estimating cumulative incidence functions when the life distributions are constrained

In competing risks studies, the Kaplan-Meier estimators of the distribution functions (DFs) of lifetimes and the corresponding estimators of cumulative incidence functions (CIFs) are used widely when no prior information is available for these distributions. In some cases better estimators of the DFs of lifetimes are available when they obey some inequality constraints, e.g., if two lifetimes are stochastically or uniformly stochastically ordered, or some functional of a DF obeys an inequality in an empirical likelihood estimation procedure. If the restricted estimator of a lifetime differs from the unrestricted one, then the usual estimators of the CIFs will not add up to the lifetime estimator. In this paper we show how to estimate the CIFs in this case. These estimators are shown to be strongly uniformly consistent. In all cases we consider, when the inequality constraints are strict the asymptotic properties of the restricted and the unrestricted estimators are the same, thus providing the asymptotic properties of the restricted estimators essentially “free of charge”. We give an example to illustrate our procedure.     

Interval estimates for posterior probabilities in a multivariate normal classification model

This paper is devoted to the asymptotic distribution of estimators for the posterior probability that a p-dimensional observation vector originates from one of k normal distributions with identical covariance matrices. The estimators are based on training samples for the k distributions involved. Observation vector and prior probabilities are regarded as given constants. The validity of various estimators and approximate confidence intervals is investigated by simulation experiments.