Robust semiparametric M-estimation and the weighted bootstrap

M-estimation is a widely used technique for statistical inference. In this paper, we study properties of ordinary and weighted M-estimators for semiparametric models, especially when there exist parameters that cannot be estimated at the convergence rate. Results on consistency, rates of convergence for all parameters, and consistency and asymptotic normality for the Euclidean parameters are provided. These results, together with a generic paradigm for studying semiparametric M-estimators, provide a valuable extension to previous related research on semiparametric maximum-likelihood estimators (MLEs). Although penalized M-estimation does not in general fit in the framework we discuss here, it is shown for a great variety of models that many of the forgoing results still hold, including the consistency and asymptotic normality of the Euclidean parameters. For semiparametric M-estimators that are not likelihood based, general inference procedures for the Euclidean parameters have not previously been developed. We demonstrate that our paradigm leads naturally to verification of the validity of the weighted bootstrap in this setting. For illustration, several examples are investigated in detail. The new M-estimation framework and accompanying weighted bootstrap technique shed light on a universal way of investigating semiparametric models.     

Limit distributions of least squares estimators in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have lately been introduced (cf. [V. Krätschmer, Induktive Statistik auf Basis unscharfer Meßkonzepte am Beispiel linearer Regressionsmodelle, unpublished postdoctoral thesis, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001; V. Krätschmer, Least squares estimation in linear regression models with vague concepts, Fuzzy Sets and Systems, accepted for publication]) to improve the empirical meaningfulness of the relationships between the involved items by a more sensitive attention to the problems of data measurement, in particular, the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least squares method. In another recent contribution (cf. [V. Krätschmer, Strong consistency of least squares estimation in linear regression models with vague concepts, J. Multivar. Anal., accepted for publication]) strong consistency and -consistency of this generalized least squares estimation have been shown. The aim of the paper is to complete these results by an investigation of the limit distributions of the estimators. It turns out that the classical results can be transferred, in some cases even asymptotic normality holds.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.     

Second order optimality for estimators in time series regression models

We consider the second order asymptotic properties of an efficient frequency domain regression coefficient estimator proposed by Hannan [Regression for time series, Proc. Sympos. Time Series Analysis (Brown Univ., 1962), Wiley, New York, 1963, pp. 17-37]. This estimator is a semiparametric estimator based on nonparametric spectral estimators. We derive the second order Edgeworth expansion of the distribution of . Then it is shown that the second order asymptotic properties are independent of the bandwidth choice for residual spectral estimator, which implies that has the same rate of convergence as in regular parametric estimation. This is a sharp contrast with the general semiparametric estimation theory. We also examine the second order Gaussian efficiency of . Numerical studies are given to confirm the theoretical results.     

On the block thresholding wavelet estimators with censored data

We consider block thresholding wavelet-based density estimators with randomly right-censored data and investigate their asymptotic convergence rates. Unlike for the complete data case, the empirical wavelet coefficients are constructed through the Kaplan-Meier estimators of the distribution functions in the censored data case. On the basis of a result of Stute [W. Stute, The central limit theorem under random censorship, Ann. Statist. 23 (1995) 422-439] that approximates the Kaplan-Meier integrals as averages of i.i.d. random variables with a certain rate in probability, we can show that these wavelet empirical coefficients can be approximated by averages of i.i.d. random variables with a certain error rate in L². Therefore we can show that these estimators, based on block thresholding of empirical wavelet coefficients, achieve optimal convergence rates over a large range of Besov function classes , p≥2, q≥1 and nearly optimal convergence rates when 1≤p<2. We also show that these estimators achieve optimal convergence rates over a large class of functions that involve many irregularities of a wide variety of types, including chirp and Doppler functions, and jump discontinuities. Therefore, in the presence of random censoring, wavelet estimators still provide extensive adaptivity to many irregularities of large function classes. The performance of the estimators is tested via a modest simulation study.     

The Stein phenomenon for monotone incomplete multivariate normal data

We establish the Stein phenomenon in the context of two-step, monotone incomplete data drawn from , a (p+q)-dimensional multivariate normal population with mean and covariance matrix . On the basis of data consisting of n observations on all p+q characteristics and an additional N−n observations on the last q characteristics, where all observations are mutually independent, denote by the maximum likelihood estimator of . We establish criteria which imply that shrinkage estimators of James-Stein type have lower risk than under Euclidean quadratic loss. Further, we show that the corresponding positive-part estimators have lower risk than their unrestricted counterparts, thereby rendering the latter estimators inadmissible. We derive results for the case in which is block-diagonal, the loss function is quadratic and non-spherical, and the shrinkage estimator is constructed by means of a nondecreasing, differentiable function of a quadratic form in . For the problem of shrinking to a vector whose components have a common value constructed from the data, we derive improved shrinkage estimators and again determine conditions under which the positive-part analogs have lower risk than their unrestricted counterparts.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Finite-sample inference with monotone incomplete multivariate normal data, II

We continue our recent work on inference with two-step, monotone incomplete data from a multivariate normal population with mean and covariance matrix . Under the assumption that is block-diagonal when partitioned according to the two-step pattern, we derive the distributions of the diagonal blocks of and of the estimated regression matrix, . We represent in terms of independent matrices; derive its exact distribution, thereby generalizing the Wishart distribution to the setting of monotone incomplete data; and obtain saddlepoint approximations for the distributions of and its partial Iwasawa coordinates. We prove the unbiasedness of a modified likelihood ratio criterion for testing , where is a given matrix, and obtain the null and non-null distributions of the test statistic. In testing , where and are given, we prove that the likelihood ratio criterion is unbiased and obtain its null and non-null distributions. For the sphericity test, , we obtain the null distribution of the likelihood ratio criterion. In testing we show that a modified locally most powerful invariant statistic has the same distribution as a Bartlett-Pillai-Nanda trace statistic in multivariate analysis of variance.     

Improving on the mle of a bounded location parameter for spherical distributions

For the problem of estimating under squared error loss the location parameter of a p-variate spherically symmetric distribution where the location parameter lies in a ball of radius m, a general sufficient condition for an estimator to dominate the maximum likelihood estimator is obtained. Dominance results are then made explicit for the case of a multivariate student distribution with d degrees of freedom and, in particular, we show that the Bayes estimator with respect to a uniform prior on the boundary of the parameter space dominates the maximum likelihood estimator whenever and d?p. The sufficient condition matches the one obtained by Marchand and Perron (Ann. Statist. 29 (2001) 1078) in the normal case with identity covariance matrix. Furthermore, we derive an explicit class of estimators which, for , dominate the maximum likelihood estimator simultaneously for the normal distribution with identity covariance matrix and for all multivariate student distributions with d degrees of freedom, d?p. Finally, we obtain estimators which dominate the maximum likelihood estimator simultaneously for all distributions in the subclass of scale mixtures of normals for which the scaling random variable is bounded below by some positive constant with probability one.     

Statistical inference for the index parameter in single-index models

In this paper, we are concerned with statistical inference for the index parameter in the single-index model . Based on the estimates obtained by the local linear method, we extend the generalized likelihood ratio test to the single-index model. We investigate the asymptotic behaviour of the proposed test and demonstrate that its limiting null distribution follows a χ²-distribution, with the scale constant and the number of degrees of freedom being independent of nuisance parameters or functions, which is called the Wilks phenomenon. A simulated example is used to illustrate the performance of the testing approach.     

Asymptotic expansions of test statistics for dimensionality and additional information in canonical correlation analysis when the dimension is large

This paper examines asymptotic expansions of test statistics for dimensionality and additional information in canonical correlation analysis based on a sample of size N=n+1 on two sets of variables, i.e.,  and . These problems are related to dimension reduction. The asymptotic approximations of the statistics have been studied extensively when dimensions p₁ and p₂ are fixed and the sample size N tends to infinity. However, the approximations worsen as p₁ and p₂ increase. This paper derives asymptotic expansions of the test statistics when both the sample size and dimension are large, assuming that and have a joint (p₁+p₂)-variate normal distribution. Numerical simulations revealed that this approximation is more accurate than the classical approximation as the dimension increases.     

High-dimensional asymptotic expansions for the distributions of canonical correlations

This paper examines asymptotic distributions of the canonical correlations between and with q≤p, based on a sample of size of N=n+1. The asymptotic distributions of the canonical correlations have been studied extensively when the dimensions q and p are fixed and the sample size N tends toward infinity. However, these approximations worsen when q or p is large in comparison to N. To overcome this weakness, this paper first derives asymptotic distributions of the canonical correlations under a high-dimensional framework such that q is fixed, m=n−p→∞ and c=p/n→c₀∈[0,1), assuming that and have a joint (q+p)-variate normal distribution. An extended Fisher’s z-transformation is proposed. Then, the asymptotic distributions are improved further by deriving their asymptotic expansions. Numerical simulations revealed that our approximations are more accurate than the classical approximations for a large range of p,q, and n and the population canonical correlations.     

Estimating the parametric component of nonlinear partial spline model

Consider a nonlinear partial spline model . This article studies the estimation problem of when g₀ is approximated by some graduating function. Some asymptotic results for are derived. In particular, it is shown that can be estimated with the usual parametric convergence rate without undersmoothing g₀.     

Nonparametric comparison of regression functions

In this work, we provide a new methodology for comparing regression functions m₁ and m₂ from two samples. Since apart from smoothness no other (parametric) assumptions are required, our approach is based on a comparison of nonparametric estimators and of m₁ and m₂, respectively. The test statistics incorporate weighted differences of and computed at selected points. Since the design variables may come from different distributions, a crucial question is where to compare the two estimators. As our main results we obtain the limit distribution of (properly standardized) under the null hypothesis H₀:m₁=m₂ and under local and global alternatives. We are also able to choose the weight function so as to maximize the power. Furthermore, the tests are asymptotically distribution free under H₀ and both shift and scale invariant. Several such ’s may then be combined to get Maximin tests when the dimension of the local alternative is finite. In a simulation study we found out that our tests achieve the nominal level and already have excellent power for small to moderate sample sizes.