Minimax estimation of means of multivariate normal mixtures

Assume X = (X₁, …, X_p)′ is a normal mixture distribution with density w.r.t. Lebesgue measure, , where Σ is a known positive definite matrix and F is any known c.d.f. on (0, ∞). Estimation of the mean vector under an arbitrary known quadratic loss function _Q(θ, a) = (a − θ)′ Q(a − θ), Q a positive definite matrix, is considered. An unbiased estimator of risk is obatined for an arbitrary estimator, and a sufficient condition for estimators to be minimax is then achieved. The result is applied to modifying all the Stein estimators for the means of independent normal random variables to be minimax estimators for the problem considered here. In particular the results apply to the Stein class of limited translation estimators.     

Nonparametric iterative estimation of multivariate binary density

In this paper, an iterative estimate of the multivariate density is proposed when the variables are binary in nature. Some properties of this estimate are also discussed. Finally, applications of this estimate are discussed in the areas of pattern recognition and reliability.     

L p-consistency of multivariate density estimates

L _p notion of the weak, mean, and strong consistency of the kernel method of multivariate density estimation is proposed and studied. The results expand, unify, or generalize most known results in the literature. Rates of convergence in mean and strongL _p-consistencies are presented.     

Root n bandwidths selectors in multivariate kernel density estimation

Based on a random sample of size n from an unknown d-dimensional density f, the problem of selecting the bandwidths in kernel estimation of f is investigated. The optimal root n relative convergence rate for bandwidth selection is established and the information bounds in this convergence are given, and a stabilized bandwidth selector (SBS) is proposed. It is known that for all d the bandwidths selected by the least squares cross-validation (LSCV) have large sample variations. The proposed SBS, as an improvement of LSCV, will reduce the variation of LSCV without significantly inflating its bias. The key idea of the SBS is to modify the d-dimensional sample characteristic function beyond some cut-off frequency in estimating the integrated squared bias. It is shown that for all d and sufficiently smooth f and kernel, if the bandwidth in each coordinate direction varies freely, then the multivariate SBS is asymptotically normal with the optimal root n relative convergence rate and achieves the (conjectured) lower bound on the covariance matrix.Part of the research was done while the first author was visiting the Institute of Statistical Science, Academia Sinica, Taipei, Taiwan. This work was supported by grant NSC-89-2118-M-006-011, NSC-90-2118-M-006-013 and NSC-91-2118-M-006-005 of National Science Council of Taiwan, R.O.C.     

Semi-parametric Bayesian estimation of mixed-effects models using the multivariate skew-normal distribution

In this paper, we develop a semi-parametric Bayesian estimation approach through the Dirichlet process (DP) mixture in fitting linear mixed models. The random-effects distribution is specified by introducing a multivariate skew-normal distribution as base for the Dirichlet process. The proposed approach efficiently deals with modeling issues in a wide range of non-normally distributed random effects. We adopt Gibbs sampling techniques to achieve the parameter estimates. A small simulation study is conducted to show that the proposed DP prior is better at the prediction of random effects. Two real data sets are analyzed and tested by several hypothetical models to illustrate the usefulness of the proposed approach.     

Nonparametric likelihood based estimation for a multivariate Lipschitz density

We consider a problem of nonparametric density estimation under shape restrictions. We deal with the case where the density belongs to a class of Lipschitz functions. Devroye [L. Devroye, A Course in Density Estimation, in: Progress in Probability and Statistics, vol. 14, Birkhäuser Boston Inc., Boston, MA, 1987] considered these classes of estimates as tailor-made estimates, in contrast in some way to universally consistent estimates. In our framework we get the existence and uniqueness of the maximum likelihood estimate as well as strong consistency. This NPMLE can be easily characterized but it is not easy to compute. Some simpler approximations are also considered.     

Tree-based multivariate regression and density estimation with right-censored data

We propose a unified strategy for estimator construction, selection, and performance assessment in the presence of censoring. This approach is entirely driven by the choice of a loss function for the full (uncensored) data structure and can be stated in terms of the following three main steps. (1) First, define the parameter of interest as the minimizer of the expected loss, or risk, for a full data loss function chosen to represent the desired measure of performance. Map the full data loss function into an observed (censored) data loss function having the same expected value and leading to an efficient estimator of this risk. (2) Next, construct candidate estimators based on the loss function for the observed data. (3) Then, apply cross-validation to estimate risk based on the observed data loss function and to select an optimal estimator among the candidates. A number of common estimation procedures follow this approach in the full data situation, but depart from it when faced with the obstacle of evaluating the loss function for censored observations. Here, we argue that one can, and should, also adhere to this estimation road map in censored data situations.Tree-based methods, where the candidate estimators in Step 2 are generated by recursive binary partitioning of a suitably defined covariate space, provide a striking example of the chasm between estimation procedures for full data and censored data (e.g., regression trees as in CART for uncensored data and adaptations to censored data). Common approaches for regression trees bypass the risk estimation problem for censored outcomes by altering the node splitting and tree pruning criteria in manners that are specific to right-censored data. This article describes an application of our unified methodology to tree-based estimation with censored data. The approach encompasses univariate outcome prediction, multivariate outcome prediction, and density estimation, simply by defining a suitable loss function for each of these problems. The proposed method for tree-based estimation with censoring is evaluated using a simulation study and the analysis of CGH copy number and survival data from breast cancer patients.     

Nonparametric estimation of mixed partial derivatives of a multivariate density

On the basis of a random sample of size n on an m-dimensional random vector X, this note proposes a class of estimators f_n^(p) of f^(p), where f is a density of X w.r.t. a σ-finite measure dominated by the Lebesgue measure on R^m, p = (p₁,…,p_m), p_j ≥ 0, fixed integers, and for x = (x₁,…,x_m) in R^m, f^(p)(x) = ?^p₁+…+p_m f(x)/(?^p₁x₁ … ?^p_mx_m). Asymptotic unbiasedness as well as both almost sure and mean square consistencies of f_n^(p) are examined. Further, a necessary and sufficient condition for uniform asymptotic unbisedness or for uniform mean square consistency of f_n^(p) is given. Finally, applications of estimators of this note to certain statistical problems are pointed out.     

Convergence properties of an empirical error criterion for multivariate density estimation

For the purpose of comparing different nonparametric density estimators, Wegman (J. Statist. Comput. Simulation 1 225–245) introduced an empirical error criterion. In a recent paper by Hall (Stochastic Process. Appl. 13 11–25) it is shown that this empirical error criterion converges to the mean integrated square error. Here, in the case of kernel estimation, the results of Hall are improved in several ways, most notably multivariate densities are treated and the range of allowable bandwidths is extended. The techniques used here are quite different from those of Hall, which demonstrates that the elegant Brownian Bridge approximation of Komlós, Major, and Tusnády (Z. Warsch. Verw. Gebrete 32 111–131) does not always give the strongest results possible.     

Convergence rates for unconstrained bandwidth matrix selectors in multivariate kernel density estimation

Progress in selection of smoothing parameters for kernel density estimation has been much slower in the multivariate than univariate setting. Within the context of multivariate density estimation attention has focused on diagonal bandwidth matrices. However, there is evidence to suggest that the use of full (or unconstrained) bandwidth matrices can be beneficial. This paper presents some results in the asymptotic analysis of data-driven selectors of full bandwidth matrices. In particular, we give relative rates of convergence for plug-in selectors and a biased cross-validation selector.     

On estimation of analytic density functions in L p

Let X ₁,X ₂, … be a sequence of independent identically distributed random variables with an unknown density function f on R. The function f is assumed to belong to a certain class of analytic functions. The problem of estimation of f using L _p -risk, 1 ≤ p < ∞, is considered. A kernel-type estimator f _n based on X ₁, …, X _n is proposed and the upper bound on its asymptotic local maximum risk is established. Our result is consistent with a conjecture of Guerre and Tsybakov [7] and augments previous work in this area.     

A continuation approach to mode-finding of multivariate Gaussian mixtures and kernel density estimates

This paper presents a novel method of multi-objective optimization by learning automata (MOLA) to solve complex multi-objective optimization problems. MOLA consists of multiple automata which perform sequential search in the solution domain. Each automaton undertakes dimensional search in the selected dimension of the solution domain, and each dimension is divided into a certain number of cells. Each automaton performs a continuous search action, instead of discrete actions, within cells. The merits of MOLA have been demonstrated, in comparison with a multi-objective evolutionary algorithm based on decomposition (MOEA/D) and non-dominated sorting genetic algorithm II (NSGA-II), on eleven multi-objective benchmark functions and an optimal problem in the midwestern American electric power system which is integrated with wind power, respectively. The simulation results have shown that MOLA can obtain more accurate and evenly distributed Pareto fronts, in comparison with MOEA/D and NSGA-II.     

Nonparametric estimation of multivariate density with direct and auxiliary data and application

We consider the problem of multivariate density estimation, using samples from the distribution of interest as well as auxiliary samples from a related distribution. We assume that the data from the target distribution and the related distribution may occur individually as well as in pairs. Using nonparametric maximum likelihood estimator of the joint distribution, we derive a kernel density estimator of the marginal density. We show theoretically, in a simple special case, that the implied estimator of the marginal density has smaller integrated mean squared error than that of a similar estimator obtained by ignoring dependence of the paired observations. We establish consistency of the marginal density estimator under suitable conditions. We demonstrate small sample superiority of the proposed estimator over the estimator that ignores dependence of the samples, through a simulation study with dependent and non-normal populations. The application of the density estimator in nonparametric classification is also discussed. It is shown that the misclassification probability of the resulting classifier is asymptotically equivalent to that of the Bayes classifier. We also include a data analytic illustration.     

The estimation of a multivariate linear relation

A multivariate linear relation η_n = β₀ξ_n is considered, in which ξ_n and η_n are observed subject to white noise errors, with covariance matrices σ₀, ω₀ respectively. If their elements lie in the null space of a suitable vector function, β₀, σ₀, ω₀ may be uniquely defined by second-order functions of the data. The asymptotic properties of estimates of β₀, σ₀, ω₀ are established under relatively mild conditions. We explore the possibility that explicit formulas for consistent estimates of β₀, σ₀, ω₀ may be available.     

Stochastic comparisons of multivariate mixtures

In this paper we establish multivariate hazard rate, multivariate reverse hazard rate, and multivariate likelihood ratio stochastic orderings among multivariate random mapping (mixture) distributions. The new results streamline and simplify the proofs of some partial results that have recently appeared in the literature. Some applications in reliability theory and risk management are described.     

The estimation of multivariate random coefficient autoregressive models

Using a two stage regression procedure estimates of the unknown parameters of a class of multivariate random coefficient autoregressive models are obtained. The estimates are shown, under fairly general conditions, to be strongly consistent and to have a distribution which converges to that of a normally distributed random vector.     

Shape mixtures of multivariate skew-normal distributions

Classes of shape mixtures of independent and dependent multivariate skew-normal distributions are considered and some of their main properties are studied. If interpreted from a Bayesian point of view, the results obtained in this paper bring tractability to the problem of inference for the shape parameter, that is, the posterior distribution can be written in analytic form. Robust inference for location and scale parameters is also obtained under particular conditions.