Density estimation by the penalized combinatorial method

Let f be an unknown multivariate density belonging to a prespecified parametric class of densities, , where k is unknown, but for all k and each has finite Vapnik-Chervonenkis dimension. Given an i.i.d. sample of size n drawn from f, we show that it is possible to select automatically, and without extra restrictions on f, an estimate with the property that . Our method is inspired by the combinatorial tools developed in Devroye and Lugosi (Combinatorial Methods in Density Estimation, Springer, New York, 2001) and it includes a wide range of density models, such as mixture models or exponential families.