Nonlinear approximation using Gaussian kernels

It is well known that nonlinear approximation has an advantage over linear schemes in the sense that it provides comparable approximation rates to those of the linear schemes, but to a larger class of approximands. This was established for spline approximations and for wavelet approximations, and more recently by DeVore and Ron (in press) [2] for homogeneous radial basis function (surface spline) approximations. However, no such results are known for the Gaussian function, the preferred kernel in machine learning and several engineering problems. We introduce and analyze in this paper a new algorithm for approximating functions using translates of Gaussian functions with varying tension parameters. At heart it employs the strategy for nonlinear approximation of DeVore-Ron, but it selects kernels by a method that is not straightforward. The crux of the difficulty lies in the necessity to vary the tension parameter in the Gaussian function spatially according to local information about the approximand: error analysis of Gaussian approximation schemes with varying tension are, by and large, an elusive target for approximators. We show that our algorithm is suitably optimal in the sense that it provides approximation rates similar to other established nonlinear methodologies like spline and wavelet approximations. As expected and desired, the approximation rates can be as high as needed and are essentially saturated only by the smoothness of the approximand.     

Positivity improvement and Gaussian kernels

We show that a positivity improving property of multilinear operators with Gaussian kernels can be determined, with sharp constants, by testing Gaussian functions only. This result can be considered as a reversed form of Lieb's theorem on maximizers of Gaussian kernels.     

Dimension dependent hypercontractivity for Gaussian kernels

We derive sharp, local and dimension dependent hypercontractive bounds on the Markov kernel of a large class of diffusion semigroups. Unlike the dimension free ones, they capture refined properties of Markov kernels, such as trace estimates. They imply classical bounds on the Ornstein?CUhlenbeck semigroup and a dimensional and refined (transportation) Talagrand inequality when applied to the Hamilton?CJacobi equation. Hypercontractive bounds on the Ornstein?CUhlenbeck semigroup driven by a non-diffusive Lévy semigroup are also investigated. Curvature-dimension criteria are the main tool in the analysis.     

On approximate approximations using Gaussian kernels

This paper discusses quasi-interpolation and interpolation withGaussians. Estimates are obtained showing a high-order approximationup to some saturation error negligible in numerical applications.The construction of local high-order quasi-interpolation formulasis given. Supported in part by the International Centre for MathematicalSciences, Edinburgh.     

Fuzzy clustering using multiple Gaussian kernels with optimized-parameters

In this paper, we propose a new kernel-based fuzzy clustering algorithm which tries to find the best clustering results using optimal parameters of each kernel in each cluster. It is known that data with nonlinear relationships can be separated using one of the kernel-based fuzzy clustering methods. Two common fuzzy clustering approaches are: clustering with a single kernel and clustering with multiple kernels. While clustering with a single kernel doesn’t work well with “multiple-density” clusters, multiple kernel-based fuzzy clustering tries to find an optimal linear weighted combination of kernels with initial fixed (not necessarily the best) parameters. Our algorithm is an extension of the single kernel-based fuzzy c-means and the multiple kernel-based fuzzy clustering algorithms. In this algorithm, there is no need to give “good” parameters of each kernel and no need to give an initial “good” number of kernels. Every cluster will be characterized by a Gaussian kernel with optimal parameters. In order to show its effective clustering performance, we have compared it to other similar clustering algorithms using different databases and different clustering validity measures.     

Cover-based bounds on the numerical rank of Gaussian kernels

A popular approach for analyzing high-dimensional datasets is to perform dimensionality reduction by applying non-parametric affinity kernels. Usually, it is assumed that the represented affinities are related to an underlying low-dimensional manifold from which the data is sampled. This approach works under the assumption that, due to the low-dimensionality of the underlying manifold, the kernel has a low numerical rank. Essentially, this means that the kernel can be represented by a small set of numerically-significant eigenvalues and their corresponding eigenvectors.We present an upper bound for the numerical rank of Gaussian convolution operators, which are commonly used as kernels by spectral manifold-learning methods. The achieved bound is based on the underlying geometry that is provided by the manifold from which the dataset is assumed to be sampled. The bound can be used to determine the number of significant eigenvalues/eigenvectors that are needed for spectral analysis purposes. Furthermore, the results in this paper provide a relation between the underlying geometry of the manifold (or dataset) and the numerical rank of its Gaussian affinities.The term cover-based bound is used because the computations of this bound are done by using a finite set of small constant-volume boxes that cover the underlying manifold (or the dataset). We present bounds for finite Gaussian kernel matrices as well as for the continuous Gaussian convolution operator. We explore and demonstrate the relations between the bounds that are achieved for finite and continuous cases. The cover-oriented methodology is also used to provide a relation between the geodesic length of a curve and the numerical rank of Gaussian kernel of datasets that are sampled from it.     

A universal envelope for Gaussian processes and their kernels

The central theme in our paper deals with mathematical issues involved in the answer to the following question: How can we generate stochastic processes from their correlation data? Since Gaussian processes are determined by moment information up to order two, we focus on the Gaussian case. Two functional analytic tools are used here, in more than one variant. They are: operator factorization; and direct integral decompositions in the form of Karhunen-Loève expansions. We define and study a new interplay between the theory of positive definition functions, and their reproducing kernels, on the one hand, and Gaussian stochastic processes, on the other. The three classes of processes we study are as follows: Processes associated with: (a) arbitrarily given sigma finite regular measures on a fixed Borel measure space; (b) with Hilbert spaces of sigma-functions; and (c) with systems of self-similar measures arising in the theory of iterated function systems. Starting with a non-degenerate positive definite function K on some fixed set S, we show that there is a choice of a universal sample space Ω, which can be realized as a “boundary” of (S,K). Its boundary-theoretic properties are analyzed, and we point out their relevance to the study of electrical networks on countable infinite graphs.     

Gaussian limit for determinantal point processes with J-Hermitian kernels

We show that the central limit theorem for linear statistics over determinantal point processes with J-Hermitian kernels holds under fairly general conditions. In particular, we establish the Gaussian limit for linear statistics over determinantal point processes on the union of two copies of Rdwhen the correlation kernels are J-Hermitian translation-invariant.     

A closer look at covering number bounds for Gaussian kernels

We establish some new bounds on the log-covering numbers of (anisotropic) Gaussian reproducing kernel Hilbert spaces. Unlike previous results in this direction we focus on small explicit constants and their dependency on crucial parameters such as the kernel bandwidth and the size and dimension of the underlying space.     

Approximate maximizers of intricacy functionals

G. Edelman, O. Sporns, and G. Tononi introduced in theoretical biology the neural complexity of a family of random variables. A previous work showed that this functional is a special case of intricacy, i.e., an average of the mutual information of subsystems with specific mathematical properties. Moreover, its maximum value grows at a definite speed with the size of the system. In this work, we compute exactly this speed of growth by building “approximate maximizers” subject to an entropy condition. These approximate maximizers work simultaneously for all intricacies. We also establish some properties of arbitrary approximate maximizers, in particular the existence of a threshold in the size of subsystems of approximate maximizers: most smaller subsystems are almost equidistributed, most larger subsystems determine the full system. The main ideas are a random construction of almost maximizers with a high statistical symmetry and the consideration of entropy profiles, i.e., the average entropies of sub-systems of a given size. The latter gives rise to interesting questions of probability and information theory.     

Absolutely continuous constrained maximizers

The modern theory of differentiable demand functions is generalized to maximizers in a constrained maximization problem. Sufficient conditions are given for almost everywhere differentiability of maximizers in the constraining capacities or other parameters. Also, sufficient conditions are given for the very useful condition (N). Finally, the somewhat stronger pointwise Lipschitz property is shown. Applications analogous to those in economic theory are indicated.     

Quadratic forms which have only large zeros

We give lower bounds for zeros of quadratic forms. For example, givenn2d>0 there are infinitely many nonsingular quadratic formsF with integral coefficients inn variables which vanish on ad-dimensional rational subspace such that anyn linearly independent integral zerosx ₁,...,x _n satisfy |x ₁|...|x _n |F(ⁿ²/^2d)–d, whereF is the maximum modulus of the coefficients ofF. This complements a recent result of the authors in the opposite direction and shows that it is best possible.Written whileSchmidt had a Senior Scientist Award from the Alexander von Humboldt-Stiftung in the Federal Republic of Germany.     

Gauss-Hermite quadratures for functions from Hilbert spaces with Gaussian reproducing kernels

We study univariate integration with the Gaussian weight for a positive variance α. This is done for the reproducing kernel Hilbert space with the Gaussian kernel for a positive shape parameter γ. We study Gauss-Hermite quadratures, although this choice of quadratures may be questionable since polynomials do not belong to this space of functions. Nevertheless, we provide the explicit formula for the error of the Gauss-Hermite quadrature using n function values. In particular, for 2αγ ²<1 we have an exponential rate of convergence, and for 2αγ ²=1 we have no convergence, whereas for 2αγ ²>1 we have an exponential divergence.     

Error estimates for approximate approximations with Gaussian kernels on compact intervals

The aim of this paper is the investigation of the error which results from the method of approximate approximations applied to functions defined on compact intervals, only. This method, which is based on an approximate partition of unity, was introduced by Maz’ya in 1991 and has mainly been used for functions defined on the whole space up to now. For the treatment of differential equations and boundary integral equations, however, an efficient approximation procedure on compact intervals is needed.In the present paper we apply the method of approximate approximations to functions which are defined on compact intervals. In contrast to the whole space case here a truncation error has to be controlled in addition. For the resulting total error pointwise estimates and L₁-estimates are given, where all the constants are determined explicitly.     

Global errors for approximate approximations with Gaussian kernels on compact intervals

This paper investigates the global errors which result when the method of approximate approximations is applied to a function defined on a compact interval. By extending the functions to a wider interval, we are able to introduce modified forms of the quasi-interpolant operators. Using these operators as approximation tools, we estimate upper bounds on the errors in terms of a uniform norm. We consider only continuous and differentiable functions. A similar problem is solved for the two-dimensional case.     

Certain differential equations have only isolated periodic orbits

Summary Some results of Poincaré and Dulac concerning non-isolated periodic orbits and singular cycles in the plane are here extended to certain classes of autonomous analytic ordinary differential equations of higher dimension. The equations in these classes are then shown to have only isolated periodic orbits provided that all their critical points satisfy a simple condition. A further condition at infinity can ensure that the equation has only finitely many periodic orbits.     

On finitep-groups which have only two conjugacy lengths

We show that the nilpotent class of any finite group which has only two conjugacy lengths is at most 3. This corresponds to a result of Isaacs and Passman for degrees of irreducible characters.     

Finite 2-groups in which distinct nonlinear irreducible characters have distinct kernels

In this paper, we study finite 2-groups in which distinct nonlinear irreducible characters have distinct kernels. We prove several results concerning these groups and completely classify 2-groups with at most five nonlinear irreducible characters satisfying this property.     

Gaussian conditional independence relations have no finite complete characterization

We show that there can be no finite list of conditional independence relations which can be used to deduce all conditional independence implications among Gaussian random variables. To do this, we construct, for each n>3 a family of n conditional independence statements on n random variables which together imply that , and such that no subset have this same implication. The proof relies on binomial primary decomposition.     

Portfolio optimization when asset returns have the Gaussian mixture distribution

In this paper we consider a portfolio optimization problem where the underlying asset returns are distributed as a mixture of two multivariate Gaussians; these two Gaussians may be associated with “distressed” and “tranquil” market regimes. In this context, the Sharpe ratio needs to be replaced by other non-linear objective functions which, in the case of many underlying assets, lead to optimization problems which cannot be easily solved with standard techniques. We obtain a geometric characterization of efficient portfolios, which reduces the complexity of the portfolio optimization problem.