Learning Rates of Least-Square Regularized Regression

This paper considers the regularized learning algorithm associated with the least-square loss and reproducing kernel Hilbert spaces. The target is the error analysis for the regression problem in learning theory. A novel regularization approach is presented, which yields satisfactory learning rates. The rates depend on the approximation property and on the capacity of the reproducing kernel Hilbert space measured by covering numbers. When the kernel is C^∞ and the regression function lies in the corresponding reproducing kernel Hilbert space, the rate is m^ζ with ζ arbitrarily close to 1, regardless of the variance of the bounded probability distribution.     

Nonsmooth analysis and Hamilton–Jacobi equations on Riemannian manifolds

We establish some perturbed minimization principles, and we develop a theory of subdifferential calculus, for functions defined on Riemannian manifolds. Then we apply these results to show existence and uniqueness of viscosity solutions to Hamilton–Jacobi equations defined on Riemannian manifolds.     

Proximal subgradient and a characterization of Lipschitz function on Riemannian manifolds

A characterization of Lipschitz behavior of functions defined on Riemannian manifolds is given in this paper. First, it is extended the concept of proximal subgradient and some results of proximal analysis from Hilbert space to Riemannian manifold setting. A technique introduced by Clarke, Stern and Wolenski [F.H. Clarke, R.J. Stern, P.R. Wolenski, Subgradient criteria for monotonicity, the Lipschitz condition, and convexity, Canad. J. Math. 45 (1993) 1167-1183], for generating proximal subgradients of functions defined on a Hilbert spaces, is also extended to Riemannian manifolds in order to provide that characterization. A number of examples of Lipschitz functions are presented so as to show that the Lipschitz behavior of functions defined on Riemannian manifolds depends on the Riemannian metric.     

Riesz transform on locally symmetric spaces and Riemannian manifolds with a spectral gap

In this paper we study the Riesz transform on complete and connected Riemannian manifolds M with a certain spectral gap in the L² spectrum of the Laplacian. We show that on such manifolds the Riesz transform is Lp bounded for all p∈(1,∞). This generalizes a result by Mandouvalos and Marias and extends a result by Auscher, Coulhon, Duong, and Hofmann to the case where zero is an isolated point of the L² spectrum of the Laplacian.     

Invex sets and preinvex functions on Riemannian manifolds

The concept of a geodesic invex subset of a Riemannian manifold is introduced. Geodesic invex and preinvex functions on a geodesic invex set with respect to particular maps are defined. The relation between geodesic invexity and preinvexity of functions on manifolds is studied. Using proximal subdifferential, certain results concerning extremum points of a non smooth geodesic preinvex function on a geodesic invex set are obtained. The main value inequality and the mean value theorem in invexity analysis are extended to Cartan-Hadamard manifolds.     

Gradient learning in a classification setting by gradient descent

Learning gradients is one approach for variable selection and feature covariation estimation when dealing with large data of many variables or coordinates. In a classification setting involving a convex loss function, a possible algorithm for gradient learning is implemented by solving convex quadratic programming optimization problems induced by regularization schemes in reproducing kernel Hilbert spaces. The complexity for such an algorithm might be very high when the number of variables or samples is huge. We introduce a gradient descent algorithm for gradient learning in classification. The implementation of this algorithm is simple and its convergence is elegantly studied. Explicit learning rates are presented in terms of the regularization parameter and the step size. Deep analysis for approximation by reproducing kernel Hilbert spaces under some mild conditions on the probability measure for sampling allows us to deal with a general class of convex loss functions.     

Dini derivative and a characterization for Lipschitz and convex functions on Riemannian manifolds

Dini derivatives in Riemannian manifold settings are studied in this paper. In addition, a characterization for Lipschitz and convex functions defined on Riemannian manifolds and sufficient optimality conditions for constraint optimization problems in terms of the Dini derivative are given.     

Learning gradients by a gradient descent algorithm

We propose a stochastic gradient descent algorithm for learning the gradient of a regression function from random samples of function values. This is a learning algorithm involving Mercer kernels. By a detailed analysis in reproducing kernel Hilbert spaces, we provide some error bounds to show that the gradient estimated by the algorithm converges to the true gradient, under some natural conditions on the regression function and suitable choices of the step size and regularization parameters.     

Derivative reproducing properties for kernel methods in learning theory

The regularity of functions from reproducing kernel Hilbert spaces (RKHSs) is studied in the setting of learning theory. We provide a reproducing property for partial derivatives up to order s when the Mercer kernel is C^2s. For such a kernel on a general domain we show that the RKHS can be embedded into the function space C^s. These observations yield a representer theorem for regularized learning algorithms involving data for function values and gradients. Examples of Hermite learning and semi-supervised learning penalized by gradients on data are considered.     

Variational Principles and Sobolev-Type Estimates for Generalized Interpolation on a Riemannian Manifold

The purpose of this paper is to study certain variational principles and Sobolev-type estimates for the approximation order resulting from using strictly positive definite kernels to do generalized Hermite interpolation on a closed (i.e., no boundary), compact, connected, orientable, m -dimensional C ^∞ Riemannian manifold , with C ^∞ metric g _ij . The rate of approximation can be more fully analyzed with rates of approximation given in terms of Sobolev norms. Estimates on the rate of convergence for generalized Hermite and other distributional interpolants can be obtained in certain circumstances and, finally, the constants appearing in the approximation order inequalities are explicit. Our focus in this paper will be on approximation rates in the cases of the circle, other tori, and the 2 -sphere. April 10, 1996. Dates revised: March 26, 1997; August 26, 1997. Date accepted: September 12, 1997. Communicated by Ronald A. DeVore.     

Forward and inverse scattering on manifolds with asymptotically cylindrical ends

We study an inverse problem for a non-compact Riemannian manifold whose ends have the following properties: On each end, the Riemannian metric is assumed to be a short-range perturbation of the metric of the form ²(dy)+h(x,dx), h(x,dx) being the metric of some compact manifold of codimension 1. Moreover one end is exactly cylindrical, i.e. the metric is equal to ²(dy)+h(x,dx). Given two such manifolds having the same scattering matrix on that exactly cylindrical end for all energies, we show that these two manifolds are isometric.     

Weighted norm inequalities,off-diagonal estimates and elliptic operators. Part IV: Riesz transforms on manifolds and weights

This is the fourth article of our series. Here, we study weighted norm inequalities for the Riesz transform of the Laplace–Beltrami operator on Riemannian manifolds and of subelliptic sum of squares on Lie groups, under the doubling volume property and Gaussian upper bounds.      

On approximation by spherical reproducing kernel Hilbert spaces

The spherical approximation between two nested reproducing kernels Hilbert spaces generated from different smooth kernels is investigated. It is shown that the functions of a space can be approximated by that of the subspace with better smoothness. Furthermore, the upper bound of approximation error is given.     

Spherical basis functions and uniform distribution of points on spheres

The main purpose of the present paper is to employ spherical basis functions (SBFs) to study uniform distribution of points on spheres. We extend Weyl's criterion for uniform distribution of points on spheres to include a characterization in terms of an SBF. We show that every set of minimal energy points associated with an SBF is uniformly distributed on the spheres. We give an error estimate for numerical integration based on the minimal energy points. We also estimate the separation of the minimal energy points.     

Analytical and numerical inversion formulas in the Gaussian convolution by using the Paley–Wiener spaces

We shall discuss the relations among sampling theory (Sinc method), reproducing kernels and the Tikhonov regularization. Here, we see the important difference of the Sobolev Hilbert spaces and the Paley–Wiener spaces when we use their reproducing kernel Hibert spaces as approximate spaces in the Tikhonov regularization. Further, by using the Paley–Wiener spaces, we shall illustrate numerical experiments for new inversion formulas for the Gaussian convolution as a much more powerful and improved method by using computers. In this article, we shall be able to give practical numerical and analytical inversion formulas for the Gaussian convolution that is realized by computers.     

Topological,affine and isometric actions on flat riemannian manifolds II

Explicit examples of finite subgroups of the group of homotopy classes of self-homotopy equivalences of some flat Riemannian manifolds which cannot be lifted to effective actions are given. It is also shown that no finite subgroups of the kernel of π₀(Homeo(M))→Out π₁(M) can be lifted back to Homeo(M), for a large class of flat manifolds M. Some results of an earlier paper by the authors are refined and related to recent work of R. Schoen and S.T. Yau.