Distributed learning with multi-penalty regularization

In this paper, we study distributed learning with multi-penalty regularization based on a divide-and-conquer approach. Using Neumann expansion and a second order decomposition on difference of operator inverses approach, we derive optimal learning rates for distributed multi-penalty regularization in expectation. As a byproduct, we also deduce optimal learning rates for multi-penalty regularization, which was not given in the literature. These results are applied to the distributed manifold regularization and optimal learning rates are given.     

The optimal solution of multi-kernel regularization learning

In regularized kernel methods, the solution of a learning problem is found by minimizing a functional consisting of a empirical risk and a regularization term. In this paper, we study the existence of optimal solution of multi-kernel regularization learning. First, we ameliorate a previous conclusion about this problem given by Micchelli and Pontil, and prove that the optimal solution exists whenever the kernel set is a compact set. Second, we consider this problem for Gaussian kernels with variance σ∈(0,∞), and give some conditions under which the optimal solution exists.     

An iteration regularization for a time-fractional inverse diffusion problem

In the present paper we consider a time-fractional inverse diffusion problem, where data is given at x = 1 and the solution is required in the interval 0 < x < 1. This problem is typically ill-posed: the solution (if it exists) does not depend continuously on the data. We give a new iteration regularization method to deal with this problem, and error estimates are obtained for a priori and a posteriori parameter choice rules, respectively. Furthermore, numerical implement shows the proposed method works effectively.     

Elastic-net regularization in learning theory

Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie [H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67(2) (2005) 301–320] for the selection of groups of correlated variables. To investigate the statistical properties of this scheme and in particular its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combinations of elements (features) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular “elastic-net representation” of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed in the above-cited work.     

Fourier regularization for a backward heat equation

In this paper a simple and convenient new regularization method for solving backward heat equation—Fourier regularization method is given. Meanwhile, some quite sharp error estimates between the approximate solution and exact solution are provided. A numerical example also shows that the method works effectively.     

A regularization method for the cauchy problem of the modified Helmholtz equation

In the present paper, an iteration regularization method for solving the Cauchy problem of the modified Helmholtz equation is proposed. The a priori and a posteriori rule for choosing regularization parameters with corresponding error estimates between the exact solution and its approximation are also given. The numerical example shows the effectiveness of this method. Copyright © 2014 John Wiley & Sons, Ltd.     

Behavior of a functional in learning theory

Let H be a Hilbert space, A ∈ L(H), y ∈ , and y ∉ R(A). We study the behavior of the distance square between y and A(B _τ), defined as a functional F(τ), as the radius τ of the ball B _τ of H tends to ∞. This problem is important in estimating the approximation error in learning theory. Our main result is to estimate the asymptotic behavior of F(τ) without the compactness assumption on the operator A. We also consider the Peetre K-functional and its convergence rates.      

Derivative reproducing properties for kernel methods in learning theory

The regularity of functions from reproducing kernel Hilbert spaces (RKHSs) is studied in the setting of learning theory. We provide a reproducing property for partial derivatives up to order s when the Mercer kernel is C^2s. For such a kernel on a general domain we show that the RKHS can be embedded into the function space C^s. These observations yield a representer theorem for regularized learning algorithms involving data for function values and gradients. Examples of Hermite learning and semi-supervised learning penalized by gradients on data are considered.     

Numerical differentiation from a viewpoint of regularization theory

In this paper, we discuss the classical ill-posed problem of numerical differentiation, assuming that the smoothness of the function to be differentiated is unknown. Using recent results on adaptive regularization of general ill-posed problems, we propose new rules for the choice of the stepsize in the finite-difference methods, and for the regularization parameter choice in numerical differentiation regularized by the iterated Tikhonov method. These methods are shown to be effective for the differentiation of noisy functions, and the order-optimal convergence results for them are proved.

     

Local gauge-invariant infrared regularization of Yang-Mills theory

We construct a local gauge invariant-infrared regularization for Yang-Mills theory based on a higher-derivative formulation of the model. __________ Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 154, No. 2, pp. 213–219, February, 2008.     

A simplified Tikhonov regularization method for determining the heat source

This paper is to discuss the inverse problem of determining a spacewise dependent heat source in one-dimensional heat equation in a bounded domain where data is given at some fixed time. This problem is ill-posed, i.e., the solution (if it exists) does not depend continuously on the data. The regularization solution is given by a simplified Tikhonov regularization. For this regularization solution, the Hölder type stability estimate between the regularization solution and the exact solution is obtained. Numerical examples show that the regularization method is effective and stable.     

Regularization properties of the sequential discrepancy principle for Tikhonov regularization in Banach spaces

The stable solution of ill-posed non-linear operator equations in Banach space requires regularization. One important approach is based on Tikhonov regularization, in which case a one-parameter family of regularized solutions is obtained. It is crucial to choose the parameter appropriately. Here, a sequential variant of the discrepancy principle is analysed. In many cases, such parameter choice exhibits the feature, called regularization property below, that the chosen parameter tends to zero as the noise tends to zero, but slower than the noise level. Here, we shall show such regularization property under two natural assumptions. First, exact penalization must be excluded, and secondly, the discrepancy principle must stop after a finite number of iterations. We conclude this study with a discussion of some consequences for convergence rates obtained by the discrepancy principle under the validity of some kind of variational inequality, a recent tool for the analysis of inverse problems.     

Bernstein polynomials and learning theory

When learning processes depend on samples but not on the order of the information in the sample, then the Bernoulli distribution is relevant and Bernstein polynomials enter into the analysis. We derive estimates of the approximation of the entropy function x log x that are sharper than the bounds from Voronovskaja's theorem. In this way we get the correct asymptotics for the Kullback–Leibler distance for an encoding problem.     

First and second order numerical differentiation with Tikhonov regularization

This work deals with the numerical differentiation for an unknown smooth function whose data on a given set are available. The numerical differentiation is an ill-posed problem. In this work, the first and second derivatives of the smooth function are approximated by using the Tikhonov regularization method. It is proved that the approximate function can be chosen as a minimizer to a cost functional. The existence and uniqueness theory of the minimizer is established. Errors in the derivatives between the smooth unknown function and the approximate function are obtained, which depend on the mesh size of the grid and the noise level in the data. The numerical results are provided to support the theoretical analysis of this work. Selected from Numerical Mathematics (A Journal of Chinese Universities), 2004, 26(1):62–74     

Tikhonov regularization method for a backward problem for the time-fractional diffusion equation

This paper is devoted to solve a backward problem for a time-fractional diffusion equation with variable coefficients in a general bounded domain by the Tikhonov regularization method. Based on the eigenfunction expansion of the solution, the backward problem for searching the initial data is changed to solve a Fredholm integral equation of the first kind. The conditional stability for the backward problem is obtained. We use the Tikhonov regularization method to deal with the integral equation and obtain the series expression of solution. Furthermore, the convergence rates for the Tikhonov regularized solution can be proved by using an a priori regularization parameter choice rule and an a posteriori regularization parameter choice rule. Two numerical examples in one-dimensional and two-dimensional cases respectively are investigated. Numerical results show that the proposed method is effective and stable.     

A modified Tikhonov regularization method for an axisymmetric backward heat equation

This paper deals with the inverse time problem for an axisymmetric heat equation. The problem is ill-posed. A modified Tikhonov regularization method is applied to formulate regularized solution which is stably convergent to the exact one. estimate between the approximate solution and exact technical inequality and improving a priori smoothness Meanwhile, a logarithmic-HSlder type error solution is obtained by introducing a rather assumption.     

A Tikhonov-type method for solving a multidimensional inverse heat source problem in an unbounded domain

In this study we prove a stability estimate for an inverse heat source problem in the n-dimensional case. We present a revised generalized Tikhonov regularization and obtain an error estimate. Numerical experiments for the one-dimensional and two-dimensional cases show that the revised generalized Tikhonov regularization works well.     

A regularization method for solving the radially symmetric backward heat conduction problem

This work is devoted to solving the radially symmetric backward heat conduction problem, starting from the final temperature distribution. The problem is ill-posed: the solution (if it exists) does not depend continuously on the given data. A modified Tikhonov regularization method is proposed for solving this inverse problem. A quite sharp estimate of the error between the approximate solution and the exact solution is obtained with a suitable choice of regularization parameter. A numerical example is presented to verify the efficiency and accuracy of the method.