Multi-penalty regularization in learning theory

In this paper we establish the error estimates for multi-penalty regularization under the general smoothness assumption in the context of learning theory. One of the motivation for this work is to study the convergence analysis of two-parameter regularization theoretically in the manifold learning setting. In this spirit, we obtain the error bounds for the manifold learning problem using more general framework of multi-penalty regularization. We propose a new parameter choice rule “the balanced-discrepancy principle” and analyze the convergence of the scheme with the help of estimated error bounds. We show that multi-penalty regularization with the proposed parameter choice exhibits the convergence rates similar to single-penalty regularization. Finally on a series of test samples we demonstrate the superiority of multi-parameter regularization over single-penalty regularization.     

Robust error estimates for the finite element approximation of elliptic optimal control problems

In this paper, we consider the finite element approximation of an elliptic optimal control problem. Based on an assumption on the adjoint state of the continuous problem with a small parameter, which represents a regularization of the bang–bang type control problem, we derive robust a priori error estimates for optimal control and state and a posteriori error estimate is also presented. Numerical experiments confirm our theoretical results.     

The regularization method for an obstacle problem

Summary. We give a relatively complete analysis for the regularization method, which is usually used in solving non-differentiable minimization problems. The model problem considered in the paper is an obstacle problem. In addition to the usual convergence result and a-priori error estimates, we provide a-posteriori error estimates which are highly desired for practical implementation of the regularization method. Received March 22, 1993 / Revised version received October 11, 1993     

二维区域上椭圆均匀化问题的收敛率（英文）

In this paper, we study the convergence rates of solutions for second order elliptic equations with rapidly oscillating periodic coefficients in two-dimensional domain. We use an extension of the "mixed formulation" approach to obtain the representation formula satisfied by the oscillatory solution and homogenized solution by means of the particularity of solutions for equations in two-dimensional case. Then we utilize this formula in combination with the asymptotic estimates of Green or Neumann functions for operators and uniform regularity estimates of solutions to obtain convergence rates in L~p for solutions as well as gradient error estimates for Dirichlet or Neumann problems respectively.     

Identification of an inverse source problem for time‐fractional diffusion equation with random noise

In this paper, we consider an inverse source problem for a time fractional diffusion equation. In general, this problem is ill posed, therefore we shall construct a regularized solution using the filter regularization method in the random noise case. We will provide appropriate conditions to guarantee the convergence of the approximate solution to the exact solution. Then, we provide examples of filters in order to obtain error estimates for their approximate solutions. Finally, we present a numerical example to show efficiency of the method.     

An iterative Bregman regularization method for optimal control problems with inequality constraints

We study an iterative regularization method of optimal control problems with control constraints. The regularization method is based on generalized Bregman distances. We provide convergence results under a combination of a source condition and a regularity condition on the active sets. We do not assume attainability of the desired state. Furthermore, a priori regularization error estimates are obtained.     

A Modified Tikhonov Regularization for Linear Operator Equations

We construct with the aid of regularizing filters a new class of improved regularization methods, called modified Tikhonov regularization (MTR), for solving ill-posed linear operator equations. Regularizing properties and asymptotic order of the regularized solutions are analyzed in the presence of noisy data and perturbation error in the operator. With some accurate estimates in the solution errors, optimal convergence order of the regularized solutions is obtained by a priori choice of the regularization parameter. Furthermore, numerical results are given for several ill-posed integral equations, which not only roughly coincide with the theoretical results but also show that MTR can be more accurate than ordinary Tikhonov regularization (OTR).     

Convergence analysis of a regularized approximation for solving Fredholm integral equations of the first kind

In this paper, we suggest a convergence analysis for solving Fredholm integral equations of the first kind using Tikhonov regularization under supremum norm. We also provide an a priori parameter choice strategy for choosing the regularization parameter and obtain an error estimate.     

On Regularization Methods for Inverse Problems of Dynamic Type

In this paper, we consider new regularization methods for linear inverse problems of dynamic type. These methods are based on dynamic programming techniques for linear quadratic optimal control problems. Two different approaches are followed: a continuous and a discrete one. We prove regularization properties and also obtain rates of convergence for the methods derived from both approaches. A numerical example concerning the dynamic EIT problem is used to illustrate the theoretical results.     

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

We revisit the gradient projection method in the framework of nonlinear optimal control problems with bang–bang solutions. We obtain the strong convergence of the iterative sequence of controls and the corresponding trajectories. Moreover, we establish a convergence rate, depending on a constant appearing in the corresponding switching function and prove that this convergence rate estimate is sharp. Some numerical illustrations are reported confirming the theoretical results.     

Error estimates for the finite volume discretization for the porous medium equation

We analyze the convergence of a numerical scheme for a class of degenerate parabolic problems modelling reactions in porous media, and involving a nonlinear, possibly vanishing diffusion. The scheme involves the Kirchhoff transformation of the regularized nonlinearity, as well as an Euler implicit time stepping and triangle based finite volumes. We prove the convergence of the approach by giving error estimates in terms of the discretization and regularization parameter.     

On the convergence of a regularizing Levenberg–Marquardt scheme for nonlinear ill-posed problems

In this note, we study the convergence of the Levenberg–Marquardt regularization scheme for nonlinear ill-posed problems. We consider the case that the initial error satisfies a source condition. Our main result shows that if the regularization parameter does not grow too fast (not faster than a geometric sequence), then the scheme converges with optimal convergence rates. Our analysis is based on our recent work on the convergence of the exponential Euler regularization scheme (Hochbruck et al. in Inverse Probl 25(7):075009, 2009).     

On regularization methods based on dynamic programming techniques

In this article, we investigate the connection between regularization theory for inverse problems and dynamic programming theory. This is done by developing two new regularization methods, based on dynamic programming techniques. The aim of these methods is to obtain stable approximations to the solution of linear inverse ill-posed problems. We follow two different approaches and derive a continuous and a discrete regularization method. Regularization properties for both methods are proved as well as rates of convergence. A numerical benchmark problem concerning integral operators with convolution kernels is used to illustrate the theoretical results.     

Online Gradient Descent Learning Algorithms

This paper considers the least-square online gradient descent algorithm in a reproducing kernel Hilbert space (RKHS) without an explicit regularization term. We present a novel capacity independent approach to derive error bounds and convergence results for this algorithm. The essential element in our analysis is the interplay between the generalization error and a weighted cumulative error which we define in the paper. We show that, although the algorithm does not involve an explicit RKHS regularization term, choosing the step sizes appropriately can yield competitive error rates with those in the literature.     

反向热传导问题的一种新的正则化方法

讨论一个高维反向热传导问题,这是一个经典的严重不适定问题.关于这一问题我们给出一种新的正则化方法-改进的Tikhonov正则化方法,以恢复解对数据的连续依赖性.通过构造一个重要的不等式和提高先验光滑条件,获得正则解在0相似文献   

Solution smoothness of ill-posed equations in Hilbert spaces: four concepts and their cross connections

Numerical solution of ill-posed operator equations requires regularization techniques. The convergence of regularized solutions to the exact solution can be usually guaranteed, but to also obtain estimates for the speed of convergence one has to exploit some kind of smoothness of the exact solution. We consider four such smoothness concepts in a Hilbert space setting: source conditions, approximate source conditions, variational inequalities, and approximate variational inequalities. Besides some new auxiliary results on variational inequalities the equivalence of the last three concepts is shown. In addition, it turns out that the classical concept of source conditions and the modern concept of variational inequalities are connected via Fenchel duality.     

Error estimation of approximate solutions to one inverse problem for a parabolic equation

This paper is devoted to solving the inverse boundary problem of the heat diagnostics by the projective regularization method. We obtain exact with respect to the order error estimates of the corresponding approximate solution.     

关于一类广义Tikhonov 正则化方法的饱和效应分析

Tikhonov正则化方法是研究不适定问题最重要的正则化方法之一,但由于这种方法的饱和效应出现的太早,使得无法随着对解的光滑性假设的提高而提高正则逼近解的收敛率,也即对高的光滑性假设,正则解与准确解的误差估计不可能达到阶数最优.Schrroter T 和Tautenhahn U给出了一类广义Tikhonov正则化方法并重点讨论了它的最优误差估计, 但却未能对该方法的饱和效应进行研究.本文对此进行了仔细分析,并发现此方法可以防止饱和效应,而且数值试验结果表明此方法计算效果良好.     

A Sequential Least Squares Method for Elliptic Equations in Non-Divergence Form

We develop a new least squares method for solving the second-order elliptic equations in non-divergence form. Two least-squares-type functionals are proposed for solving the equation in two sequential steps. We first obtain a numerical approximation to the gradient in a piecewise irrotational polynomial space. Then together with the numerical gradient, we seek a numerical solution of the primitive variable in the continuous Lagrange finite element space. The variational setting naturally provides an a posteriori error which can be used in an adaptive refinement algorithm. The error estimates under the $L^2$ norm and the energy norm for both two unknowns are derived. By a series of numerical experiments, we verify the convergence rates and show the efficiency of the adaptive algorithm.     

Multi-kernel regularized classifiers

A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve multi-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory estimates for the excess misclassification error of these multi-kernel regularized classifiers when the loss functions achieve the zero value. The error analysis consists of two parts: regularization error and sample error. Allowing multi-kernels in the algorithm improves the regularization error and approximation error, which is one advantage of the multi-kernel setting. For a general loss function, we show how to bound the regularization error by the approximation in some weighted L^q$L^q$ spaces. For the sample error, we use a projection operator. The projection in connection with the decay of the regularization error enables us to improve convergence rates in the literature even for the one-kernel schemes and special loss functions: least-square loss and hinge loss for support vector machine soft margin classifiers. Existence of the optimization problem for the regularization scheme associated with multi-kernels is verified when the kernel functions are continuous with respect to the index set. Concrete examples, including Gaussian kernels with flexible variances and probability distributions with some noise conditions, are used to illustrate the general theory.