Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters

In this paper, we study the multi-parameter Tikhonov regularization method which adds multiple different penalties to exhibit multi-scale features of the solution. An optimal error bound of the regularization solution is obtained by a priori choice of multiple regularization parameters. Some theoretical results of the regularization solution about the dependence on regularization parameters are presented. Then, an a posteriori parameter choice, i.e., the damped Morozov discrepancy principle, is introduced to determine multiple regularization parameters. Five model functions, i.e., two hyperbolic model functions, a linear model function, an exponential model function and a logarithmic model function, are proposed to solve the damped Morozov discrepancy principle. Furthermore, four efficient model function algorithms are developed for finding reasonable multiple regularization parameters, and their convergence properties are also studied. Numerical results of several examples show that the damped discrepancy principle is competitive with the standard one, and the model function algorithms are efficient for choosing regularization parameters.     

Two regularization methods to identify time-dependent heat source through an internal measurement of temperature

This paper deals with an inverse problem for identifying an unknown time-dependent heat source in a one-dimensional heat equation, with the aid of an extra measurement of temperature at an internal point. Since this problem is ill-posed, two regularization solutions are obtained by employing a Fourier truncation regularization and a Quasi-reversibility regularization. Furthermore, the Hölder type stability estimate between the regularization solutions and the exact solution, are obtained, respectively. Numerical examples show that these regularization methods are effective and stable.     

Ill-Posedness of sublinear minimization problems

It is well known that minimization problems involving sublinear regularization terms are ill-posed, in Sobolev spaces. Extended results to spaces of bounded variation functions BV were recently showed in the special case of bounded regularization terms. In this note, a generalization to sublinear regularization is presented in BV spaces. Notice that our results are optimal in the sense that linear regularization leads to well-posed minimization problems in BV spaces.     

On the choice of solution subspace for nonstationary iterated Tikhonov regularization

Tikhonov regularization is a popular method for the solution of linear discrete ill-posed problems with error-contaminated data. Nonstationary iterated Tikhonov regularization is known to be able to determine approximate solutions of higher quality than standard Tikhonov regularization. We investigate the choice of solution subspace in iterative methods for nonstationary iterated Tikhonov regularization of large-scale problems. Generalized Krylov subspaces are compared with Krylov subspaces that are generated by Golub–Kahan bidiagonalization and the Arnoldi process. Numerical examples illustrate the effectiveness of the methods.     

Identification of a time-dependent source term for a time fractional diffusion problem

In this paper, we study an inverse problem of identifying a time-dependent term of an unknown source for a time fractional diffusion equation using nonlocal measurement data. Firstly, we establish the conditional stability for this inverse problem. Then two regularization methods are proposed to for reconstructing the time-dependent source term from noisy measurements. The first method is an integral equation method which formulates the inverse source problem into an integral equation of the second kind; and a prior convergence rate of regularized solutions is derived with a suitable choice strategy of regularization parameters. The second method is a standard Tikhonov regularization method and formulates the inverse source problem as a minimizing problem of the Tikhonov functional. Based on the superposition principle and the technique of finite-element interpolation, a numerical scheme is proposed to implement the second regularization method. One- and two-dimensional examples are carried out to verify efficiency and stability of the second regularization method.     

Two Tikhonov‐type regularization methods for inverse source problem on the Poisson equation

In this paper, we investigate a problem of the identification of an unknown source on Poisson equation from some fixed location. A conditional stability estimate for an inverse heat source problem is proved. We show that such a problem is mildly ill‐posed and further present two Tikhonov‐type regularization methods (a generalized Tikhonov regularization method and a simplified generalized Tikhonov regularization method) to deal with this problem. Convergence estimates are presented under the a priori choice of the regularization parameter. Numerical results are presented to illustrate the accuracy and efficiency of our methods. Copyright © 2012 John Wiley & Sons, Ltd.     

Orthogonal projection regularization operators

Tikhonov regularization often is applied with a finite difference regularization operator that approximates a low-order derivative. This paper proposes the use of orthogonal projections as regularization operators, e.g., with the same null space as commonly used finite difference operators. Applications to iterative and SVD-based methods for Tikhonov regularization are described. Truncated iterative and SVD methods are also considered. Research of L. Reichel was supported in part by an OBR Research Challenge Grant. Research of F. Sgallari was supported in part by PRIN 2004 grant 2004014411-005.     

On the reduction of Tikhonov minimization problems and the construction of regularization matrices

Tikhonov regularization replaces a linear discrete ill-posed problem by a penalized least-squares problem, whose solution is less sensitive to errors in the data and round-off errors introduced during the solution process. The penalty term is defined by a regularization matrix and a regularization parameter. The latter generally has to be determined during the solution process. This requires repeated solution of the penalized least-squares problem. It is therefore attractive to transform the least-squares problem to simpler form before solution. The present paper describes a transformation of the penalized least-squares problem to simpler form that is faster to compute than available transformations in the situation when the regularization matrix has linearly dependent columns and no exploitable structure. Properties of this kind of regularization matrices are discussed and their performance is illustrated.     

Inverse problems for regularization matrices

Discrete ill-posed problems are difficult to solve, because their solution is very sensitive to errors in the data and to round-off errors introduced during the solution process. Tikhonov regularization replaces the given discrete ill-posed problem by a nearby penalized least-squares problem whose solution is less sensitive to perturbations. The penalization term is defined by a regularization matrix, whose choice may affect the quality of the computed solution significantly. We describe several inverse matrix problems whose solution yields regularization matrices adapted to the desired solution. Numerical examples illustrate the performance of the regularization matrices determined.     

A regularization method for solving the Cauchy problem for the Helmholtz equation

In this paper, we investigate a Cauchy problem associated with Helmholtz-type equation in an infinite “strip”. This problem is well known to be severely ill-posed. The optimal error bound for the problem with only nonhomogeneous Neumann data is deduced, which is independent of the selected regularization methods. A framework of a modified Tikhonov regularization in conjunction with the Morozov’s discrepancy principle is proposed, it may be useful to the other linear ill-posed problems and helpful for the other regularization methods. Some sharp error estimates between the exact solutions and their regularization approximation are given. Numerical tests are also provided to show that the modified Tikhonov method works well.     

Regularization methods for accretive variational inequalities over the set of common fixed points of nonexpansive semigroups

In this paper, we introduce a regularization method based on the Browder–Tikhonov regularization method for solving a class of accretive variational inequalities over the set of common fixed points of a nonexpansive semigroup on a uniformly smooth Banach space. Three algorithms based on this regularization method are given and their strong convergence is studied. Finally, a finite-dimensional example is developed to illustrate the numerical behaviour of the algorithms.     

Lagrange multipliers, (exact) regularization and error bounds for monotone variational inequalities

We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stopping criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one.     

Two regularization methods for inverse source problem on the Poisson equation

This paper is devoted to discuss an inverse problem of determining an unknown source on the Poisson equation. This is a mildly ill-posed problem. Two regularization methods, one based on the mollification of the data and the other based on the modification of the ‘kernel’ of the solution, are proposed to solve this problem. The convergence estimates between the exact solution and the regularization solution are presented using a priori regularization parameter choice rule. Numerical results are presented to illustrate the accuracy and efficiency of the proposed methods.     

GCV for Tikhonov regularization via global Golub–Kahan decomposition

Generalized cross validation is a popular approach to determining the regularization parameter in Tikhonov regularization. The regularization parameter is chosen by minimizing an expression, which is easy to evaluate for small‐scale problems, but prohibitively expensive to compute for large‐scale ones. This paper describes a novel method, based on Gauss‐type quadrature, for determining upper and lower bounds for the desired expression. These bounds are used to determine the regularization parameter for large‐scale problems. Computed examples illustrate the performance of the proposed method and demonstrate its competitiveness. Copyright © 2016 John Wiley & Sons, Ltd.     

Distributed learning with multi-penalty regularization

In this paper, we study distributed learning with multi-penalty regularization based on a divide-and-conquer approach. Using Neumann expansion and a second order decomposition on difference of operator inverses approach, we derive optimal learning rates for distributed multi-penalty regularization in expectation. As a byproduct, we also deduce optimal learning rates for multi-penalty regularization, which was not given in the literature. These results are applied to the distributed manifold regularization and optimal learning rates are given.     

A new Tikhonov regularization method

The numerical solution of linear discrete ill-posed problems typically requires regularization, i.e., replacement of the available ill-conditioned problem by a nearby better conditioned one. The most popular regularization methods for problems of small to moderate size, which allow evaluation of the singular value decomposition of the matrix defining the problem, are the truncated singular value decomposition and Tikhonov regularization. The present paper proposes a novel choice of regularization matrix for Tikhonov regularization that bridges the gap between Tikhonov regularization and truncated singular value decomposition. Computed examples illustrate the benefit of the proposed method.     

Iterated Tikhonov regularization with a general penalty term

Tikhonov regularization is one of the most popular approaches to solving linear discrete ill‐posed problems. The choice of the regularization matrix may significantly affect the quality of the computed solution. When the regularization matrix is the identity, iterated Tikhonov regularization can yield computed approximate solutions of higher quality than (standard) Tikhonov regularization. This paper provides an analysis of iterated Tikhonov regularization with a regularization matrix different from the identity. Computed examples illustrate the performance of this method.     

Regularization parameter selection and an efficient algorithm for total variation-regularized positron emission tomography

In positron emission tomography, image data corresponds to measurements of emitted photons from a radioactive tracer in the subject. Such count data is typically modeled using a Poisson random variable, leading to the use of the negative-log Poisson likelihood fit-to-data function. Regularization is needed, however, in order to guarantee reconstructions with minimal artifacts. Given that tracer densities are primarily smoothly varying, but also contain sharp jumps (or edges), total variation regularization is a natural choice. However, the resulting computational problem is quite challenging. In this paper, we present an efficient computational method for this problem. Convergence of the method has been shown for quadratic regularization functions and here convergence is shown for total variation regularization. We also present three regularization parameter choice methods for use on total variation-regularized negative-log Poisson likelihood problems. We test the computational and regularization parameter selection methods on two synthetic data sets.     

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current-voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.     

Regularized numerical integration of multibody dynamics with the generalized method

This paper discusses the consistent regularization property of the generalized α method when applied as an integrator to an initial value high index and singular differential-algebraic equation model of a multibody system. The regularization comes from within the discretization itself and the discretization remains consistent over the range of values the regularization parameter may take. The regularization involves increase of the smallest singular values of the ill-conditioned Jacobian of the discretization and is different from Baumgarte and similar techniques which tend to be inconsistent for poor choice of regularization parameter. This regularization also helps where pre-conditioning the Jacobian by scaling is of limited effect, for example, when the scleronomic constraints contain multiple closed loops or singular configuration or when high index path constraints are present. The feed-forward control in Kane’s equation models is additionally considered in the numerical examples to illustrate the effect of regularization. The discretization presented in this work is adopted to the first order DAE system (unlike the original method which is intended for second order systems) for its A-stability and same order of accuracy for positions and velocities.