GCV for Tikhonov regularization via global Golub–Kahan decomposition

Generalized cross validation is a popular approach to determining the regularization parameter in Tikhonov regularization. The regularization parameter is chosen by minimizing an expression, which is easy to evaluate for small‐scale problems, but prohibitively expensive to compute for large‐scale ones. This paper describes a novel method, based on Gauss‐type quadrature, for determining upper and lower bounds for the desired expression. These bounds are used to determine the regularization parameter for large‐scale problems. Computed examples illustrate the performance of the proposed method and demonstrate its competitiveness. Copyright © 2016 John Wiley & Sons, Ltd.     

Regularized numerical integration of multibody dynamics with the generalized method

This paper discusses the consistent regularization property of the generalized α method when applied as an integrator to an initial value high index and singular differential-algebraic equation model of a multibody system. The regularization comes from within the discretization itself and the discretization remains consistent over the range of values the regularization parameter may take. The regularization involves increase of the smallest singular values of the ill-conditioned Jacobian of the discretization and is different from Baumgarte and similar techniques which tend to be inconsistent for poor choice of regularization parameter. This regularization also helps where pre-conditioning the Jacobian by scaling is of limited effect, for example, when the scleronomic constraints contain multiple closed loops or singular configuration or when high index path constraints are present. The feed-forward control in Kane’s equation models is additionally considered in the numerical examples to illustrate the effect of regularization. The discretization presented in this work is adopted to the first order DAE system (unlike the original method which is intended for second order systems) for its A-stability and same order of accuracy for positions and velocities.     

Estimation of the L-Curve via Lanczos Bidiagonalization

The L-curve criterion is often applied to determine a suitable value of the regularization parameter when solving ill-conditioned linear systems of equations with a right-hand side contaminated by errors of unknown norm. However, the computation of the L-curve is quite costly for large problems; the determination of a point on the L-curve requires that both the norm of the regularized approximate solution and the norm of the corresponding residual vector be available. Therefore, usually only a few points on the L-curve are computed and these values, rather than the L-curve, are used to determine a value of the regularization parameter. We propose a new approach to determine a value of the regularization parameter based on computing an L-ribbon that contains the L-curve in its interior. An L-ribbon can be computed fairly inexpensively by partial Lanczos bidiagonalization of the matrix of the given linear system of equations. A suitable value of the regularization parameter is then determined from the L-ribbon, and we show that an associated approximate solution of the linear system can be computed with little additional work.     

求解病态问题的一种改进的Tikhonov正则化--(1)正则化方法的建立

对于带有右端扰动数据的第一类紧算子方程的病态问题 ,本文应用正则化子建立了一类新的正则化求解方法 ,称之为改进的Tikonov正则化 ;通过适当选取正则参数 ,证明了正则解具有最优的渐近收敛阶 .与通常的Tikhonov正则化相比 ,这种改进的正则化可使正则解取到足够高的最优渐近阶     

Regularization of backward heat conduction problem

We study the backward heat conduction problem in an unbounded region. The problem is ill-posed, in the sense that the solution if it exists, does not depend continuously on the data. Continuous dependence of the data is restored by cutting-off high frequencies in Fourier domain. The cut-off parameter acts as a regularization parameter. The discrepancy principle, for choosing the regularization parameter and double exponential transformation methods for numerical implementation of regularization method have been used. An example is presented to illustrate applicability and accuracy of the proposed method.     

Regularized collocation method for Fredholm integral equations of the first kind

In this paper we consider a collocation method for solving Fredholm integral equations of the first kind, which is known to be an ill-posed problem. An “unregularized” use of this method can give reliable results in the case when the rate at which smallest singular values of the collocation matrices decrease is known a priori. In this case the number of collocation points plays the role of a regularization parameter. If the a priori information mentioned above is not available, then a combination of collocation with Tikhonov regularization can be the method of choice. We analyze such regularized collocation in a rather general setting, when a solution smoothness is given as a source condition with an operator monotone index function. This setting covers all types of smoothness studied so far in the theory of Tikhonov regularization. One more issue discussed in this paper is an a posteriori choice of the regularization parameter, which allows us to reach an optimal order of accuracy for deterministic noise model without any knowledge of solution smoothness.     

Lavrentiev's regularization method for nonlinear ill-posed equations in Banach spaces

In this paper, we deal with nonlinear ill-posed problems involving m-accretive mappings in Banach spaces. We consider a derivative and inverse free method for the implementation of Lavrentiev regularization method. Using general H¨older type source condition we obtain an optimal order error estimate. Also we consider the adaptive parameter choice strategy proposed by Pereverzev and Schock(2005) for choosing the regularization parameter.     

Image denoising via solution paths

In this paper, we propose a two-step kernel learning method based on the support vector regression (SVR) for financial time series forecasting. Given a number of candidate kernels, our method learns a sparse linear combination of these kernels so that the resulting kernel can be used to predict well on future data. The L ₁-norm regularization approach is used to achieve kernel learning. Since the regularization parameter must be carefully selected, to facilitate parameter tuning, we develop an efficient solution path algorithm that solves the optimal solutions for all possible values of the regularization parameter. Our kernel learning method has been applied to forecast the S&P500 and the NASDAQ market indices and showed promising results.     

Financial market forecasting using a two-step kernel learning method for the support vector regression

In this paper, we propose a two-step kernel learning method based on the support vector regression (SVR) for financial time series forecasting. Given a number of candidate kernels, our method learns a sparse linear combination of these kernels so that the resulting kernel can be used to predict well on future data. The L ₁-norm regularization approach is used to achieve kernel learning. Since the regularization parameter must be carefully selected, to facilitate parameter tuning, we develop an efficient solution path algorithm that solves the optimal solutions for all possible values of the regularization parameter. Our kernel learning method has been applied to forecast the S&P500 and the NASDAQ market indices and showed promising results.     

High-order total variation-based Poissonian image deconvolution with spatially adapted regularization parameter

Images captured by image acquisition systems using photon-counting devices such as astronomical imaging, positron emission tomography and confocal microscopy imaging, are often contaminated by Poisson noise. Total variation (TV) regularization, which is a classic regularization technique in image restoration, is well-known for recovering sharp edges of an image. Since the regularization parameter is important for a good recovery, Chen and Cheng (2012) proposed an effective TV-based Poissonian image deblurring model with a spatially adapted regularization parameter. However, it has drawbacks since the TV regularization produces staircase artifacts. In this paper, in order to remedy the shortcoming of TV of their model, we introduce an extra high-order total variation (HTV) regularization term. Furthermore, to balance the trade-off between edges and the smooth regions in the images, we also incorporate a weighting parameter to discriminate the TV and the HTV penalty. The proposed model is solved by an iterative algorithm under the framework of the well-known alternating direction method of multipliers. Our numerical results demonstrate the effectiveness and efficiency of the proposed method, in terms of signal-to-noise ratio (SNR) and relative error (RelRrr).     

THE QUASI-BOUNDARY VALUE METHOD FOR IDENTIFYING THE INITIAL VALUE OF THE SPACE-TIME FRACTIONAL DIFFUSION EQUATION

In this article, we consider to solve the inverse initial value problem for an inhomogeneous space-time fractional diffusion equation. This problem is ill-posed and the quasi-boundary value method is proposed to deal with this inverse problem and obtain the series expression of the regularized solution for the inverse initial value problem. We prove the error estimates between the regularization solution and the exact solution by using an a priori regularization parameter and an a posteriori regularization parameter choice rule. Some numerical results in one-dimensional case and two-dimensional case show that our method is effcient and stable.     

Optimal order results for a class of regularization methods using unbounded operators

A class of regularization methods using unbounded regularizing operators is considered for obtaining stable approximate solutions for ill-posed operator equations. With an a posteriori as well as an a priori parameter choice strategy, it is shown that the method yields the optimal order. Error estimates have also been obtained under stronger assumptions on the generalized solution. The results of the paper unify and simplify many of the results available in the literature. For example, the optimal results of the paper include, as particular cases for Tikhonov regularization, the main result of Mair (1994) with an a priori parameter choice, and a result of Nair (1999) with an a posteriori parameter choice. Thus the observations of Mair (1994) on Tikhonov regularization of ill-posed problems involving finitely and infinitely smoothing operators is applicable to various other regularization procedures as well. Subsequent results on error estimates include, as special cases, an optimal result of Vainikko (1987) and also some recent results of Tautenhahn (1996) in the setting of Hilbert scales.     

Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters

In this paper, we study the multi-parameter Tikhonov regularization method which adds multiple different penalties to exhibit multi-scale features of the solution. An optimal error bound of the regularization solution is obtained by a priori choice of multiple regularization parameters. Some theoretical results of the regularization solution about the dependence on regularization parameters are presented. Then, an a posteriori parameter choice, i.e., the damped Morozov discrepancy principle, is introduced to determine multiple regularization parameters. Five model functions, i.e., two hyperbolic model functions, a linear model function, an exponential model function and a logarithmic model function, are proposed to solve the damped Morozov discrepancy principle. Furthermore, four efficient model function algorithms are developed for finding reasonable multiple regularization parameters, and their convergence properties are also studied. Numerical results of several examples show that the damped discrepancy principle is competitive with the standard one, and the model function algorithms are efficient for choosing regularization parameters.     

Two Tikhonov‐type regularization methods for inverse source problem on the Poisson equation

In this paper, we investigate a problem of the identification of an unknown source on Poisson equation from some fixed location. A conditional stability estimate for an inverse heat source problem is proved. We show that such a problem is mildly ill‐posed and further present two Tikhonov‐type regularization methods (a generalized Tikhonov regularization method and a simplified generalized Tikhonov regularization method) to deal with this problem. Convergence estimates are presented under the a priori choice of the regularization parameter. Numerical results are presented to illustrate the accuracy and efficiency of our methods. Copyright © 2012 John Wiley & Sons, Ltd.     

Regularization Using QR Factorization and the Estimation of the Optimal Parameter

In this paper we propose a direct regularization method using QR factorization for solving linear discrete ill-posed problems. The decomposition of the coefficient matrix requires less computational cost than the singular value decomposition which is usually used for Tikhonov regularization. This method requires a parameter which is similar to the regularization parameter of Tikhonov's method. In order to estimate the optimal parameter, we apply three well-known parameter choice methods for Tikhonov regularization.This revised version was published online in October 2005 with corrections to the Cover Date.     

A new variant of L-curve for Tikhonov regularization

In this paper we introduce a new variant of L-curve to estimate the Tikhonov regularization parameter for the regularization of discrete ill-posed problems. This method uses the solution norm versus the regularization parameter. The numerical efficiency of this new method is also discussed by considering some test problems.     

求解热源识别问题的修正吉洪诺夫正则化方法

该文研究了一个热源识别问题,通过引入修正吉洪诺夫方法来处理问题的不适定性,在一种先验和一种后验参数选取准则下,分别获得了问题的误差估计.数值例子进一步验证了方法的有效性和稳定性.     

Lanczos-Based Exponential Filtering for Discrete Ill-Posed Problems

We describe regularizing iterative methods for the solution of large ill-conditioned linear systems of equations that arise from the discretization of linear ill-posed problems. The regularization is specified by a filter function of Gaussian type. A parameter determines the amount of regularization applied. The iterative methods are based on a truncated Lanczos decomposition and the filter function is approximated by a linear combination of Lanczos polynomials. A suitable value of the regularization parameter is determined by an L-curve criterion. Computed examples that illustrate the performance of the methods are presented.     

The balancing principle in solving semi-discrete inverse problems in Sobolev scales by Tikhonov method

In this article we discuss a regularization of semi-discrete ill-posed problem appearing as a result of application of a collocation method to Fredholm integral equation of the first kind. In this context we analyse Tikhonov regularization in Sobolev scales and prove error bounds under general source conditions. Moreover, we study an a posteriori regularization parameter choice by means of the balancing principle.     

Modular solvers for image restoration problems using the discrepancy principle

Many problems in image restoration can be formulated as either an unconstrained non‐linear minimization problem, usually with a Tikhonov‐like regularization, where the regularization parameter has to be determined; or as a fully constrained problem, where an estimate of the noise level, either the variance or the signal‐to‐noise ratio, is available. The formulations are mathematically equivalent. However, in practice, it is much easier to develop algorithms for the unconstrained problem, and not always obvious how to adapt such methods to solve the corresponding constrained problem. In this paper, we present a new method which can make use of any existing convergent method for the unconstrained problem to solve the constrained one. The new method is based on a Newton iteration applied to an extended system of non‐linear equations, which couples the constraint and the regularized problem, but it does not require knowledge of the Jacobian of the irregularity functional. The existing solver is only used as a black box solver, which for a fixed regularization parameter returns an improved solution to the unconstrained minimization problem given an initial guess. The new modular solver enables us to easily solve the constrained image restoration problem; the solver automatically identifies the regularization parameter, during the iterative solution process. We present some numerical results. The results indicate that even in the worst case the constrained solver requires only about twice as much work as the unconstrained one, and in some instances the constrained solver can be even faster. Copyright © 2002 John Wiley & Sons, Ltd.