Modular solvers for image restoration problems using the discrepancy principle

Many problems in image restoration can be formulated as either an unconstrained non‐linear minimization problem, usually with a Tikhonov‐like regularization, where the regularization parameter has to be determined; or as a fully constrained problem, where an estimate of the noise level, either the variance or the signal‐to‐noise ratio, is available. The formulations are mathematically equivalent. However, in practice, it is much easier to develop algorithms for the unconstrained problem, and not always obvious how to adapt such methods to solve the corresponding constrained problem. In this paper, we present a new method which can make use of any existing convergent method for the unconstrained problem to solve the constrained one. The new method is based on a Newton iteration applied to an extended system of non‐linear equations, which couples the constraint and the regularized problem, but it does not require knowledge of the Jacobian of the irregularity functional. The existing solver is only used as a black box solver, which for a fixed regularization parameter returns an improved solution to the unconstrained minimization problem given an initial guess. The new modular solver enables us to easily solve the constrained image restoration problem; the solver automatically identifies the regularization parameter, during the iterative solution process. We present some numerical results. The results indicate that even in the worst case the constrained solver requires only about twice as much work as the unconstrained one, and in some instances the constrained solver can be even faster. Copyright © 2002 John Wiley & Sons, Ltd.     

求解病态问题的一种改进的Tikhonov正则化--(1)正则化方法的建立

对于带有右端扰动数据的第一类紧算子方程的病态问题 ,本文应用正则化子建立了一类新的正则化求解方法 ,称之为改进的Tikonov正则化 ;通过适当选取正则参数 ,证明了正则解具有最优的渐近收敛阶 .与通常的Tikhonov正则化相比 ,这种改进的正则化可使正则解取到足够高的最优渐近阶     

Regularization of nonlinear illposed problems with closed operators

In this paper Tikhonov regularization for nonlinear illposed problems is investigated. The regularization term is characterized by a closed linear operator, permitting seminorm regularization in applications. Results for existence, stability, convergence and con- vergence rates of the solution of the regularized problem in terms of the noise level are given. An illustrating example involving parameter estimation for a one dimensional stationary heat equation is given.     

解第一类算子方程的一种新的正则化方法

对算子与右端都为近似给定的第一类算子方程提出一种新的正则化方法，依据广义Ａｒｃａｎｇｅｌｉ方法选取正则参数，建立了正则解的收敛性。这种新的正则化方法与通常的Ｔｉｋｈｏｎｏｖ正则化方法相比较，提高了正则解的渐近阶估计。     

Regularized collocation method for Fredholm integral equations of the first kind

In this paper we consider a collocation method for solving Fredholm integral equations of the first kind, which is known to be an ill-posed problem. An “unregularized” use of this method can give reliable results in the case when the rate at which smallest singular values of the collocation matrices decrease is known a priori. In this case the number of collocation points plays the role of a regularization parameter. If the a priori information mentioned above is not available, then a combination of collocation with Tikhonov regularization can be the method of choice. We analyze such regularized collocation in a rather general setting, when a solution smoothness is given as a source condition with an operator monotone index function. This setting covers all types of smoothness studied so far in the theory of Tikhonov regularization. One more issue discussed in this paper is an a posteriori choice of the regularization parameter, which allows us to reach an optimal order of accuracy for deterministic noise model without any knowledge of solution smoothness.     

Homotopy method for a mean curvature-based denoising model

Variational image denoising models based on regularization of gradients have been extensively studied. The total variation model by Rudin, Osher, and Fatemi (1992) [38] can preserve edges well but for images without edges (jumps), the solution to this model has the undesirable staircasing effect. To overcome this, mean curvature-based energy minimization models offer one approach for restoring both smooth (no edges) and nonsmooth (with edges) images. As such models lead to fourth order (instead of the usual second order) nonlinear partial differential equations, development of fast solvers is a challenging task. Previously stabilized fixed point methods and their associated multigrid methods were developed but the underlying operators must be regularized by a relatively large parameter. In this paper, we first present a fixed point curvature method for solving such equations and then propose a homotopy approach for varying the regularized parameter so that the Newton type method becomes applicable in a predictor-corrector framework. Numerical experiments show that both of our methods are able to maintain all important information in the image, and at the same time to filter out noise.     

High-order total variation-based Poissonian image deconvolution with spatially adapted regularization parameter

Images captured by image acquisition systems using photon-counting devices such as astronomical imaging, positron emission tomography and confocal microscopy imaging, are often contaminated by Poisson noise. Total variation (TV) regularization, which is a classic regularization technique in image restoration, is well-known for recovering sharp edges of an image. Since the regularization parameter is important for a good recovery, Chen and Cheng (2012) proposed an effective TV-based Poissonian image deblurring model with a spatially adapted regularization parameter. However, it has drawbacks since the TV regularization produces staircase artifacts. In this paper, in order to remedy the shortcoming of TV of their model, we introduce an extra high-order total variation (HTV) regularization term. Furthermore, to balance the trade-off between edges and the smooth regions in the images, we also incorporate a weighting parameter to discriminate the TV and the HTV penalty. The proposed model is solved by an iterative algorithm under the framework of the well-known alternating direction method of multipliers. Our numerical results demonstrate the effectiveness and efficiency of the proposed method, in terms of signal-to-noise ratio (SNR) and relative error (RelRrr).     

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current-voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.     

A regularized parameter choice in regularization for a common solution of a finite system of ill-posed equations involving Lipschitz continuous and accretive mappings

In this paper, we present a regularized parameter choice in a new regularization method of Browder-Tikhonov type, for finding a common solution of a finite system of ill-posed operator equations involving Lipschitz continuous and accretive mappings in a real reflexive and strictly convex Banach space with a uniformly Gateaux differentiate norm. An estimate for convergence rates of regularized solution is also established.     

A Modified Tikhonov Regularization for Linear Operator Equations

We construct with the aid of regularizing filters a new class of improved regularization methods, called modified Tikhonov regularization (MTR), for solving ill-posed linear operator equations. Regularizing properties and asymptotic order of the regularized solutions are analyzed in the presence of noisy data and perturbation error in the operator. With some accurate estimates in the solution errors, optimal convergence order of the regularized solutions is obtained by a priori choice of the regularization parameter. Furthermore, numerical results are given for several ill-posed integral equations, which not only roughly coincide with the theoretical results but also show that MTR can be more accurate than ordinary Tikhonov regularization (OTR).     

Fraction-order total variation blind image restoration based on L1-norm

A fraction-order total variation blind image restoration algorithm based on L1-norm was proposed for restoring the images blurred by unknown point spread function (PSF) during imaging. According to the form of total variation, this paper introduced an arithmetic operator of fraction-order total variation and generated a mathematical model of cost. Semi-quadratic regularization was used to solve the model iteratively so that the solution of this algorithm became easier. This paper also analyzed the convergence of this algorithm and then testified its feasibility in theory. The experimental results showed the proposed algorithm can increase the PSNR of the restored image by 1 dB in relation to the first order total variation blind restoration method and Bayesian blind restoration method. The details in real blurred image were also pretty well restored. The effectiveness of the proposed algorithm revealed that it was practical in the blind image restoration.     

Regularization properties of the sequential discrepancy principle for Tikhonov regularization in Banach spaces

The stable solution of ill-posed non-linear operator equations in Banach space requires regularization. One important approach is based on Tikhonov regularization, in which case a one-parameter family of regularized solutions is obtained. It is crucial to choose the parameter appropriately. Here, a sequential variant of the discrepancy principle is analysed. In many cases, such parameter choice exhibits the feature, called regularization property below, that the chosen parameter tends to zero as the noise tends to zero, but slower than the noise level. Here, we shall show such regularization property under two natural assumptions. First, exact penalization must be excluded, and secondly, the discrepancy principle must stop after a finite number of iterations. We conclude this study with a discussion of some consequences for convergence rates obtained by the discrepancy principle under the validity of some kind of variational inequality, a recent tool for the analysis of inverse problems.     

A “Nonconvex+Nonconvex” approach for image restoration with impulse noise removal

In this paper, we propose a new method for image restoration problems, which are degraded by impulsive noise, with nonconvex data fitting term and nonconvex regularizer.The proposed method possesses the advantages of nonconvex data fitting and nonconvex regularizer simultaneously, namely, robustness for impulsive noise and efficiency for restoring neat edge images.Further, we propose an efficient algorithm to solve the “Nonconvex+Nonconvex” structure problem via using the alternating direction minimization, and prove that the algorithm is globally convergent when the regularization parameter is known. However, the regularization parameter is unavailable in general. Thereby, we combine the algorithm with the continuation technique and modified Morozov’s discrepancy principle to get an improved algorithm in which a suitable regularization parameter can be chosen automatically. The experiments reveal the superior performances of the proposed algorithm in comparison with some existing methods.     

On dynamics of quantum states generated by the Cauchy problem for the Schrödinger equation with degeneration on the half-line

The paper considers the Cauchy problem for the Schrödinger equation with operator degenerate on the semiaxis and the family of regularized Cauchy problems with uniformly elliptic operators whose solutions approximate the solution of the degenerate problem. The author studies the strong and weak convergences of the regularized problems and the convergence of values of quadratic forms of bounded operators on solutions of the regularized problems when the regularization parameter tends to zero.     

REGULARIZATION METHOD FOR IMPROVING OPTIMAL CONVERGENCE RATE OF THE REGULARIZED SOLUTION OF ILL-POSED PROBLEMS

1IntroductionThestudyoflllallymathematicalphysicsproblemsleadstosolvingoperatorequatiollsofthefirstkind,andtheoperatorequatiollsofthefirstkindaretypicallyill--posedprobellis[1,2,3,4].Themethodsforsolvillgill-posedproblellishavebeenstudiedbyagreatnumberofresearchers.WementionTikhonovandArsellill[2],Morozov[3]IGroetscll[4],Engll51,HouandLi['],ChenandHouI7]alldoillerscholars.Illtheirresearches,they11avediscussedtheproblemoffindingstableapproxilllatesolutiollsand11aveillvestigatedtileconverge…     

Adaptive Arnoldi-Tikhonov regularization for image restoration

In the framework of the numerical solution of linear systems arising from image restoration, in this paper we present an adaptive approach based on the reordering of the image approximations obtained with the Arnoldi-Tikhonov method. The reordering results in a modified regularization operator, so that the corresponding regularization can be interpreted as problem dependent. Numerical experiments are presented.     

The Lagrange method for the regularization of discrete ill-posed problems

In many science and engineering applications, the discretization of linear ill-posed problems gives rise to large ill-conditioned linear systems with the right-hand side degraded by noise. The solution of such linear systems requires the solution of minimization problems with one quadratic constraint, depending on an estimate of the variance of the noise. This strategy is known as regularization. In this work, we propose a modification of the Lagrange method for the solution of the noise constrained regularization problem. We present the numerical results of test problems, image restoration and medical imaging denoising. Our results indicate that the proposed Lagrange method is effective and efficient in computing good regularized solutions of ill-conditioned linear systems and in computing the corresponding Lagrange multipliers. Moreover, our numerical experiments show that the Lagrange method is computationally convenient. Therefore, the Lagrange method is a promising approach for dealing with ill-posed problems. This work was supported by the Italian FIRB Project “Parallel algorithms and Nonlinear Numerical Optimization” RBAU01JYPN.     

Optimal control for systems described by hyperbolic equations with strong nonlinearity

An optimization control problem for a hyperbolic equation is considered. The system is nonlinear with respect to the state derivative. The regularization technique for the state equation is applied. The necessary conditions of optimality for the regularized control problem are proved. It uses the extended differentiability of the control-state mapping for the regularized equation. The convergence of the regularization method is proved. Thus the optimal control for the regularized problem with a small enough regularization parameter can be chosen as an approximate solution of the initial optimization problem.     

Auxiliary Equations Approach for the Stochastic Unsteady Navier-Stokes Equations with Additive Random Noise

This paper presents a Martingale regularization method for the stochastic Navier-Stokes equations with additive noise. The original system is split into two equivalent parts, the linear stochastic Stokes equations with Martingale solution and the stochastic modified Navier-Stokes equations with relatively-higher regularities. Meanwhile, a fractional Laplace operator is introduced to regularize the noise term. The stability and convergence of numerical scheme for the pathwise modified Navier-Stokes equations are proved.The comparisons of non-regularized and regularized noises for the Navier-Stokes system are numerically presented to further demonstrate the efficiency of our numerical scheme.     

On a terminal value problem for a generalization of the fractional diffusion equation with hyper-Bessel operator

In this paper, we consider an inverse problem of recovering the initial value for a generalization of time-fractional diffusion equation, where the time derivative is replaced by a regularized hyper-Bessel operator. First, we investigate the existence and regularity of our terminal value problem. Then we show that the backward problem is ill-posed, and we propose a regularizing scheme using a fractional Tikhonov regularization method. We also present error estimates between the regularized solution and the exact solution using two parameter choice rules.