Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters

In this paper, we study the multi-parameter Tikhonov regularization method which adds multiple different penalties to exhibit multi-scale features of the solution. An optimal error bound of the regularization solution is obtained by a priori choice of multiple regularization parameters. Some theoretical results of the regularization solution about the dependence on regularization parameters are presented. Then, an a posteriori parameter choice, i.e., the damped Morozov discrepancy principle, is introduced to determine multiple regularization parameters. Five model functions, i.e., two hyperbolic model functions, a linear model function, an exponential model function and a logarithmic model function, are proposed to solve the damped Morozov discrepancy principle. Furthermore, four efficient model function algorithms are developed for finding reasonable multiple regularization parameters, and their convergence properties are also studied. Numerical results of several examples show that the damped discrepancy principle is competitive with the standard one, and the model function algorithms are efficient for choosing regularization parameters.     

Identification of a time-dependent source term for a time fractional diffusion problem

In this paper, we study an inverse problem of identifying a time-dependent term of an unknown source for a time fractional diffusion equation using nonlocal measurement data. Firstly, we establish the conditional stability for this inverse problem. Then two regularization methods are proposed to for reconstructing the time-dependent source term from noisy measurements. The first method is an integral equation method which formulates the inverse source problem into an integral equation of the second kind; and a prior convergence rate of regularized solutions is derived with a suitable choice strategy of regularization parameters. The second method is a standard Tikhonov regularization method and formulates the inverse source problem as a minimizing problem of the Tikhonov functional. Based on the superposition principle and the technique of finite-element interpolation, a numerical scheme is proposed to implement the second regularization method. One- and two-dimensional examples are carried out to verify efficiency and stability of the second regularization method.     

New regularized algorithms based on the spectral method for solving deformable layer tomography

The deformable layer tomography (DLT) is now a popular way to characterize the unknown geometry of the velocity interface by using the traveling time observed in data, which is difficult to solve accurately, because of the strong ill-posedness. In this paper, new regularization approaches based on the spectral method are introduced, which can invert the velocity value and the geometry of the interface simultaneously. The unknown interfaces are parameterized by Legendre spectral expansion, and various regularization methods combined with traditional regularization parameters selections are utilized to solve the ill-conditioned algebraic equation system. Moreover, a regularized algorithm with prior choice of regularization parameters is proposed to solve the DLT.     

Application of Two-Parameter Stabilizing Functions in Solving a Convolution-Type Integral Equation by Regularization Method

This paper examines the solution of convolution-type integral equations of the first kind by applying the Tikhonov regularization method with two-parameter stabilizing functions. The class of stabilizing functions is expanded in order to improve the accuracy of the resulting solution. The features of the problem formulation for identification and adaptive signal correction are described. A method for choosing regularization parameters in problems of identification and adaptive signal correction is suggested.     

On nondecreasing sequences of regularization parameters for nonstationary iterated Tikhonov

Nonstationary iterated Tikhonov is an iterative regularization method that requires a strategy for defining the Tikhonov regularization parameter at each iteration and an early termination of the iterative process. A classical choice for the regularization parameters is a decreasing geometric sequence which leads to a linear convergence rate. The early iterations compute quickly a good approximation of the true solution, but the main drawback of this choice is a rapid growth of the error for later iterations. This implies that a stopping criteria, e.g. the discrepancy principle, could fail in computing a good approximation. In this paper we show by a filter factor analysis that a nondecreasing sequence of regularization parameters can provide a rapid and stable convergence. Hence, a reliable stopping criteria is no longer necessary. A geometric nondecreasing sequence of the Tikhonov regularization parameters into a fixed interval is proposed and numerically validated for deblurring problems.     

正则参数求解的微分进化算法

研究了正则化方法中正则参数的求解问题,提出了利用微分进化算法获取正则参数.微分进化算法属于全局最优化算法,具有鲁棒性强、收敛速度快、计算精度高的优点.把正则参数的求解问题转化为非线性优化问题,通过保持在解空间不同区域中各个点的搜索,以最大的概率找到问题的全局最优解,同时还利用数值模拟将此方法与广义交叉原理、L-曲线准则、逆最优准则等进行了对比,数值模拟结果表明该方法具有一定的可行性和有效性.     

Landweber iterative method for an inverse source problem of time-fractional diffusion-wave equation on spherically symmetric domain

In this paper, an inverse source problem of time-fractional diffusion-wave equation on spherically symmetric domain is considered. In general, this problem is ill-posed. Landweber iterative method is used to solve this inverse source problem. The error estimates between the regularization solution and the exact solution are derived by an a-priori and an a-posteriori regularization parameters choice rules. The numerical examples are presented to verify the efficiency and accuracy of the proposed methods.     

Nonlinear regression modeling and detecting change points via the relevance vector machine

We consider the problem of constructing nonlinear regression models in the case that the structure of data has discontinuities at some unknown points. We propose two-stage procedure in which the change points are detected by relevance vector machine at the first stage, and the smooth curve are effectively estimated along with the technique of regularization method at the second. In order to select tuning parameters in the regularization method, we derive a model selection and evaluation criterion from information-theoretic viewpoints. Simulation results and real data analyses demonstrate that our methodology performs well in various situations.     

A regularization method for the cauchy problem of the modified Helmholtz equation

In the present paper, an iteration regularization method for solving the Cauchy problem of the modified Helmholtz equation is proposed. The a priori and a posteriori rule for choosing regularization parameters with corresponding error estimates between the exact solution and its approximation are also given. The numerical example shows the effectiveness of this method. Copyright © 2014 John Wiley & Sons, Ltd.     

Proximal Point Methods and Nonconvex Optimization

The goal of this paper is to discover some possibilities for applying the proximal point method to nonconvex problems. It can be proved that – for a wide class of problems – proximal regularization performed with appropriate regularization parameters ensures convexity of the auxiliary problems and each accumulation point of the method satisfies the necessary optimality conditions.     

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current-voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.     

A new method for numerical differentiation based on direct and inverse problems of partial differential equations

In this paper, we mainly study a numerical differentiation problem which aims to approximate the second order derivative of a single variable function from its noise data. By transforming the problem into a combination of direct and inverse problems of partial differential equations (heat conduction equations), a new method that we call the PDEs-based numerical differentiation method is proposed. By means of the finite element method and the Tikhonov regularization, implementations of the proposed PDEs-based method are presented with a posterior strategy for choosing regularization parameters. Numerical results show that the PDEs-based numerical differentiation method is highly feasible and stable with respect to data noise.     

Identification of distributed systems and the theory of the regularization

The identification problems, i.e., the problems of finding unknown parameters in distributed systems from the observations are very important in modern control theory. The solutions of these identification problems can be obtained by solving the equations of the first kind. However, the solutions are often unstable. In other words, they are not continuously dependent on the data. The regularization or Tihonov's regularization is known as one of the stabilizing algorithms to solve these non well-posed problems. In this paper is studied the regularization method for identification of distributed systems. Several approximation theorems are proved to solve the equations of the first kind. Then, identification problems are reduced to the minimization of quadratic cost functionals by virtue of these theorems. On the other hand, it is known that the statistical methods for identification such as the maximum likelihood lead to the minimization problems of certain quadratic functionals. Comparing these quadratic cost functionals, the relations between the regularization and the statistical methods are discussed. Further, numerical examples are given to show the effectiveness of this method.     

Cubically convergent methods for selecting the regularization parameters in linear inverse problems

We present three cubically convergent methods for choosing the regularization parameters in linear inverse problems. The detailed algorithms are given and the convergence rates are estimated. Our basic tools are Tikhonov regularization and Morozov's discrepancy principle. We prove that, in comparison with the standard Newton method, the computational costs for our cubically convergent methods are nearly the same, but the number of iteration steps is even less. Numerical experiments for an elliptic boundary value problem illustrate the efficiency of the proposed algorithms.     

A new trust region algorithm for image restoration

The image restoration problems play an important role in remote sensing and astronomical image analysis. One common method for the recovery of a true image from corrupted or blurred image is the least squares error (LSE) method. But the LSE method is unstable in practical applications. A popular way to overcome instability is the Tikhonov regularization. However, difficulties will encounter when adjusting the so-called regularization parameter a. Moreover, how to truncate the iteration at appropriate steps is also challenging. In this paper we use the trust region method to deal with the image restoration problem, meanwhile, the trust region subproblem is solved by the truncated Lanczos method and the preconditioned truncated Lanczos method. We also develop a fast algorithm for evaluating the Kronecker matrix-vector product when the matrix is banded. The trust region method is very stable and robust, and it has the nice property of updating the trust region automatically. This releases us from tedious fi     

Regularized Adaptation of Fuzzy Inference Systems. Modelling the Opinion of a Medical Expert about Physical Fitness: An Application

This study presents a new approach to adaptation of Sugeno type fuzzy inference systems using regularization, since regularization improves the robustness of standard parameter estimation algorithms leading to stable fuzzy approximation. The proposed method can be used for modelling, identification and control of physical processes. A recursive method for on-line identification of fuzzy parameters employing Tikhonov regularization is suggested. The power of approach was shown by applying it to the modelling, identification, and adaptive control problems of dynamic processes. The proposed approach was used for modelling of human-decisions (experience) with a fuzzy inference system and for the fuzzy approximation of physical fitness with real world medical data.     

Dependence of locally linear embedding on the regularization parameter

This paper deals with a method, called locally linear embedding (LLE). It is a nonlinear dimensionality reduction technique that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional data and attempts to discover a nonlinear structure (including manifolds) in high-dimensional data. In practice, the nonlinear manifold learning methods are applied in image processing, text mining, etc. The implementation of the LLE algorithm is fairly straightforward, because the algorithm has only two control parameters: the number of neighbors of each data point and the regularization parameter. The mapping quality is quite sensitive to these parameters. In this paper, we propose a new way of selecting a regularization parameter of a local Gram matrix.     

Regularization of Linear Regression Problems

The study considers robust estimation of linear regression parameters by the regularization method, the pseudoinverse method, and the Bayesian method allowing for correlations and errors in the data. Regularizing algorithms are constructed and their relationship with pseudoinversion, the Bayesian approach, and BLUE is investigated.     

散乱数据的数值微分及其误差估计

1 背景及问题的提出 导数是数学分析中的一个基本的慨念。对于数学工作者来讲,计算导数不是一项特别困难的工作。但是,对于研究实际问题的科学工作者来讲,这项工作就不是一件简单的工作了,首先,求导数的问题是一个典型的Hadamard意义下的不适定问题([5],[12]等)     

解不适定问题的快速多层方法

Many industrial and engineering applications require numerically solving ill-posed problems. Regularization methods are employed to find approximate solutions of these problems. The choice of regularization parameters by numerical algorithms is one of the most important issues for the success of regularization methods. When we use some discrepancy principles to determine the regularization parameter,