A Discretizing Levenberg-Marquardt Scheme for Solving Nonlinear Ill-Posed Integral Equations

To reduce the computational cost, we propose a regularizing modified Levenberg-Marquardt scheme via multiscale Galerkin method for solving nonlinear ill-posed problems. Convergence results for the regularizing modified Levenberg-Marquardt scheme for the solution of nonlinear ill-posed problems have been proved. Based on these results, we propose a modified heuristic parameter choice rule to terminate the regularizing modified Levenberg-Marquardt scheme. By imposing certain conditions on the noise, we derive optimal convergence rates on the approximate solution under special source conditions. Numerical results are presented to illustrate the performance of the regularizing modified Levenberg-Marquardt scheme under the modified heuristic parameter choice.     

A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level

We consider Tikhonov regularization of linear ill-posed problems with noisy data. The choice of the regularization parameter by classical rules, such as discrepancy principle, needs exact noise level information: these rules fail in the case of an underestimated noise level and give large error of the regularized solution in the case of very moderate overestimation of the noise level. We propose a general family of parameter choice rules, which includes many known rules and guarantees convergence of approximations. Quasi-optimality is proved for a sub-family of rules. Many rules from this family work well also in the case of many times under- or overestimated noise level. In the case of exact or overestimated noise level we propose to take the regularization parameter as the minimum of parameters from the post-estimated monotone error rule and a certain new rule from the proposed family. The advantages of the new rules are demonstrated in extensive numerical experiments.     

Convergence Rates for Linear Inverse Problems in the Presence of an Additive Normal Noise

Abstract

In this work, we examine a finite-dimensional linear inverse problem where the measurements are disturbed by an additive normal noise. The problem is solved both in the frequentist and in the Bayesian frameworks. Convergence of the used methods when the noise tends to zero is studied in the Ky Fan metric. The obtained convergence rate results and parameter choice rules are of a similar structure for both approaches.     

Parameter choice methods using minimization schemes

Regularization is typically based on the choice of some parametric family of nearby solutions, and the choice of this family is a task in itself. Then, a suitable parameter must be chosen in order to find an approximation of good quality. We focus on the second task. There exist deterministic and stochastic models for describing noise and solutions in inverse problems. We will establish a unified framework for treating different settings for the analysis of inverse problems, which allows us to prove the convergence and optimality of parameter choice schemes based on minimization in a generic way. We show that the well known quasi-optimality criterion falls in this class. Furthermore we present a new parameter choice method and prove its convergence by using this newly established tool.     

On simplified Tikhonov regularization

Approximations to the minimal norm, least-square solution of a linear equation with positive semidefinite operator are defined in such a way that fewer computations are needed than in Tikhonov's approach. We establish necessary and sufficient conditions for convergence, and we provide a choice for the regularization parameter that brings the optimal rate of convergence.The author expresses his gratitude to C. W. Groetsch for his guidance and the recent visit to the University of Puerto Rico.     

Convergence analysis of a regularized approximation for solving Fredholm integral equations of the first kind

In this paper, we suggest a convergence analysis for solving Fredholm integral equations of the first kind using Tikhonov regularization under supremum norm. We also provide an a priori parameter choice strategy for choosing the regularization parameter and obtain an error estimate.     

Regularization properties of the sequential discrepancy principle for Tikhonov regularization in Banach spaces

The stable solution of ill-posed non-linear operator equations in Banach space requires regularization. One important approach is based on Tikhonov regularization, in which case a one-parameter family of regularized solutions is obtained. It is crucial to choose the parameter appropriately. Here, a sequential variant of the discrepancy principle is analysed. In many cases, such parameter choice exhibits the feature, called regularization property below, that the chosen parameter tends to zero as the noise tends to zero, but slower than the noise level. Here, we shall show such regularization property under two natural assumptions. First, exact penalization must be excluded, and secondly, the discrepancy principle must stop after a finite number of iterations. We conclude this study with a discussion of some consequences for convergence rates obtained by the discrepancy principle under the validity of some kind of variational inequality, a recent tool for the analysis of inverse problems.     

Necessary and sufficient conditions for power convergence rate of approximations in Tikhonov’s scheme for solving ill-posed optimization problems

We investigate a rate of convergence of estimates for approximations generated by Tikhonov’s scheme for solving ill-posed optimization problems with smooth functionals under a structural nonlinearity condition in a Hilbert space, in the cases of exact and noisy input data. In the noise-free case, we prove that the power source representation of the desired solution is close to a necessary and sufficient condition for the power convergence estimate having the same exponent with respect to the regularization parameter. In the presence of a noise, we give a parameter choice rule that leads for Tikhonov’s scheme to a power accuracy estimate with respect to the noise level.     

Multi-penalty regularization in learning theory

In this paper we establish the error estimates for multi-penalty regularization under the general smoothness assumption in the context of learning theory. One of the motivation for this work is to study the convergence analysis of two-parameter regularization theoretically in the manifold learning setting. In this spirit, we obtain the error bounds for the manifold learning problem using more general framework of multi-penalty regularization. We propose a new parameter choice rule “the balanced-discrepancy principle” and analyze the convergence of the scheme with the help of estimated error bounds. We show that multi-penalty regularization with the proposed parameter choice exhibits the convergence rates similar to single-penalty regularization. Finally on a series of test samples we demonstrate the superiority of multi-parameter regularization over single-penalty regularization.     

A Regularized Iterative Scheme for Solving Nonlinear Ill-Posed Problems

In this article, we consider a regularized iterative scheme for solving nonlinear ill-posed problems. The convergence analysis and error estimates are derived by choosing the regularization parameter according to both a priori and a posteriori methods. The iterative scheme is stopped using an a posteriori stopping rule, and we prove that the scheme converges to the solution of the well-known Lavrentiev scheme. The salient features of the proposed scheme are: (i) convergence and error estimate analysis require only weaker assumptions compared to standard assumptions followed in literature, and (ii) consideration of an adaptive a posteriori stopping rule and a parameter choice strategy that gives the same convergence rate as that of an a priori method without using the smallness assumption, the source condition. The above features are very useful from theory and application points of view. We also supply the numerical results to illustrate that the method is adaptable. Further, we compare the numerical result of the proposed method with the standard approach to demonstrate that our scheme is stable and achieves good computational output.     

On the convergence of Filon quadrature

We analyze the convergence behavior of Filon-type quadrature rules by making explicit the dependence on both k, the parameter that controls the oscillatory behavior of the integrand, and n, the number of function evaluations. We provide explicit conditions on the domain of analyticity of the integrand to ensure convergence for n→∞.     

Tikhonov regularization of nonlinear III-posed problems in hilbert scales

In this paper we consider nonlinear ill-posed problems F(x) = y ₀, where x and y ₀ are elements of Hilbert spaces X and Y, respectively. We solve these problems by Tikhonov regularization in a Hilbert scale. This means that the regularizing norm is stronger than the norm in X. Smoothness conditions are given that guarantee convergence rates with respect to the data noise in the original norm in X. We also propose a variant of Tikhonov regularization that yields these rates without needing the knowledge of the smoothness conditions. In this variant F is allowed to be known only approximately and X can be approximated by a finite-dimensional subspace. Finally, we illustrate the required conditions for a simple parameter estimation problem for regularization in Sobolev spaces.     

A saddle point variational formulation for projection–regularized parameter identification

Summary. This paper is concerned with the ill-posed problem of identifying a parameter in an elliptic equation and its solution applying regularization by projection. As the theory has shown, the ansatz functions for the parameter have to be sufficiently smooth. In this paper we show that these – for a practical implementation unrealistic – smoothness assumptions can be circumvented by reformulating the problem under consideration as a mixed variational equation. We prove convergence as the discretization gets finer in the noise free case and convergence as the data noise level goes to zero in the case of noisy data, as well as convergence rates under additional smoothness conditions. Received August 4, 2000 / Revised version received March 21, 2001 / Published online October 17, 2001     

Regularized collocation method for Fredholm integral equations of the first kind

In this paper we consider a collocation method for solving Fredholm integral equations of the first kind, which is known to be an ill-posed problem. An “unregularized” use of this method can give reliable results in the case when the rate at which smallest singular values of the collocation matrices decrease is known a priori. In this case the number of collocation points plays the role of a regularization parameter. If the a priori information mentioned above is not available, then a combination of collocation with Tikhonov regularization can be the method of choice. We analyze such regularized collocation in a rather general setting, when a solution smoothness is given as a source condition with an operator monotone index function. This setting covers all types of smoothness studied so far in the theory of Tikhonov regularization. One more issue discussed in this paper is an a posteriori choice of the regularization parameter, which allows us to reach an optimal order of accuracy for deterministic noise model without any knowledge of solution smoothness.     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

Modified Landweber Iteration in Banach Spaces—Convergence and Convergence Rates

We introduce and discuss an iterative method of modified Landweber type for regularization of nonlinear operator equations in Banach spaces. Under smoothness and convexity assumptions on the solution space we present convergence and stability results. Furthermore, we will show that under the so-called approximate source conditions convergence rates may be achieved by a proper a-priori choice of the parameter of the presented algorithm. We will illustrate these theoretical results with a numerical example.     

A superquadratic infeasible-interior-point method for linear complementarity problems

We consider a modification of a path-following infeasible-interior-point algorithm described by Wright. In the new algorithm, we attempt to improve each major iterate by reusing the coefficient matrix factors from the latest step. We show that the modified algorithm has similar theoretical global convergence properties to those of the earlier algorithm while its asymptotic convergence rate can be made superquadratic by an appropriate parameter choice. The work of this author was based on research supported by the Office of Scientific Computing, US Department of Energy, under Contract W-31-109-Eng-38. The work of this author was based on research supported in part by the US Department of Energy under Grant DE-FG02-93ER25171.     

Numerical differentiation from a viewpoint of regularization theory

In this paper, we discuss the classical ill-posed problem of numerical differentiation, assuming that the smoothness of the function to be differentiated is unknown. Using recent results on adaptive regularization of general ill-posed problems, we propose new rules for the choice of the stepsize in the finite-difference methods, and for the regularization parameter choice in numerical differentiation regularized by the iterated Tikhonov method. These methods are shown to be effective for the differentiation of noisy functions, and the order-optimal convergence results for them are proved.

     

On general hemiequilibrium problems

In this paper, we introduce and analyze a new class of equilibrium problems known as general hemiequilibrium problems. It is shown that this class includes hemiequilibrium problems, hemivariational inequalities and complementarity problems as special cases. We use the auxiliary principle techniques to suggest some iterative-type methods for solving multivalued hemiequilibrium problems. We also analyze the convergence analysis of these new iterative methods under some mild conditions. As special cases, we obtain several new and known methods for solving variational inequalities and equilibrium problems.     

Single-machine scheduling problems with time and position dependent processing times

We consider single-machine scheduling problems with time and position dependent job processing times. In many industrial settings, the processing time of a job changes due to either job deterioration over time or machine/worker’s learning through experiences. In the models we study, each job has its normal processing time. However, a job’s actual processing time depends on when its processing starts and how many jobs have completed before its start. We prove that the classical SPT (Shortest Processing Time) rule remains optimal when we minimize the makespan or the total completion time. For problems of minimizing the total weighted completion time, the maximum lateness, and the discounted total weighted completion time, we present heuristic sequencing rules and analyze the worst-case bounds for performance ratios. We also show that these heuristic rules can be optimal under some agreeable conditions between the normal processing times and job due dates or weights.