Iterative Methods of Richardson-Lucy-Type for Image Deblurring

Image deconvolution problems with a symmetric point-spread function arise in many areas of science and engineering. These problems often are solved by the Richardson-Lucy method, a nonlinear iterative method. We first show a convergence result for the Richardson-Lucy method. The proof sheds light on why the method may converge slowly. Subsequently, we describe an iterative active set method that imposes the same constraints on the computed solution as the Richardson-Lucy method. Computed examples show the latter method to yield better restorations than the Richardson-Lucy method and typically require less computational effort.     

A Truncated Newton Method for the Solution of Large-Scale Inequality Constrained Minimization Problems

A new active set Newton-type algorithm for the solution of inequality constrained minimization problems is proposed. The algorithm possesses the following favorable characteristics: (i) global convergence under mild assumptions; (ii) superlinear convergence of primal variables without strict complementarity; (iii) a Newton-type direction computed by means of a truncated conjugate gradient method. Preliminary computational results are reported to show viability of the approach in large scale problems having only a limited number of constraints.     

On diagonally preconditioning the truncated Newton method for super-scale linearly constrained nonlinear programming

We present an algorithm for super-scale linearly constrained nonlinear programming (LCNP) based on Newton's method. In large-scale programming solving the Newton equation at each iteration can be expensive and may not be justified when far from a local solution. For super-scale problems, the truncated Newton method (where an inaccurate solution is computed by using the conjugate-gradient method) is recommended; a diagonal BFGS preconditioning of the gradient is used, so that the number of iterations to solve the equation is reduced. The procedure for updating that preconditioning is described for LCNP when the set of active constraints or the partition of basic, superbasic and nonbasic (structural) variables have been changed.     

An active set strategy for solving optimization problems with up to 200,000,000 nonlinear constraints

Numerical test results are presented for solving smooth nonlinear programming problems with a large number of constraints, but a moderate number of variables. The active set method proceeds from a given bound for the maximum number of expected active constraints at an optimal solution, which must be less than the total number of constraints. A quadratic programming subproblem is generated with a reduced number of linear constraints from the so-called working set, which is internally changed from one iterate to the next. Only for active constraints, i.e., a certain subset of the working set, new gradient values must be computed. The line search is adapted to avoid too many active constraints which do not fit into the working set. The active set strategy is an extension of an algorithm described earlier by the author together with a rigorous convergence proof. Numerical results for some simple academic test problems show that nonlinear programs with up to 200,000,000 nonlinear constraints are efficiently solved on a standard PC.     

A regularized projection method for complementarity problems with non-Lipschitzian functions

We consider complementarity problems involving functions which are not Lipschitz continuous at the origin. Such problems arise from the numerical solution for differential equations with non-Lipschitzian continuity, e.g. reaction and diffusion problems. We propose a regularized projection method to find an approximate solution with an estimation of the error for the non-Lipschitzian complementarity problems. We prove that the projection method globally and linearly converges to a solution of a regularized problem with any regularization parameter. Moreover, we give error bounds for a computed solution of the non-Lipschitzian problem. Numerical examples are presented to demonstrate the efficiency of the method and error bounds.

     

Iterative exponential filtering for large discrete ill-posed problems

Summary. We describe a new iterative method for the solution of large, very ill-conditioned linear systems of equations that arise when discretizing linear ill-posed problems. The right-hand side vector represents the given data and is assumed to be contaminated by measurement errors. Our method applies a filter function of the form with the purpose of reducing the influence of the errors in the right-hand side vector on the computed approximate solution of the linear system. Here is a regularization parameter. The iterative method is derived by expanding in terms of Chebyshev polynomials. The method requires only little computer memory and is well suited for the solution of large-scale problems. We also show how a value of and an associated approximate solution that satisfies the Morozov discrepancy principle can be computed efficiently. An application to image restoration illustrates the performance of the method. Received January 25, 1997 / Revised version received February 9, 1998 / Published online July 28, 1999     

A generalized global Arnoldi method for ill-posed matrix equations

This paper discusses the solution of large-scale linear discrete ill-posed problems with a noise-contaminated right-hand side. Tikhonov regularization is used to reduce the influence of the noise on the computed approximate solution. We consider problems in which the coefficient matrix is the sum of Kronecker products of matrices and present a generalized global Arnoldi method, that respects the structure of the equation, for the solution of the regularized problem. Theoretical properties of the method are shown and applications to image deblurring are described.     

A Galerkin method for mixed parabolic–elliptic partial differential equations

In this article boundary value problems for partial differential equations of mixed elliptic–parabolic type are considered. To ensure that the considered problems possess a unique solution, the usual variational existence proof for parabolic problems is extended to the mixed situation. Further, the convergence of approximations computed by a time-space Galerkin method to the solution of the mixed problem is proven and error estimates are given.     

A truncated projected SVD method for linear discrete ill-posed problems

Truncated singular value decomposition is a popular solution method for linear discrete ill-posed problems. However, since the singular value decomposition of the matrix is independent of the right-hand side, there are linear discrete ill-posed problems for which this method fails to yield an accurate approximate solution. This paper describes a new approach to incorporating knowledge about properties of the desired solution into the solution process through an initial projection of the linear discrete ill-posed problem. The projected problem is solved by truncated singular value decomposition. Computed examples illustrate that suitably chosen projections can enhance the accuracy of the computed solution.     

Discrete ill-posed least-squares problems with a solution norm constraint

Straightforward solution of discrete ill-posed least-squares problems with error-contaminated data does not, in general, give meaningful results, because propagated error destroys the computed solution. Error propagation can be reduced by imposing constraints on the computed solution. A commonly used constraint is the discrepancy principle, which bounds the norm of the computed solution when applied in conjunction with Tikhonov regularization. Another approach, which recently has received considerable attention, is to explicitly impose a constraint on the norm of the computed solution. For instance, the computed solution may be required to have the same Euclidean norm as the unknown solution of the error-free least-squares problem. We compare these approaches and discuss numerical methods for their implementation, among them a new implementation of the Arnoldi–Tikhonov method. Also solution methods which use both the discrepancy principle and a solution norm constraint are considered.     

Solving the minimal least squares problem subject to bounds on the variables

A computational procedure is developed for determining the solution of minimal length to a linear least squares problem subject to bounds on the variables. In the first stage, a solution to the least squares problem is computed and then in the second stage, the solution of minimal length is determined. The objective function in each step is minimized by an active set method adapted to the special structure of the problem.The systems of linear equations satisfied by the descent direction and the Lagrange multipliers in the minimization algorithm are solved by direct methods based on QR decompositions or iterative preconditioned conjugate gradient methods. The direct and the iterative methods are compared in numerical experiments, where the solutions are sought to a sequence of related, minimal least squares problems subject to bounds on the variables. The application of the iterative methods to large, sparse problems is discussed briefly.This work was supported by The National Swedish Board for Technical Development under contract dnr 80-3341.     

Validated Infeasible Interior-Point Predictor–Corrector Methods for Linear Programming

Many problems arising in practical applications lead to linear programming problems. Hence, they are fundamentally tractable. Recent interior-point methods can exploit problem structure to solve such problems very efficiently. Infeasible interior-point predictor–corrector methods using floating-point arithmetic sometimes compute an approximate solution with duality gap less than a given tolerance even when the problem may not have a solution. We present an efficient verification method for solving linear programming problems which computes a guaranteed enclosure of the optimal solution and which verifies the existence of the solution within the computed interval.     

Numerical methods for solving terminal optimal control problems

Numerical methods for solving optimal control problems with equality constraints at the right end of the trajectory are discussed. Algorithms for optimal control search are proposed that are based on the multimethod technique for finding an approximate solution of prescribed accuracy that satisfies terminal conditions. High accuracy is achieved by applying a second-order method analogous to Newton’s method or Bellman’s quasilinearization method. In the solution of problems with direct control constraints, the variation of the control is computed using a finite-dimensional approximation of an auxiliary problem, which is solved by applying linear programming methods.     

On the Lanczos and Golub–Kahan reduction methods applied to discrete ill‐posed problems

The symmetric Lanczos method is commonly applied to reduce large‐scale symmetric linear discrete ill‐posed problems to small ones with a symmetric tridiagonal matrix. We investigate how quickly the nonnegative subdiagonal entries of this matrix decay to zero. Their fast decay to zero suggests that there is little benefit in expressing the solution of the discrete ill‐posed problems in terms of the eigenvectors of the matrix compared with using a basis of Lanczos vectors, which are cheaper to compute. Similarly, we show that the solution subspace determined by the LSQR method when applied to the solution of linear discrete ill‐posed problems with a nonsymmetric matrix often can be used instead of the solution subspace determined by the singular value decomposition without significant, if any, reduction of the quality of the computed solution. Copyright © 2015 John Wiley & Sons, Ltd.     

A comparison of the String Gradient Weighted Moving Finite Element method and a Parabolic Moving Mesh Partial Differential Equation method for solutions of partial differential equations

We compare numerical experiments from the String Gradient Weighted Moving Finite Element method and a Parabolic Moving Mesh Partial Differential Equation method, applied to three benchmark problems based on two different partial differential equations. Both methods are described in detail and we highlight some strengths and weaknesses of each method via the numerical comparisons. The two equations used in the benchmark problems are the viscous Burgers’ equation and the porous medium equation, both in one dimension. Simulations are made for the two methods for: a) a travelling wave solution for the viscous Burgers’ equation, b) the Barenblatt selfsimilar analytical solution of the porous medium equation, and c) a waiting-time solution for the porous medium equation. Simulations are carried out for varying mesh sizes, and the numerical solutions are compared by computing errors in two ways. In the case of an analytic solution being available, the errors in the numerical solutions are computed directly from the analytic solution. In the case of no availability of an analytic solution, an approximation to the error is computed using a very fine mesh numerical solution as the reference solution.     

On the computation of a truncated SVD of a large linear discrete ill-posed problem

The singular value decomposition is commonly used to solve linear discrete ill-posed problems of small to moderate size. This decomposition not only can be applied to determine an approximate solution but also provides insight into properties of the problem. However, large-scale problems generally are not solved with the aid of the singular value decomposition, because its computation is considered too expensive. This paper shows that a truncated singular value decomposition, made up of a few of the largest singular values and associated right and left singular vectors, of the matrix of a large-scale linear discrete ill-posed problems can be computed quite inexpensively by an implicitly restarted Golub–Kahan bidiagonalization method. Similarly, for large symmetric discrete ill-posed problems a truncated eigendecomposition can be computed inexpensively by an implicitly restarted symmetric Lanczos method.     

Error estimates for large-scale ill-posed problems

The computation of an approximate solution of linear discrete ill-posed problems with contaminated data is delicate due to the possibility of severe error propagation. Tikhonov regularization seeks to reduce the sensitivity of the computed solution to errors in the data by replacing the given ill-posed problem by a nearby problem, whose solution is less sensitive to perturbation. This regularization method requires that a suitable value of the regularization parameter be chosen. Recently, Brezinski et al. (Numer Algorithms 49, 2008) described new approaches to estimate the error in approximate solutions of linear systems of equations and applied these estimates to determine a suitable value of the regularization parameter in Tikhonov regularization when the approximate solution is computed with the aid of the singular value decomposition. This paper discusses applications of these and related error estimates to the solution of large-scale ill-posed problems when approximate solutions are computed by Tikhonov regularization based on partial Lanczos bidiagonalization of the matrix. The connection between partial Lanczos bidiagonalization and Gauss quadrature is utilized to determine inexpensive bounds for a family of error estimates. In memory of Gene H. Golub. This work was supported by MIUR under the PRIN grant no. 2006017542-003 and by the University of Cagliari.     

Pathfollowing for essentially singular boundary value problems with application to the complex Ginzburg-Landau equation

We present a pathfollowing strategy based on pseudo-arclength parametrization for the solution of parameter-dependent boundary value problems for ordinary differential equations. We formulate criteria which ensure the successful application of this method for the computation of solution branches with turning points for problems with an essential singularity. The advantages of our approach result from the possibility to use efficient mesh selection, and a favorable conditioning even for problems posed on a semi-infinite interval and subsequently transformed to an essentially singular problem. This is demonstrated by a Matlab implementation of the solution method based on an adaptive collocation scheme which is well suited to solve problems of practical relevance. As one example, we compute solution branches for the complex Ginzburg-Landau equation which start from non-monotone ‘multi-bump’ solutions of the nonlinear Schrödinger equation. Following the branches around turning points, real-valued solutions of the nonlinear Schrödinger equation can easily be computed.     

A new L-curve for ill-posed problems

The truncated singular value decomposition is a popular method for the solution of linear ill-posed problems. The method requires the choice of a truncation index, which affects the quality of the computed approximate solution. This paper proposes that an L-curve, which is determined by how well the given data (right-hand side) can be approximated by a linear combination of the first (few) left singular vectors (or functions), be used as an aid for determining the truncation index.     

Second order sufficient conditions and sensitivity analysis for the optimal control of a container crane under state constraints

Stability and sensitivity analysis of parametric control problems has recently been elaborated for optimal control problems subject to pure state constraints. This paper illustrates the numerical aspects of sensitivity analysis for a complex practical example: the optimal control of a container crane with a state constraint on the vertical velocity. The multiple shooting method is used to determine a nominal solution satisfying first order necessary conditions. Second order sufficient conditions are checked by showing that an associated Riccati equation has a bounded solution. Sensitivity differentials of optimal solutions an computed with respect to variations in the swing angle