Minimization of Linear Functionals Defined on Solutions of Large-Scale Discrete Ill-Posed Problems

The minimization of linear functionals defined on the solutions of discrete ill-posed problems arises, e.g., in the computation of confidence intervals for these solutions. In 1990, Eldén proposed an algorithm for this minimization problem based on a parametric programming reformulation involving the solution of a sequence of trust-region problems, and using matrix factorizations. In this paper, we describe MLFIP, a large-scale version of this algorithm where a limited-memory trust-region solver is used on the subproblems. We illustrate the use of our algorithm in connection with an inverse heat conduction problem. AMS subject classification (2000) 65F22     

Adaptive Algorithm for Constrained Least-Squares Problems

This paper is concerned with the implementation and testing of an algorithm for solving constrained least-squares problems. The algorithm is an adaptation to the least-squares case of sequential quadratic programming (SQP) trust-region methods for solving general constrained optimization problems. At each iteration, our local quadratic subproblem includes the use of the Gauss–Newton approximation but also encompasses a structured secant approximation along with tests of when to use this approximation. This method has been tested on a selection of standard problems. The results indicate that, for least-squares problems, the approach taken here is a viable alternative to standard general optimization methods such as the Byrd–Omojokun trust-region method and the Powell damped BFGS line search method.     

Inverse problems for regularization matrices

Discrete ill-posed problems are difficult to solve, because their solution is very sensitive to errors in the data and to round-off errors introduced during the solution process. Tikhonov regularization replaces the given discrete ill-posed problem by a nearby penalized least-squares problem whose solution is less sensitive to perturbations. The penalization term is defined by a regularization matrix, whose choice may affect the quality of the computed solution significantly. We describe several inverse matrix problems whose solution yields regularization matrices adapted to the desired solution. Numerical examples illustrate the performance of the regularization matrices determined.     

Preconditioning and globalizing conjugate gradients in dual space for quadratically penalized nonlinear-least squares problems

When solving nonlinear least-squares problems, it is often useful to regularize the problem using a quadratic term, a practice which is especially common in applications arising in inverse calculations. A solution method derived from a trust-region Gauss-Newton algorithm is analyzed for such applications, where, contrary to the standard algorithm, the least-squares subproblem solved at each iteration of the method is rewritten as a quadratic minimization subject to linear equality constraints. This allows the exploitation of duality properties of the associated linearized problems. This paper considers a recent conjugate-gradient-like method which performs the quadratic minimization in the dual space and produces, in exact arithmetic, the same iterates as those produced by a standard conjugate-gradients method in the primal space. This dual algorithm is computationally interesting whenever the dimension of the dual space is significantly smaller than that of the primal space, yielding gains in terms of both memory usage and computational cost. The relation between this dual space solver and PSAS (Physical-space Statistical Analysis System), another well-known dual space technique used in data assimilation problems, is explained. The use of an effective preconditioning technique is proposed and refined convergence bounds derived, which results in a practical solution method. Finally, stopping rules adequate for a trust-region solver are proposed in the dual space, providing iterates that are equivalent to those obtained with a Steihaug-Toint truncated conjugate-gradient method in the primal space.     

TRESNEI, a Matlab trust-region solver for systems of?nonlinear equalities and inequalities

The Matlab implementation of a trust-region Gauss-Newton method for bound-constrained nonlinear least-squares problems is presented. The solver, called TRESNEI, is adequate for zero and small-residual problems and handles the solution of nonlinear systems of equalities and inequalities. The structure and the usage of the solver are described and an extensive numerical comparison with functions from the Matlab Optimization Toolbox is carried out.     

A line search trust-region algorithm with nonmonotone adaptive radius for a system of nonlinear equations

In this paper, a trust-region procedure is proposed for the solution of nonlinear equations. The proposed approach takes advantages of an effective adaptive trust-region radius and a nonmonotone strategy by combining both of them appropriately. It is believed that selecting an appropriate adaptive radius based on a suitable nonmonotone strategy can improve the efficiency and robustness of the trust-region frameworks as well as decrease the computational cost of the algorithm by decreasing the required number subproblems that must be solved. The global convergence and the local Q-quadratic convergence rate of the proposed approach are proved. Preliminary numerical results of the proposed algorithm are also reported which indicate the promising behavior of the new procedure for solving the nonlinear system.     

An a posteriori parameter choice for simplified regularization of ill-posed problems

An a posteriori parameter choice strategy is proposed for the simplified regularization of ill-posed problems where no information about the smoothness of the unknown solution is required. If the smoothness of the solution is known then, as a particular case, the optimal rate is achieved. Our result also includes a recent result of Guacanme (1990).     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

On the convergence of an inexact Gauss–Newton trust-region method for nonlinear least-squares problems with simple bounds

We introduce an inexact Gauss–Newton trust-region method for solving bound-constrained nonlinear least-squares problems where, at each iteration, a trust-region subproblem is approximately solved by the Conjugate Gradient method. Provided a suitable control on the accuracy to which we attempt to solve the subproblems, we prove that the method has global and asymptotic fast convergence properties. Some numerical illustration is also presented.     

Large-scale Tikhonov regularization via reduction by orthogonal projection

This paper presents a new approach to computing an approximate solution of Tikhonov-regularized large-scale ill-posed least-squares problems with a general regularization matrix. The iterative method applies a sequence of projections onto generalized Krylov subspaces. A suitable value of the regularization parameter is determined by the discrepancy principle.     

On the balancing principle for some problems of Numerical Analysis

We discuss a choice of weight in penalization methods. The motivation for the use of penalization in computational mathematics is to improve the conditioning of the numerical solution. One example of such improvement is a regularization, where a penalization substitutes an ill-posed problem for a well-posed one. In modern numerical methods for PDEs a penalization is used, for example, to enforce a continuity of an approximate solution on non-matching grids. A choice of penalty weight should provide a balance between error components related with convergence and stability, which are usually unknown. In this paper we propose and analyze a simple adaptive strategy for the choice of penalty weight which does not rely on a priori estimates of above mentioned components. It is shown that under natural assumptions the accuracy provided by our adaptive strategy is worse only by a constant factor than one could achieve in the case of known stability and convergence rates. Finally, we successfully apply our strategy for self-regularization of Volterra-type severely ill-posed problems, such as the sideways heat equation, and for the choice of a weight in interior penalty discontinuous approximation on non-matching grids. Numerical experiments on a series of model problems support theoretical results.     

Adaptive Strategy for the Damping Parameters in an Iteratively Regularized Gauss–Newton Method

In this paper, a convergence analysis of an adaptive choice of the sequence of damping parameters in the iteratively regularized Gauss–Newton method for solving nonlinear ill-posed operator equations is presented. The selection criterion is motivated from the damping parameter choice criteria, which are used for the efficient solution of nonlinear least-square problems. The performance of this selection criterion is tested for the solution of nonlinear ill-posed model problems.     

THE QUASI-BOUNDARY VALUE METHOD FOR IDENTIFYING THE INITIAL VALUE OF THE SPACE-TIME FRACTIONAL DIFFUSION EQUATION

In this article, we consider to solve the inverse initial value problem for an inhomogeneous space-time fractional diffusion equation. This problem is ill-posed and the quasi-boundary value method is proposed to deal with this inverse problem and obtain the series expression of the regularized solution for the inverse initial value problem. We prove the error estimates between the regularization solution and the exact solution by using an a priori regularization parameter and an a posteriori regularization parameter choice rule. Some numerical results in one-dimensional case and two-dimensional case show that our method is effcient and stable.     

On efficiently combining limited-memory and trust-region techniques

Limited-memory quasi-Newton methods and trust-region methods represent two efficient approaches used for solving unconstrained optimization problems. A straightforward combination of them deteriorates the efficiency of the former approach, especially in the case of large-scale problems. For this reason, the limited-memory methods are usually combined with a line search. We show how to efficiently combine limited-memory and trust-region techniques. One of our approaches is based on the eigenvalue decomposition of the limited-memory quasi-Newton approximation of the Hessian matrix. The decomposition allows for finding a nearly-exact solution to the trust-region subproblem defined by the Euclidean norm with an insignificant computational overhead as compared with the cost of computing the quasi-Newton direction in line-search limited-memory methods. The other approach is based on two new eigenvalue-based norms. The advantage of the new norms is that the trust-region subproblem is separable and each of the smaller subproblems is easy to solve. We show that our eigenvalue-based limited-memory trust-region methods are globally convergent. Moreover, we propose improved versions of the existing limited-memory trust-region algorithms. The presented results of numerical experiments demonstrate the efficiency of our approach which is competitive with line-search versions of the L-BFGS method.     

A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level

We consider Tikhonov regularization of linear ill-posed problems with noisy data. The choice of the regularization parameter by classical rules, such as discrepancy principle, needs exact noise level information: these rules fail in the case of an underestimated noise level and give large error of the regularized solution in the case of very moderate overestimation of the noise level. We propose a general family of parameter choice rules, which includes many known rules and guarantees convergence of approximations. Quasi-optimality is proved for a sub-family of rules. Many rules from this family work well also in the case of many times under- or overestimated noise level. In the case of exact or overestimated noise level we propose to take the regularization parameter as the minimum of parameters from the post-estimated monotone error rule and a certain new rule from the proposed family. The advantages of the new rules are demonstrated in extensive numerical experiments.     

Iterative processes of Fejér type in ill-posed problems with a priori information

In the latter thirty years, the solution of ill-posed problems with a priori information formed a separate field of research in the theory of ill-posed problems. We mean the class of problems, where along with the basic equation one has some additional data on the desired solution. Namely, one states some relations and constraints which contain important information on the object under consideration. As a rule, taking into account these data in a solution algorithm, one can essentially increase its accuracy for solving ill-posed (unstable) problems. It is especially important in the solution of applied problems in the case when a solution is not unique, because this approach allows one to choose a solution that meets the reality. In this paper we survey the methods for solving such problems. We briefly describe all relevant approaches (known to us), but we pay the main attention to the method proposed by us. This technique is based on the application of iterative processes of Fejér type which admit a flexible and effective realization for a wide class of a priori constraints.     

二维Helmholtz方程不适定问题的一种算子软化正则法

本文研究带有混合边界的二维Helmholtz方程不适定问题.为了获得稳定的数值解，利用基于de la ValléePoussin算子的软化正则方法，得到了正则近似解，给出正则近似解与精确解之间在先验参数选取规则之下的误差估计，并通过数值实验检验了数据有噪声扰动时方法的有效性和稳定性.     

On Lanczos Based Methods for the Regularization of Discrete Ill-Posed Problems

We study hybrid methods for the solution of linear ill-posed problems. Hybrid methods are based on he Lanczos process, which yields a sequence of small bidiagonal systems approximating the original ill-posed problem. In a second step, some additional regularization, typically the truncated SVD, is used to stabilize the iteration. We investigate two different hybrid methods and interpret these schemes as well-known projection methods, namely least-squares projection and the dual least-squares method. Numerical results are provided to illustrate the potential of these methods. This gives interesting insight in to the behavior of hybrid methods in practice.This revised version was published online in October 2005 with corrections to the Cover Date.     

A trust-region method with a conic model for unconstrained optimization

In this paper, we propose and analyze a new conic trust-region algorithm for solving the unconstrained optimization problems. A new strategy is proposed to construct the conic model and the relevant conic trust-region subproblems are solved by an approximate solution method. This approximate solution method is not only easy to implement but also preserves the strong convergence properties of the exact solution methods. Under reasonable conditions, the locally linear and superlinear convergence of the proposed algorithm is established. The numerical experiments show that this algorithm is both feasible and efficient. Copyright © 2008 John Wiley & Sons, Ltd.     

Globally convergent DC trust-region methods

In this paper, we investigate the use of DC (Difference of Convex functions) models and algorithms in the application of trust-region methods to the solution of a class of nonlinear optimization problems where the constrained set is closed and convex (and, from a practical point of view, where projecting onto the feasible region is computationally affordable). We consider DC local models for the quadratic model of the objective function used to compute the trust-region step, and apply a primal-dual subgradient method to the solution of the corresponding trust-region subproblems. One is able to prove that the resulting scheme is globally convergent to first-order stationary points. The theory requires the use of exact second-order derivatives but, in turn, the computation of the trust-region step asks only for one projection onto the feasible region (in comparison to the calculation of the generalized Cauchy point which may require more). The numerical efficiency and robustness of the proposed new scheme when applied to bound-constrained problems is measured by comparing its performance against some of the current state-of-the-art nonlinear programming solvers on a vast collection of test problems.