Parametric dual regularization for an optimal control problem with pointwise state constraints

The perturbation method is used in the dual regularization theory for a linear convex optimal control problem with a strongly convex objective functional and pointwise state constraints understood as ones in L ₂. Primary attention is given to the qualitative properties of the dual regularization method, depending on the differential properties of the value function (S-function) in the optimization problem. It is shown that the convergence of the method is closely related to the Lagrange principle and the Pontryagin maximum principle. The dual regularization scheme is shown to provide a new method for proving the maximum principle in the problem with pointwise state constraints understood in L ₂ or C. The regularized Lagrange principle in nondifferential form and the regularized Pontryagin maximum principle are discussed. Illustrative examples are presented.     

Approximate source conditions in Tikhonov–Phillips regularization and consequences for inverse problems with multiplication operators

The object of this paper is threefold. First, we investigate in a Hilbert space setting the utility of approximate source conditions in the method of Tikhonov–Phillips regularization for linear ill‐posed operator equations. We introduce distance functions measuring the violation of canonical source conditions and derive convergence rates for regularized solutions based on those functions. Moreover, such distance functions are verified for simple multiplication operators in L²(0, 1). The second aim of this paper is to emphasize that multiplication operators play some interesting role in inverse problem theory. In this context, we give examples of non‐linear inverse problems in natural sciences and stochastic finance that can be written as non‐linear operator equations in L²(0, 1), for which the forward operator is a composition of a linear integration operator and a non‐linear superposition operator. The Fréchet derivative of such a forward operator is a composition of a compact integration and a non‐compact multiplication operator. If the multiplier function defining the multiplication operator has zeros, then for the linearization an additional ill‐posedness factor arises. By considering the structure of canonical source conditions for the linearized problem it could be expected that different decay rates of multiplier functions near a zero, for example the decay as a power or as an exponential function, would lead to completely different ill‐posedness situations. As third we apply the results on approximate source conditions to such composite linear problems in L²(0, 1) and indicate that only integrals of multiplier functions and not the specific character of the decay of multiplier functions in a neighbourhood of a zero determine the convergence behaviour of regularized solutions. Copyright © 2005 John Wiley & Sons, Ltd.     

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

We consider an inverse quadratic programming (QP) problem in which the parameters in the objective function of a given QP problem are adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a minimization problem with a positive semidefinite cone constraint and its dual is a linearly constrained semismoothly differentiable (SC¹) convex programming problem with fewer variables than the original one. We demonstrate the global convergence of the augmented Lagrangian method for the dual problem and prove that the convergence rate of primal iterates, generated by the augmented Lagrange method, is proportional to 1/r, and the rate of multiplier iterates is proportional to  $1/\sqrt{r}$ , where r is the penalty parameter in the augmented Lagrangian. As the objective function of the dual problem is a SC¹ function involving the projection operator onto the cone of symmetrically semi-definite matrices, the analysis requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and properties of the projection operator in the symmetric-matrix space. Furthermore, the semismooth Newton method with Armijo line search is applied to solve the subproblems in the augmented Lagrange approach, which is proven to have global convergence and local quadratic rate. Finally numerical results, implemented by the augmented Lagrangian method, are reported.     

The residual method for regularizing ill-posed problems

Although the residual method, or constrained regularization, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals.We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on L^p-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.     

A Modified Tikhonov Regularization for Linear Operator Equations

We construct with the aid of regularizing filters a new class of improved regularization methods, called modified Tikhonov regularization (MTR), for solving ill-posed linear operator equations. Regularizing properties and asymptotic order of the regularized solutions are analyzed in the presence of noisy data and perturbation error in the operator. With some accurate estimates in the solution errors, optimal convergence order of the regularized solutions is obtained by a priori choice of the regularization parameter. Furthermore, numerical results are given for several ill-posed integral equations, which not only roughly coincide with the theoretical results but also show that MTR can be more accurate than ordinary Tikhonov regularization (OTR).     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

A generalized proximal-point method for convex optimization problems in Hilbert spaces

We deal with a generalization of the proximal-point method and the closely related Tikhonov regularization method for convex optimization problems. The prime motivation behind this is the well-known connection between the classical proximal-point and augmented Lagrangian methods, and the emergence of modified augmented Lagrangian methods in recent years. Our discussion includes a formal proof of a corresponding connection between the generalized proximal-point method and the modified augmented Lagrange approach in infinite dimensions. Several examples and counterexamples illustrate the convergence properties of the generalized proximal-point method and indicate that the corresponding assumptions are sharp.     

A penalty function proof of a Lagrange multiplier theorem with application to linear delay systems

Using a penalty function method, a Lagrange multiplier theorem in dual Banach spaces is proved. This theorem is applied to the optimal control of linear, autonomous time-delay systems with function space equality end condition and pointwise control restrictions. Under an additional regularity condition, the resulting Lagrange multiplier can be identified with an element ofL .     

Computational inversion of electron micrographs using L2‐gradient flows—convergence analysis

A gradient flow‐based explicit finite element method (L2GF) for reconstructing the 3D density function from a set of 2D electron micrographs has been proposed in recently published papers. The experimental results showed that the proposed method was superior to the other classical algorithms, especially for the highly noisy data. However, convergence analysis of the L2GF method has not been conducted. In this paper, we present a complete analysis on the convergence of L2GF method for the case of using a more general form regularization term, which includes the Tikhonov‐type regularizer and modified or smoothed total variation regularizer as two special cases. We further prove that the L²‐gradient flow method is stable and robust. These results demonstrate that the iterative variational reconstruction method derived from the L²‐gradient flow approach is mathematically sound and effective and has desirable properties. Copyright © 2013 John Wiley & Sons, Ltd.     

On a characterization of convergence for the Hestenes method of multipliers

The connection between the convergence of the Hestenes method of multipliers and the existence of augmented Lagrange multipliers for the constrained minimum problem (P): minimizef(x), subject tog(x)=0, is investigated under very general assumptions onX,f, andg.In the first part, we use the existence of augmented Lagrange multipliers as a sufficient condition for the convergence of the algorithm. In the second part, we prove that this is also a necessary condition for the convergence of the method and the boundedness of the sequence of the multiplier estimates.Further, we give very simple examples to show that the existence of augmented Lagrange multipliers is independent of smoothness condition onf andg. Finally, an application to the linear-convex problem is given.     

Regularization of Ill-Posed Problems by Envelope Guided Conjugate Gradients

Abstract

We propose a new way to iteratively solve large scale ill-posed problems by exploiting the relation between Tikhonov regularization and multiobjective optimization to obtain, iteratively, approximations to the Tikhonov L-curve and its corner. Monitoring the change of the approximate L-curves allows us to adjust the regularization parameter adaptively during a preconditioned conjugate gradient iteration, so that the desired solution can be reconstructed with a low number of iterations. We apply the technique to an idealized image reconstruction problem in positron emission tomography.     

Lagrange interpolation on extended generalized Jacobi nodes

We consider theL ^p -convergence of interpolatory processes for nonsmooth functions. Therefore we use generalizations of the well-known Marcinkiewicz-Zygmund inequality for trigonometric polynomials to the case of algebraic polynomials, extending a result of Y. Xu. Particularly, we obtain the order of convergence for certain Lagrange and quasi-Lagrange interpolatory processes on generalized Jacobi nodes. Our approach enables us also to discuss the influence of additional nodes near the endpoints ±1.     

Necessary and sufficient conditions for linear convergence of ℓ1‐regularization

Motivated by the theoretical and practical results in compressed sensing, efforts have been undertaken by the inverse problems community to derive analogous results, for instance linear convergence rates, for Tikhonov regularization with ℓ¹‐penalty term for the solution of ill‐posed equations. Conceptually, the main difference between these two fields is that regularization in general is an uncon strained optimization problem, while in compressed sensing a constrained one is used. Since the two methods have been developed in two different communities, the theoretical approaches to them appear to be rather different: In compressed sensing, the restricted isometry property seems to be central for proving linear convergence rates, whereas in regularization theory range or source conditions are imposed. The paper gives a common meaning to the seemingly different conditions and puts them into perspective with the conditions from the respective other community. A particularly important observation is that the range condition together with an injectivity condition is weaker than the restricted isometry property. Under the weaker conditions, linear convergence rates can be proven for compressed sensing and for Tikhonov regularization. Thus existing results from the literature can be improved based on a unified analysis. In particular, the range condition is shown to be the weakest possible condition that permits the derivation of linear convergence rates for Tikhonov regularization with a priori parameter choice. © 2010 Wiley Periodicals, Inc.     

Iterated hard-thresholding for linear inverse problems with sparsity constraints

We describe an iterative algorithm for the minimization of Tikhonov type functionals which involve sparsity constraints in form of ℓ^p -penalties which have been proposed recently for the regularization of ill-posed problems. In contrast to the well-known algorithm considered by Daubechies, Defrise and De Mol, it uses hard instead of soft thresholding. This hard thresholding algorithm is based on the generalized conditional gradient method. General results on the convergence of the generalized conditional gradient method enable us to prove strong convergence of the iterates. Furthermore we are able to establish convergence rates of O (n^–1/2) and O (λⁿ) for p = 1 and 1 < p ≤ 2 respectively. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Converse and saturation results on Tikhonov regularization together witha posteriori parameter choice

Some converse and saturation results on Tikhonov regularization of nonlinear ill-posed problems are proved and the aposteriori parameter choice yielding optimal rates of convergence is discussed Project supported by the National Natural Science Foundation of China (Grant No. 9801018).     

Convergence Rate of the Augmented Lagrangian SQP Method

In this paper, the augmented Lagrangian SQP method is considered for the numerical solution of optimization problems with equality constraints. The problem is formulated in a Hilbert space setting. Since the augmented Lagrangian SQP method is a type of Newton method for the nonlinear system of necessary optimality conditions, it is conceivable that q-quadratic convergence can be shown to hold locally in the pair (x, ). Our interest lies in the convergence of the variable x alone. We improve convergence estimates for the Newton multiplier update which does not satisfy the same convergence properties in x as for example the least-square multiplier update. We discuss these updates in the context of parameter identification problems. Furthermore, we extend the convergence results to inexact augmented Lagrangian methods. Numerical results for a control problem are also presented.     

Finite-dimensional approximation of tikhonov regularized solutions of non-linear ill-posed problems

In this paper we consider non-linear ill-posed problems F(x)=y0 in a Hilbert space setting. We solve these problems with Tikhonov regularization combined with finite-dimensional approximation where the data y0 and the non-linear operator F are assumed to be known only approximately. Conditions are given that guarantee optimal convergence rates with respect to both, the data noise and the finite-dimensional approximation. Finally, we present some numerical results for parameter estimation problems that verify the theoretical results.     

Homogenization of integrals with pointwise gradient constraints via the periodic unfolding method

Abstract The pointwise gradient constrained homogenization process, for Neumann and Dirichlet type problems, is analyzed by means of the periodic unfolding method recently introduced in [21]. Classically, the proof of the homogenization formula in presence of pointwise gradient constraints relies on elaborated measure theoretic arguments. The one proposed here is elementary: it is based on weak convergence arguments in L^p spaces, coupled with suitable regularization techniques. Keywords: Homogenization, Gradient constrained problems, Periodic unfolding method Mathematics Subject Classification (2000): 49J45, 35B27, 74Q05     

Optimal Control of PDEs with Regularized Pointwise State Constraints

This paper addresses the regularization of pointwise state constraints in optimal control problems. By analyzing the associated dual problem, it is shown that the regularized problems admit Lagrange multipliers in L²-spaces. Under a certain boundedness assumption, the solution of the regularized problem converges to the one of the original state constrained problem. The results of our analysis are confirmed by numerical tests. Supported by the DFG Research Center “Mathematics for key technologies” (FZT 86) in Berlin.     

The convergence of an interior point method for an elliptic control problem with mixed control-state constraints

The paper addresses a primal interior point method for state-constrained PDE optimal control problems in function space. By a Lavrentiev regularization, the state constraint is transformed to a mixed control-state constraint with bounded Lagrange multiplier. Existence and convergence of the central path are established, and linear convergence of a short-step pathfollowing method is shown. The behaviour of the method is demonstrated by numerical examples. Research supported by the DFG Research Center “Mathematics for key technologies” (Matheon) in Berlin.