Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters

In this paper, we study the multi-parameter Tikhonov regularization method which adds multiple different penalties to exhibit multi-scale features of the solution. An optimal error bound of the regularization solution is obtained by a priori choice of multiple regularization parameters. Some theoretical results of the regularization solution about the dependence on regularization parameters are presented. Then, an a posteriori parameter choice, i.e., the damped Morozov discrepancy principle, is introduced to determine multiple regularization parameters. Five model functions, i.e., two hyperbolic model functions, a linear model function, an exponential model function and a logarithmic model function, are proposed to solve the damped Morozov discrepancy principle. Furthermore, four efficient model function algorithms are developed for finding reasonable multiple regularization parameters, and their convergence properties are also studied. Numerical results of several examples show that the damped discrepancy principle is competitive with the standard one, and the model function algorithms are efficient for choosing regularization parameters.     

Ill-Posedness of sublinear minimization problems

It is well known that minimization problems involving sublinear regularization terms are ill-posed, in Sobolev spaces. Extended results to spaces of bounded variation functions BV were recently showed in the special case of bounded regularization terms. In this note, a generalization to sublinear regularization is presented in BV spaces. Notice that our results are optimal in the sense that linear regularization leads to well-posed minimization problems in BV spaces.     

The residual method for regularizing ill-posed problems

Although the residual method, or constrained regularization, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals.We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on L^p-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.     

General regularization schemes for signal detection in inverse problems

The authors discuss how general regularization schemes, in particular, linear regularization schemes and projection schemes, can be used to design tests for signal detection in statistical inverse problems. It is shown that such tests can attain the minimax separation rates when the regularization parameter is chosen appropriately. It is also shown how to modify these tests in order to obtain a test which adapts (up to a log log factor) to the unknown smoothness in the alternative. Moreover, the authors discuss how the so-called direct and indirect tests are related in terms of interpolation properties.     

Recovering the initial distribution for space-fractional diffusion equation by a logarithmic regularization method

In this paper, the backward problem for space-fractional diffusion equation is investigated. We proposed a so-called logarithmic regularization method to solve it. Based on the conditional stability and an a posteriori regularization parameter choice rule, the convergence rate estimates are given under a-priori bound assumption for the exact solution.     

A sharp nonasymptotic bound and phase diagram of L 1/2 regularization

We derive a sharp nonasymptotic bound of parameter estimation of the L1/2 regularization.The bound shows that the solutions of the L1/2 regularization can achieve a loss within logarithmic factor of an ideal mean squared error and therefore underlies the feasibility and effectiveness of the L1/2regularization.Interestingly,when applied to compressive sensing,the L1/2 regularization scheme has exhibited a very promising capability of completed recovery from a much less sampling information.As compared with the Lp(0 p 1) penalty,it is appeared that the L1/2 penalty can always yield the most sparse solution among all the Lp penalty when 1/2 ≤ p 1,and when 0 p 1/2,the Lp penalty exhibits the similar properties as the L1/2 penalty.This suggests that the L1/2 regularization scheme can be accepted as the best and therefore the representative of all the Lp(0 p 1) regularization schemes.     

A smoothing and regularization Broyden-like method for nonlinear inequalities

In this paper, we consider the smoothing and regularization Broyden-like algorithm for the system of nonlinear inequalities. By constructing a new smoothing function $\phi(\mu,a)=\frac{1}{2}(a+\mu(\ln2+\ln(1+\cosh\frac{a}{\mu})))$ , the problem is approximated via a family of parameterized smooth equations H(μ,ε,x)=0. A smoothing and regularization Broyden-like algorithm with a non-monotone linear search is proposed for solving the system of nonlinear inequalities based on the new smoothing function. The global convergence of the algorithm is established under suitable assumptions. In addition, the smoothing parameter μ and the regularization parameter ε in our algorithm are viewed as two different independent variables. Preliminary numerical results show the efficiency of the algorithm and reveal that the regularization parameter ε in our algorithm plays an important role in numerical improvement, hence, our algorithm seems to be simpler and more easily implemented compared to many previous methods.     

A Tikhonov-type method for solving a multidimensional inverse heat source problem in an unbounded domain

In this study we prove a stability estimate for an inverse heat source problem in the n-dimensional case. We present a revised generalized Tikhonov regularization and obtain an error estimate. Numerical experiments for the one-dimensional and two-dimensional cases show that the revised generalized Tikhonov regularization works well.     

ON A REGULARIZATION OF INDEX 2 DIFFERENTIAL-ALGEBRAIC EQUATIONS WITH PROPERLY STATED LEADING TERM

In this article, linear regular index 2 DAEs A(t)[D(t)x(t)]′+B(t)x(t)=q(t) are considered. Using a decoupling technique, initial condition and boundary condition are properly formulated. Regular index 1 DAEs are obtained by a regularization method. We study the behavior of the solution of the regularization system via asymptotic expansions. The error analysis between the solutions of the DAEs and its regularization system is given.     

Balancing principle in supervised learning for a general regularization scheme

We discuss the problem of parameter choice in learning algorithms generated by a general regularization scheme. Such a scheme covers well-known algorithms as regularized least squares and gradient descent learning. It is known that in contrast to classical deterministic regularization methods, the performance of regularized learning algorithms is influenced not only by the smoothness of a target function, but also by the capacity of a space, where regularization is performed. In the infinite dimensional case the latter one is usually measured in terms of the effective dimension. In the context of supervised learning both the smoothness and effective dimension are intrinsically unknown a priori. Therefore we are interested in a posteriori regularization parameter choice, and we propose a new form of the balancing principle. An advantage of this strategy over the known rules such as cross-validation based adaptation is that it does not require any data splitting and allows the use of all available labeled data in the construction of regularized approximants. We provide the analysis of the proposed rule and demonstrate its advantage in simulations.     

Numerical differentiation procedures for non-exact data

The numerical differentiation of data divides naturally into two distinct problems:

1. the differentiation of exact data, and
2. the differentiation of non-exact (experimental) data.

In this paper, we examine the latter. Because methods developed for exact data are based on abstract formalisms which are independent of the structure within the data, they prove, except for the regularization procedure of Cullum, to be unsatisfactory for non-exact data. We therefore adopt the point of view that satisfactory methods for non-exact data must take the structure within the data into account in some natural way, and use the concepts of regression and spectrum analysis as a basis for the development of such methods. The regression procedure is used when either the structure within the non-exact data is known on independent grounds, or the assumptions which underlie the spectrum analysis procedure [viz., stationarity of the (detrended) data] do not apply. In this latter case, the data could be modelled using splines. The spectrum analysis procedure is used when the structure within the nonexact data (or a suitable transformation of it, where the transformation can be differentiated exactly) behaves as if it were generated by a stationary stochastic process. By proving that the regularization procedure of Cullum is equivalent to a certain spectrum analysis procedure, we derive a fast Fourier transform implementation for regularization (based on this equivalence) in which an acceptable value of the regularization parameter is estimated directly from a time series formulation based on this equivalence. Compared with the regularization procedure, which involvesO(n ³) operations (wheren is the number of data points), the fast Fourier transform implementation only involvesO(n logn).     

On the condition number of the antireflective transform

Deconvolution problems with a finite observation window require appropriate models of the unknown signal in order to guarantee uniqueness of the solution. For this purpose it has recently been suggested to impose some kind of antireflectivity of the signal. With this constraint, the deconvolution problem can be solved with an appropriate modification of the fast sine transform, provided that the convolution kernel is symmetric. The corresponding transformation is called the antireflective transform. In this work we determine the condition number of the antireflective transform to first order, and use this to show that the so-called reblurring variant of Tikhonov regularization for deconvolution problems is a regularization method. Moreover, we establish upper bounds for the regularization error of the reblurring strategy that hold uniformly with respect to the size n of the algebraic system, even though the condition number of the antireflective transform grows with n. We briefly sketch how our results extend to higher space dimensions.     

Iterative methods for the reconstruction of astronomical images with high dynamic range

In most cases astronomical images contain objects with very different intensities such as bright stars combined with faint nebulae. Since the noise is mainly due to photon counting (Poisson noise), the signal-to-noise ratio may be very different in different regions of the image. Moreover, the bright and faint objects have, in general, different angular scales. These features imply that the iterative methods which are most frequently used for the reconstruction of astronomical images, namely the Richardson–Lucy Method (RLM), also known in tomography as Expectation Maximization (EM) method, and the Iterative Space Reconstruction Algorithm (ISRA) do not work well in these cases. Also standard regularization approaches do not provide satisfactory results since a kind of adaptive regularization is required, in the sense that one needs a different regularization for bright and faint objects. In this paper we analyze a number of regularization functionals with this particular kind of adaptivity and we propose a simple modification of RLM and ISRA which takes into account these regularization terms. The preliminary results on a test object are promising.     

Lavrentiev's regularization method for nonlinear ill-posed equations in Banach spaces

In this paper, we deal with nonlinear ill-posed problems involving m-accretive mappings in Banach spaces. We consider a derivative and inverse free method for the implementation of Lavrentiev regularization method. Using general H¨older type source condition we obtain an optimal order error estimate. Also we consider the adaptive parameter choice strategy proposed by Pereverzev and Schock(2005) for choosing the regularization parameter.     

Regularized robust optimization: the optimal portfolio execution case

An uncertainty set is a crucial component in robust optimization. Unfortunately, it is often unclear how to specify it precisely. Thus it is important to study sensitivity of the robust solution to variations in the uncertainty set, and to develop a method which improves stability of the robust solution. In this paper, to address these issues, we focus on uncertainty in the price impact parameters in an optimal portfolio execution problem. We first illustrate that a small variation in the uncertainty set may result in a large change in the robust solution. We then propose a regularized robust optimization formulation which yields a solution with a better stability property than the classical robust solution. In this approach, the uncertainty set is regularized through a regularization constraint, defined by a linear matrix inequality using the Hessian of the objective function and a regularization parameter. The regularized robust solution is then more stable with respect to variation in the uncertainty set specification, in addition to being more robust to estimation errors in the price impact parameters. The regularized robust optimal execution strategy can be computed by an efficient method based on convex optimization. Improvement in the stability of the robust solution is analyzed. We also study implications of the regularization on the optimal execution strategy and its corresponding execution cost. Through the regularization parameter, one can adjust the level of conservatism of the robust solution.     

Learning rates for least square regressions with coefficient regularization

We analyze the learning rates for the least square regression with data dependent hypothesis spaces and coefficient regularization algorithms based on general kernels. Under a very mild regularity condition on the regression function, we obtain a bound for the approximation error by estimating the corresponding K-functional. Combining this estimate with the previous result of the sample error, we derive a dimensional free learning rate by the proper choice of the regularization parameter.     

Convergence analysis for the Cauchy problem of Laplace’s equation by a regularized method of fundamental solutions

In this paper, we consider the Cauchy problem of Laplace’s equation in the neighborhood of a circle. The method of fundamental solutions (MFS) combined with the discrete Tikhonov regularization is applied to obtain a regularized solution from noisy Cauchy data. Under the suitable choices of a regularization parameter and an a priori assumption to the Cauchy data, we obtain a convergence result for the regularized solution. Numerical experiments are presented to show the effectiveness of the proposed method.     

Lagrange multipliers, (exact) regularization and error bounds for monotone variational inequalities

We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stopping criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one.     

On a regularization of the compressible Euler equations for an isothermal gas

We consider a Leray-type regularization of the compressible Euler equations for an isothermal gas. The regularized system depends on a small parameter α>0. Using Riemann invariants, we prove the existence of smooth solutions for the regularized system for every α>0. The regularization mechanism is a non-linear bending of characteristics that prevents their finite-time crossing. We prove that, in the α→0 limit, the regularized solutions converge strongly. However, based on our analysis and numerical simulations, the limit is not the unique entropy solution of the Euler equations. The numerical method used to support this claim is derived from the Riemann invariants for the regularized system. This method is guaranteed to preserve the monotonicity of characteristics.     

On Blattner's formula for the discrete series representations of SO(2n, 1)

Blattner's conjecture gives a formula for the multiplicity with which a unitary irreducible representation of the maximal compact subgroup K appears in any discrete series representation of a semisimple Lie group G. We give an elementary derivation of this formula from Harish Chandra's character formula for G ～- SO_e(2n, 1) (n ? 2). The idea is to regularize the character (on the Cartan subgroup), to show that the regularization is unique, and to derive the multiplicities by expanding the resulting distribution in a Fourier series. To prove uniqueness of the regularization one uses a priori constraints on the multiplicities (=Fourier coefficients) that follow from the subquotient theorem.