A parameter estimation method for model analysis

To solve a class of nonlinear parameter estimation problems, a method combining the regularized structured nonlinear total least norm (RSNTLN) method and parameter separation scheme is suggested. The method guarantees the convergence of parameters and has an advantages in reducing the residual norm over the use of RSNTLN only. Numerical experiments for two models appeared in signal processing show that the suggested method is more effective in obtaining solution and parameter with minimum residual norm.     

Averaging operators for exponential splittings

Averaging operations are considered in connection with exponential splitting methods. Toeplitz plus Hankel related matrices are resplit by applying appropriate averaging operators leading to a hierarchy of structured matrices. With the resulting parts, the option of using exponential splitting methods becomes available. A related, seemingly important group of unitary unipotents is looked at. Based on a formula due to Lenard, a very fast iterative method to find the nearest Toeplitz plus Hankel matrix in the Frobenius norm is devised.     

Tensor properties of multilevel Toeplitz and related matrices

A general proposal is presented for fast algorithms for multilevel structured matrices. It is based on investigation of their tensor properties and develops the idea recently introduced by Kamm and Nagy in the block Toeplitz case. We show that tensor properties of multilevel Toeplitz matrices are related to separation of variables in the corresponding symbol, present analytical tools to study the latter, expose truncation algorithms preserving the structure, and report on some numerical results confirming advantages of the proposal.     

Regularizers for structured sparsity

We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vector is sparse. The combinatorial problem of selecting the nonzero components of this vector can be “relaxed” by regularizing the squared error with a convex penalty function like the ?₁ norm. However, in many applications, additional conditions on the structure of the regression vector and its sparsity pattern are available. Incorporating this information into the learning method may lead to a significant decrease of the estimation error. In this paper, we present a family of convex penalty functions, which encode prior knowledge on the structure of the vector formed by the absolute values of the regression coefficients. This family subsumes the ?₁ norm and is flexible enough to include different models of sparsity patterns, which are of practical and theoretical importance. We establish the basic properties of these penalty functions and discuss some examples where they can be computed explicitly. Moreover, we present a convergent optimization algorithm for solving regularized least squares with these penalty functions. Numerical simulations highlight the benefit of structured sparsity and the advantage offered by our approach over the Lasso method and other related methods.     

Minimal Zero Norm Solutions of Linear Complementarity Problems

In this paper, we study minimal zero norm solutions of the linear complementarity problems, defined as the solutions with smallest cardinality. Minimal zero norm solutions are often desired in some real applications such as bimatrix game and portfolio selection. We first show the uniqueness of the minimal zero norm solution for Z-matrix linear complementarity problems. To find minimal zero norm solutions is equivalent to solve a difficult zero norm minimization problem with linear complementarity constraints. We then propose a p norm regularized minimization model with p in the open interval from zero to one, and show that it can approximate minimal zero norm solutions very well by sequentially decreasing the regularization parameter. We establish a threshold lower bound for any nonzero entry in its local minimizers, that can be used to identify zero entries precisely in computed solutions. We also consider the choice of regularization parameter to get desired sparsity. Based on the theoretical results, we design a sequential smoothing gradient method to solve the model. Numerical results demonstrate that the sequential smoothing gradient method can effectively solve the regularized model and get minimal zero norm solutions of linear complementarity problems.     

Two regularization methods for a Cauchy problem for the Laplace equation

A Cauchy problem for the Laplace equation in a rectangle is considered. Cauchy data are given for y=0, and boundary data are for x=0 and x=π. The solution for 0<y?1 is sought. We propose two different regularization methods on the ill-posed problem based on separation of variables. Both methods are applied to formulate regularized solutions which are stably convergent to the exact one with explicit error estimates.     

Fast Structured Matrix Computations: Tensor Rank and Cohn–Umans Method

We discuss a generalization of the Cohn–Umans method, a potent technique developed for studying the bilinear complexity of matrix multiplication by embedding matrices into an appropriate group algebra. We investigate how the Cohn–Umans method may be used for bilinear operations other than matrix multiplication, with algebras other than group algebras, and we relate it to Strassen’s tensor rank approach, the traditional framework for investigating bilinear complexity. To demonstrate the utility of the generalized method, we apply it to find the fastest algorithms for forming structured matrix–vector product, the basic operation underlying iterative algorithms for structured matrices. The structures we study include Toeplitz, Hankel, circulant, symmetric, skew-symmetric, f-circulant, block Toeplitz–Toeplitz block, triangular Toeplitz matrices, Toeplitz-plus-Hankel, sparse/banded/triangular. Except for the case of skew-symmetric matrices, for which we have only upper bounds, the algorithms derived using the generalized Cohn–Umans method in all other instances are the fastest possible in the sense of having minimum bilinear complexity. We also apply this framework to a few other bilinear operations including matrix–matrix, commutator, simultaneous matrix products, and briefly discuss the relation between tensor nuclear norm and numerical stability.     

Deconvolution and Regularization with Toeplitz Matrices

By deconvolution we mean the solution of a linear first-kind integral equation with a convolution-type kernel, i.e., a kernel that depends only on the difference between the two independent variables. Deconvolution problems are special cases of linear first-kind Fredholm integral equations, whose treatment requires the use of regularization methods. The corresponding computational problem takes the form of structured matrix problem with a Toeplitz or block Toeplitz coefficient matrix. The aim of this paper is to present a tutorial survey of numerical algorithms for the practical treatment of these discretized deconvolution problems, with emphasis on methods that take the special structure of the matrix into account. Wherever possible, analogies to classical DFT-based deconvolution problems are drawn. Among other things, we present direct methods for regularization with Toeplitz matrices, and we show how Toeplitz matrix–vector products are computed by means of FFT, being useful in iterative methods. We also introduce the Kronecker product and show how it is used in the discretization and solution of 2-D deconvolution problems whose variables separate.     

Toeplitz矩阵填充的四种流形逼近算法比较

 本文提出Toeplitz矩阵填充的四种流形逼近算法。在左奇异向量空间中对已知部分运用最小二乘法逼近,形成新的可行矩阵;并将对角线上的元素分别用均值,l₁范数,l_∞范数和中间数四种方法逼近使得迭代后的矩阵仍保持Toeplitz结构,节约了奇异向量空间的分解时间。最终找到合理的低秩矩阵来逼近未知的高秩矩阵,进而精确地完成Toeplitz矩阵的填充。理论上,分析了在一定条件下算法的收敛性。实验上,通过取不同的采样密度进行数值实验展示了四种算法的优劣。实验结果说明均值算法和l_∞范数算法大多用的时间较少,但是当采样密度和矩阵规模较大时,中间数算法的精度较高。     

Proximal methods for the latent group lasso penalty

We consider a regularized least squares problem, with regularization by structured sparsity-inducing norms, which extend the usual ? ₁ and the group lasso penalty, by allowing the subsets to overlap. Such regularizations lead to nonsmooth problems that are difficult to optimize, and we propose in this paper a suitable version of an accelerated proximal method to solve them. We prove convergence of a nested procedure, obtained composing an accelerated proximal method with an inner algorithm for computing the proximity operator. By exploiting the geometrical properties of the penalty, we devise a new active set strategy, thanks to which the inner iteration is relatively fast, thus guaranteeing good computational performances of the overall algorithm. Our approach allows to deal with high dimensional problems without pre-processing for dimensionality reduction, leading to better computational and prediction performances with respect to the state-of-the art methods, as shown empirically both on toy and real data.     

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current-voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.     

Eigenvalue Characterization of Radial Operators on Weighted Bergman Spaces Over the Unit Ball

We study the so-called radial operators, and in particular radial Toeplitz operators, acting on the standard weighted Bergman space on the unit ball in ${\mathbb{C}^n}$ . They turn out to be diagonal with respect to the standard monomial basis, and the elements of their eigenvalue sequences depend only on the length of multi-indexes enumerating basis elements. We explicitly characterize the eigenvalue sequences of radial Toeplitz operators by giving a solution for the weighted extension of the classical Hausdorff moment problem, and show that the norm closure of the set of all radial Toeplitz operators with bounded measurable radial symbols coincides with the C*-algebra generated by these Toeplitz operators and is isomorphic and isometric to the C*-algebra of sequences that slowly oscillate in the sense of Schmidt.     

Approximation of Solution Components for Ill-Posed Problems by the Tikhonov Method with Total Variation

An ill-posed problem in the form of a linear operator equation given on a pair of Banach spaces is considered. Its solution is representable as a sum of a smooth and a discontinuous component. A stable approximation of the solution is obtained using a modified Tikhonov method in which the stabilizer is constructed as a sum of the Lebesgue norm and total variation. Each of the functionals involved in the stabilizer depends only on one component and takes into account its properties. Theorems on the componentwise convergence of the regularization method are stated, and a general scheme for the finite-difference approximation of the regularized family of approximate solutions is substantiated in the n-dimensional case.     

Efficient Pattern Matching on Graph Patterns of Bounded Treewidth

This paper deals with a problem to decide whether a given graph structure appears as a pattern in the structure of a given graph. A graph pattern is a connected graph with structured variables. A variable is an ordered list of vertices that can be replaced with a connected graph by a kind of hyperedge replacements. The graph pattern matching problem (GPMP) is the computational problem to decide whether a given graph pattern matches a given graph. In this paper, we show that GPMP is solvable in polynomial time if for a given graph pattern p, the lengths of all variables of p are 2 and the base graph of p is of bounded treewidth.     

Differential variational inequality approach to dynamic games with shared constraints

The dynamic Nash equilibrium problem with shared constraints (NEPSC) involves a dynamic decision process with multiple players, where not only the players’ cost functionals but also their admissible control sets depend on the rivals’ decision variables through shared constraints. For a class of the dynamic NEPSC, we propose a differential variational inequality formulation. Using this formulation, we show the existence of solutions of the dynamic NEPSC, and develop a regularized smoothing method to find a solution of it. We prove that the regularized smoothing method converges to the least norm solution of the differential variational inequality, which is a solution of the dynamic NEPSC as the regularization parameter \(\lambda \) and smoothing parameter \(\mu \) go to zero with the order \(\mu =o(\lambda )\) . Numerical examples are given to illustrate the existence and convergence results.     

Nouveau noyau de Green associé au problème de Poisson–Dirichlet sur un rectangle

This work is focused on the study of a ‘discretization’ method for the Laplacian operator, in the two-dimensional Poisson problem on a rectangle, with Dirichlet boundary conditions. The Laplacian operator is approximated by a block Toeplitz matrix, the blocks of which are Toeplitz matrices again, and a formula of the inverse matrix blocks is given. Then an asymptotic development of the inverse matrix trace and the Toeplitz matrix determinant are obtained. Finally, the continuum expression of the Laplacian operator is found by calculating the ergodic limit of the inverse matrix. A new asymptotic formula for the well known Green function for the Poisson problem that we obtain converges more rapidly than the usual one. To cite this article: J. Chanzy, C. R. Acad. Sci. Paris, Ser. I 341 (2005).     

On HSS and AHSS iteration methods for nonsymmetric positive definite Toeplitz systems

Two iteration methods are proposed to solve real nonsymmetric positive definite Toeplitz systems of linear equations. These methods are based on Hermitian and skew-Hermitian splitting (HSS) and accelerated Hermitian and skew-Hermitian splitting (AHSS). By constructing an orthogonal matrix and using a similarity transformation, the real Toeplitz linear system is transformed into a generalized saddle point problem. Then the structured HSS and the structured AHSS iteration methods are established by applying the HSS and the AHSS iteration methods to the generalized saddle point problem. We discuss efficient implementations and demonstrate that the structured HSS and the structured AHSS iteration methods have better behavior than the HSS iteration method in terms of both computational complexity and convergence speed. Moreover, the structured AHSS iteration method outperforms the HSS and the structured HSS iteration methods. The structured AHSS iteration method also converges unconditionally to the unique solution of the Toeplitz linear system. In addition, an upper bound for the contraction factor of the structured AHSS iteration method is derived. Numerical experiments are used to illustrate the effectiveness of the structured AHSS iteration method.     

Solvability of the boundary-value problem for a mixed equation involving hyper-Bessel fractional differential operator and bi-ordinal Hilfer fractional derivative

In a rectangular domain, a boundary-value problem is considered for a mixed equation with a regularized Caputo-like counterpart of hyper-Bessel differential operator and the bi-ordinal Hilfer's fractional derivative. By using the method of separation of variables a unique solvability of the considered problem has been established. Moreover, we have found the explicit solution of initial-boundary problems for the heat equation with the regularized Caputo-like counterpart of the hyper-Bessel differential operator with the non-zero starting point.     

Analysis of the Structured Total Least Squares Problem for Hankel/Toeplitz Matrices

The Structured Total Least Squares (STLS) problem is a natural extension of the Total Least Squares (TLS) approach when structured matrices are involved and a similarly structured rank deficient approximation of that matrix is desired. In many of those cases the STLS approach yields a Maximum Likelihood (ML) estimate as opposed to, e.g., TLS.In this paper we analyze the STLS problem for Hankel matrices (the theory can be extended in a straightforward way to Toeplitz matrices, block Hankel and block Toeplitz matrices). Using a particular parametrisation of rank-deficient Hankel matrices, we show that this STLS problem suffers from multiple local minima, the properties of which depend on the parameters of the new parametrisation. The latter observation makes initial estimates an important issue in STLS problems and a new initialization method is proposed. The new initialization method is applied to a speech compression example and the results confirm the improved performance compared to other previously proposed initialization methods.     

On the coupling of regularization techniques and the boundary element method for a hemivariational inequality modelling a delamination problem

In this paper, we couple regularization techniques of nondifferentiable optimization with the h‐version of the boundary element method (h‐BEM) to solve nonsmooth variational problems arising in contact mechanics. As a model example, we consider the delamination problem. The variational formulation of this problem leads to a hemivariational inequality with a nonsmooth functional defined on the contact boundary. This problem is first regularized and then discretized by an h‐BEM. We prove convergence of the h‐BEM Galerkin solution of the regularized problem in the energy norm, provide an a priori error estimate and give a numerical examples. Copyright © 2016 John Wiley & Sons, Ltd.