On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them

A full-rank under-determined linear system of equations Ax = b has in general infinitely many possible solutions. In recent years there is a growing interest in the sparsest solution of this equation—the one with the fewest non-zero entries, measured by ∥x∥₀. Such solutions find applications in signal and image processing, where the topic is typically referred to as “sparse representation”. Considering the columns of A as atoms of a dictionary, it is assumed that a given signal b is a linear composition of few such atoms. Recent work established that if the desired solution x is sparse enough, uniqueness of such a result is guaranteed. Also, pursuit algorithms, approximation solvers for the above problem, are guaranteed to succeed in finding this solution.Armed with these recent results, the problem can be reversed, and formed as an implied matrix factorization problem: Given a set of vectors {b_i}, known to emerge from such sparse constructions, Ax_i = b_i, with sufficiently sparse representations x_i, we seek the matrix A. In this paper we present both theoretical and algorithmic studies of this problem. We establish the uniqueness of the dictionary A, depending on the quantity and nature of the set {b_i}, and the sparsity of {x_i}. We also describe a recently developed algorithm, the K-SVD, that practically find the matrix A, in a manner similar to the K-Means algorithm. Finally, we demonstrate this algorithm on several stylized applications in image processing.     

The characterization of a class of multivariate MRA and semi-orthogonal parseval frame wavelets

Let A be a d × d expansive matrix with ∣detA∣ = 2. This paper addresses Parseval frame wavelets (PFWs) in the setting of reducing subspaces of L²(R^d). We prove that all semi-orthogonal PFWs (semi-orthogonal MRA PFWs) are precisely the ones with their dimension functions being non-negative integer-valued (0 or 1). We also characterize all MRA PFWs. Some examples are provided.     

Gradient based and least squares based iterative algorithms for matrix equations AXB + CXD = F

This paper develops a gradient based and a least squares based iterative algorithms for solving matrix equation AXB + CX^TD = F. The basic idea is to decompose the matrix equation (system) under consideration into two subsystems by applying the hierarchical identification principle and to derive the iterative algorithms by extending the iterative methods for solving Ax = b and AXB = F. The analysis shows that when the matrix equation has a unique solution (under the sense of least squares), the iterative solution converges to the exact solution for any initial values. A numerical example verifies the proposed theorems.     

Particular formulae for the Moore-Penrose inverse of a columnwise partitioned matrix

An essential part of Cegielski’s [Obtuse cones and Gram matrices with non-negative inverse, Linear Algebra Appl. 335 (2001) 167-181] considerations of some properties of Gram matrices with nonnegative inverses, which are pointed out to be crucial in constructing obtuse cones, consists in developing some particular formulae for the Moore-Penrose inverse of a columnwise partitioned matrix A = (A₁ : A₂) under the assumption that it is of full column rank. In the present paper, these results are generalized and extended. The generalization consists in weakening the assumption mentioned above to the requirement that the ranges of A₁ and A₂ are disjoint, while the extension consists in introducing the conditions referring to the class of all generalized inverses of A.     

Polynomials satisfied by two linked matrices

Polynomials in two variables, evaluated at A and with A being a square complex matrix and being its transform belonging to the set {A⁼, A^†, A^∗}, in which A⁼, A^†, and A^∗ denote, respectively, any reflexive generalized inverse, the Moore-Penrose inverse, and the conjugate transpose of A, are considered. An essential role, in characterizing when such polynomials are satisfied by two matrices linked as above, is played by the condition that the column space of A is the column space of . The results given unify a number of prior, isolated results.     

On the projectors FF and FF

A particular version of the singular value decomposition is exploited for an extensive analysis of two orthogonal projectors, namely FF^† and F^†F, determined by a complex square matrix F and its Moore-Penrose inverse F^†. Various functions of the projectors are considered from the point of view of their nonsingularity, idempotency, nilpotency, or their relation to the known classes of matrices, such as EP, bi-EP, GP, DR, or SR. This part of the paper was inspired by Benítez and Rako?evi? [J. Benítez, V. Rako?evi?, Matrices A such that AA^† − A^†A are nonsingular, Appl. Math. Comput. 217 (2010) 3493-3503]. Further characteristics of FF^† and F^†F, with a particular attention paid on the results dealing with column and null spaces of the functions and their eigenvalues, are derived as well. Besides establishing selected exemplary results dealing with FF^† and F^†F, the paper develops a general approach whose applicability extends far beyond the characteristics provided therein.     

On commutativity of projectors

The purpose of this paper is to revisit two problems discussed previously in the literature, both related to the commutativity property P₁P₂ = P₂P₁, where P₁ and P₂ denote projectors (i.e., idempotent matrices). The first problem was considered by Baksalary et al. [J.K. Baksalary, O.M. Baksalary, T. Szulc, A property of orthogonal projectors, Linear Algebra Appl. 354 (2002) 35-39], who have shown that if P₁ and P₂ are orthogonal projectors (i.e., Hermitian idempotent matrices), then in all nontrivial cases a product of any length having P₁ and P₂ as its factors occurring alternately is equal to another such product if and only if P₁ and P₂ commute. In the present paper a generalization of this result is proposed and validity of the equivalence between commutativity property and any equality involving two linear combinations of two any length products having orthogonal projectors P₁ and P₂ as their factors occurring alternately is investigated. The second problem discussed in this paper concerns specific generalized inverses of the sum P₁ + P₂ and the difference P₁ − P₂ of (not necessary orthogonal) commuting projectors P₁ and P₂. The results obtained supplement those provided in Section 4 of Baksalary and Baksalary [J.K. Baksalary, O.M. Baksalary, Commutativity of projectors, Linear Algebra Appl. 341 (2002) 129-142].     

Mixed finite element methods for general quadrilateral grids

We study a new mixed finite element of lowest order for general quadrilateral grids which gives optimal order error in the H(div)-norm. This new element is designed so that the H(div)-projection Π_h satisfies ∇ · Π_h = P_hdiv. A rigorous optimal order error estimate is carried out by proving a modified version of the Bramble-Hilbert lemma for vector variables. We show that a local H(div)-projection reproducing certain polynomials suffices to yield an optimal L²-error estimate for the velocity and hence our approach also provides an improved error estimate for original Raviart-Thomas element of lowest order. Numerical experiments are presented to verify our theory.     

Strong linear preservers of rank reverse permutability on triangular matrices

Let F be a field with ∣F∣ > 2 and T_n(F) be the set of all n × n upper triangular matrices, where n ? 2. Let k ? 2 be a given integer. A k-tuple of matrices A₁, …, A_k ∈ T_n(F) is called rank reverse permutable if rank(A₁ A₂ ? A_k) = rank(A_k A_k−1 ? A₁). We characterize the linear maps on T_n(F) that strongly preserve the set of rank reverse permutable matrix k-tuples.     

Bounds for the error of linear systems of equations using the theory of moments

Consider the system, of linear equations Ax = b where A is an n × n real symmetric, positive definite matrix and b is a known vector. Suppose we are given an approximation to x, ξ, and we wish to determine upper and lower bounds for ∥ x − ξ ∥ where ∥ ··· ∥ indicates the euclidean norm. Given the sequence of vectors {r_i}_{i^k = 0}, where r_i = Ar_{i − 1} and r₀ = b − Aξ, it is shown how to construct a sequence of upper and lower bounds for ∥ x − ξ ∥ using the theory of moments.     

A full row-rank system matrix generated by the strip-based projection model in discrete tomography

Let Cu = k be an underdetermined linear system generated by the strip-based projection model in discrete tomography, where C is row-rank deficient. In the case of one scanning direction the linear dependency of the rows of C is studied in this paper. An index set H is specified such that if all rows of C with row indices in H are deleted then the rows of resultant matrix F are maximum linearly independent rows of C. Therefore, the corresponding system is equivalent to Cu = k and consequently, the cost of an image reconstruction from is reduced.     

The spectra of some families of digraphs

Let G be a digraph (or a graph, when seen as a symmetric digraph) with adjacency matrix A, having the eigenvalue λ with associated eigenvector v. As it is well known, the entries of v can be interpreted as charges in each vertex. Then, the linear transformation v ? Av corresponds to a natural displacement of charges, where each vertex sends a copy of its charge to its in-neighbors and absorbs a copy of the charges of its out-neighbors, so the resulting charge distribution is just λv. In this work we use this approach to derive some old and new results about the spectral characterization of G. More precisely, we show how to obtain the spectra of some families of (di)graphs, such as the partial line digraphs and the line graphs of regular or semiregular graphs.     

Rank One Perturbations in a Pontryagin Space with One Negative Square

Let N₁ denote the class of generalized Nevanlinna functions with one negative square and let N_1, 0 be the subclass of functions Q(z)∈N₁ with the additional properties lim_y→∞ Q(iy)/y=0 and lim sup_y→∞ y |Im Q(iy)|<∞. These classes form an analytic framework for studying (generalized) rank one perturbations A(τ)=A+τ[·, ω] ω in a Pontryagin space setting. Many functions appearing in quantum mechanical models of point interactions either belong to the subclass N_1, 0 or can be associated with the corresponding generalized Friedrichs extension. In this paper a spectral theoretical analysis of the perturbations A(τ) and the associated Friedrichs extension is carried out. Many results, such as the explicit characterizations for the critical eigenvalues of the perturbations A(τ), are based on a recent factorization result for generalized Nevanlinna functions.     

Power series solutions to Volterra integral equations

Power series type solutions are given for a wide class of linear and q-dimensional nonlinear Volterra equations on R^p. The basic assumption on the kernel K(x, y) is that K(x, xt) has a power series in x. For example, this holds for any analytic kernel.The kernel may be strongly singular, provided certain constants are finite. One and only one such power series solution exists. Its coefficients are given by a simple iterative formula. In many cases this may be solved explicitly. In particular an explicit formula for the resolvent is given.     

Some explicit solutions of the additive Deligne-Simpson problem and their applications

In this paper we construct three infinite series and two extra triples (E₈ and ) of complex matrices B, C, and A=B+C of special spectral types associated to Simpson's classification in Amer. Math. Soc. Proc. 1 (1992) 157 and Magyar et al. classification in Adv. Math. 141 (1999) 97. This enables us to construct Fuchsian systems of differential equations which generalize the hypergeometric equation of Gauss-Riemann. In a sense, they are the closest relatives of the famous equation, because their triples of spectral flags have finitely many orbits for the diagonal action of the general linear group in the space of solutions. In all the cases except for E₈, we also explicitly construct scalar products such that A, B, and C are self-adjoint with respect to them. In the context of Fuchsian systems, these scalar products become monodromy invariant complex symmetric bilinear forms in the spaces of solutions.When the eigenvalues of A, B, and C are real, the matrices and the scalar products become real as well. We find inequalities on the eigenvalues of A, B, and C which make the scalar products positive-definite.As proved by Klyachko, spectra of three hermitian (or real symmetric) matrices B, C, and A=B+C form a polyhedral convex cone in the space of triple spectra. He also gave a recursive algorithm to generate inequalities describing the cone. The inequalities we obtain describe non-recursively some faces of the Klyachko cone.     

Small divisors of Bernoulli sums

Let ?= {?_i,i ≥1} be a sequence of independent Bernoulli random variables (P{?_i = 0} = P{?_i = 1 } = 1/2) with basic probability space (Ω, A, P). Consider the sequence of partial sums B_n=?₁+...+?_n, n=1,2..... We obtain an asymptotic estimate for the probability P{P^-(B_n) > >} for >≤n^{e/log log n}, c a positive constant.