Heuristics for exact nonnegative matrix factorization

The exact nonnegative matrix factorization (exact NMF) problem is the following: given an m-by-n nonnegative matrix X and a factorization rank r, find, if possible, an m-by-r nonnegative matrix W and an r-by-n nonnegative matrix H such that \(X = WH\). In this paper, we propose two heuristics for exact NMF, one inspired from simulated annealing and the other from the greedy randomized adaptive search procedure. We show empirically that these two heuristics are able to compute exact nonnegative factorizations for several classes of nonnegative matrices (namely, linear Euclidean distance matrices, slack matrices, unique-disjointness matrices, and randomly generated matrices) and as such demonstrate their superiority over standard multi-start strategies. We also consider a hybridization between these two heuristics that allows us to combine the advantages of both methods. Finally, we discuss the use of these heuristics to gain insight on the behavior of the nonnegative rank, i.e., the minimum factorization rank such that an exact NMF exists. In particular, we disprove a conjecture on the nonnegative rank of a Kronecker product, propose a new upper bound on the extension complexity of generic n-gons and conjecture the exact value of (i) the extension complexity of regular n-gons and (ii) the nonnegative rank of a submatrix of the slack matrix of the correlation polytope.     

Rational realization of the minimum ranks of nonnegative sign pattern matrices

A sign pattern matrix (or nonnegative sign pattern matrix) is a matrix whose entries are from the set {+,?, 0} ({+, 0}, respectively). The minimum rank (or rational minimum rank) of a sign pattern matrix A is the minimum of the ranks of the matrices (rational matrices, respectively) whose entries have signs equal to the corresponding entries of A. Using a correspondence between sign patterns with minimum rank r ≥ 2 and point-hyperplane configurations in R^r?1 and Steinitz’s theorem on the rational realizability of 3-polytopes, it is shown that for every nonnegative sign pattern of minimum rank at most 4, the minimum rank and the rational minimum rank are equal. But there are nonnegative sign patterns with minimum rank 5 whose rational minimum rank is greater than 5. It is established that every d-polytope determines a nonnegative sign pattern with minimum rank d + 1 that has a (d + 1) × (d + 1) triangular submatrix with all diagonal entries positive. It is also shown that there are at most min{3m, 3n} zero entries in any condensed nonnegative m × n sign pattern of minimum rank 3. Some bounds on the entries of some integer matrices achieving the minimum ranks of nonnegative sign patterns with minimum rank 3 or 4 are established.     

On the nonnegative rank of Euclidean distance matrices

The Euclidean distance matrix for n distinct points in R^r is generically of rank r + 2. It is shown in this paper via a geometric argument that its nonnegative rank for the case r = 1 is generically n.     

Nonnegative rank factorization—a heuristic approach via rank reduction

Given any nonnegative matrix $A \in \mathbb{R}^{m \times n}$ , it is always possible to express A as the sum of a series of nonnegative rank-one matrices. Among the many possible representations of A, the number of terms that contributes the shortest nonnegative rank-one series representation is called the nonnegative rank of A. Computing the exact nonnegative rank and the corresponding factorization are known to be NP-hard. Even if the nonnegative rank is known a priori, no simple procedure exists presently that is able to perform the nonnegative factorization. Based on the Wedderburn rank reduction formula, this paper proposes a heuristic approach to tackle this difficult problem numerically. Starting with A, the idea is to recurrently extrat, whenever possible, a rank-one nonnegative portion from the previous matrix while keeping the residual nonnegative and lowering its rank by one. With a slight modification for symmetry, the method can equally be applied to another important class of completely positive matrices. No convergence can be guaranteed, but repeated restart might help alleviate the difficulty. Extensive numerical testing seems to suggest that the proposed algorithm might serve as a first-step numerical means for exploring the intriguing problem of nonnegative rank factorization.     

A comparison of nonnegative real ranks and their preservers

This paper concerns three notions of rank of matrices over semirings; real rank, semiring rank and column rank. These three rank functions are the same over subfields of reals but differ for matrices over subsemirings of nonnegative reals. We investigate the largest values of r for which the real rank and semiring rank, real rank and column rank of all m×n matrices over a given semiring are both r, respectively. We also characterize the linear operators which preserve the column rank of matrices over certain subsemirings of the nonnegative reals.     

Nonnegative rank-preserving operators

Analogues of characterizations of rank-preserving operators on field-valued matrices are determined for matrices witheentries in certain structures $\text{S}$ contained in the nonnegative reals. For example, if $\text{S}$ is the set of nonnegative members of a real unique factorization domain (e.g. the nonnegative reals or the nonnegative integers), M is the set of m×n matrices with entries in $\text{S}$, and min(m,n)?4, then a “linear” operator on M preserves the “rank” of each matrix in M if and only if it preserves the ranks of those matrices in M of ranks 1, 2, and 4. Notions of rank and linearity are defined analogously to the field-valued concepts. Other characterizations of rank-preserving operators for matrices over these and other structures $\text{S}$ are also given.     

The nonnegative rank factorizations of nonnegative matrices

Let A ∈ P^{m × n}_r, the set of all m × n nonnegative matrices having the same rank r. For matrices A in P^{m × n}_n, we introduce the concepts of “A has only trivial nonnegative rank factorizations” and “A can have nontrivial nonnegative rank factorizations.” Correspondingly, the set P^{m × n}_n is divided into two disjoint subsets P₍₁₎ and P₍₂₎ such that P₍₁₎∪P₍₂₎ = P^{m × n}_n. It happens that the concept of “A has only trivial nonnegative rank factorizations” is a generalization of “A is prime in P^{n × n}_n.” We characterize the sets P₍₁₎ and P₍₂₎. Some of our results generalize some theorems in the paper of Daniel J. Richman and Hans Schneider [9].     

On generalized inverses of matrices over integral domains

It is proved that a matrix A over an integral domain admits a 1-inverse if and only if a linear combination of all the r × r minors of A is equal to one, where r is the rank of A. Some results on the existence of Moore-Penrose inverses are also obtained.     

On approximation by polynomials increasing to the right of the interval

We obtain sufficient conditions on a real valued function ?, continuous on [0, + ∞), to insure that, for some nonnegative integer n, there is a nonnegative number r(n) so that for any r ? r(n), the polynomial of best approximation to ? on [0, r] from π_n is increasing and nonnegative on [r, + ∞). Here, π_n denotes the set of all real polynomials of degree n or less. The proofs of Theorems 1 and 2 use only properties of Lagrange interpolation while that of Theorem 3 employs results on the location of interpolation points in Chebyshev approximation.     

Characterizations of rectangular totally and strictly totally positive matrices

An n×m real matrix A is said to be totally positive (strictly totally positive) if every minor is nonnegative (positive). In this paper, we study characterizations of these classes of matrices by minors, by their full rank factorization and by their thin QR factorization.     

Real rank versus nonnegative rank

We consider the set of m×n nonnegative real matrices and define the nonnegative rank of a matrix A to be the minimum k such that A=BC where B is m×k and C is k×n. Given that the real rank of A is j for some j, we give bounds on the nonnegative rank of A and A².     

On the structure of the Wishart distribution

In this paper it is shown that every nonnegative definite symmetric random matrix with independent diagonal elements and at least one nondegenerate nondiagonal element has a noninfinitely divisible distribution. Using this result it is established that every Wishart distribution W_p(k, Σ, M) with both p and rank (Σ) ≥ 2 is noninfinitely divisible. The paper also establishes that any Wishart matrix having distribution W_p(k, Σ, 0) has the joint distribution of its elements in the rth row and rth column to be infinitely divisible for every r = 1,2,…,p.     

On the plane term rank of three dimensional matrices

It is shown that if A is a p>×q×r matrix such that each of the horizontal plane sections of A has full term rank, then the plane term rank of A is greater than m?√m where m= min {p,q,r}. In particular, if A is a three dimensional line stochastic matrix of order n, then the plane term rank of A is greater than n?√n.     

The Moore-Penrose inverse of a morphism with factorization

Given an m-by-n matrix A of rank r over a field with an involutory automorphism, it is well known that A has a Moore-Penrose inverse if and only if rank A¹A=r= rank AA¹. By use of the full-rank factorization theorem, this result may be restated in the category of finite matrices as follows: if (A₁, r, A₂) is an (epic, monic) factorization of A:m→n through r, then A has a Moore-Penrose inverse if and only if (A¹A₁, r, A₂) and (A₁, r, A₂A¹) are, respectively, (epic, monic) factorizations of A¹A:n→n and AA¹:m→m through r. This characterization of the existence of Moore-Penrose inverses is extended to arbitrary morphisms with (epic, monic) factorizations.     

r-Numbers completion problems of partial upper canonical form I

A pair (A, B), where A is an n × n matrix and B is an n × m matrix, is said to have the nonnegative integers sequence {r_j}_j=1^p as the r-numbers sequence if r₁ = rank(B) and r_j = rank[B AB … A^j−1 B] − rank[B AB … A^j−2B], 2 ≤ j ≤ p. Given a partial upper triangular matrix A of size n × n in upper canonical form and an n × m matrix B, we develop an algorithm that obtains a completion A_c of A, such that the pair (A^c, B) has an r-numbers sequence prescribed under some restrictions.     

Hoffman polynomials of nonnegative irreducible matrices and strongly connected digraphs

For a nonnegative n × n matrix A, we find that there is a polynomial f(x)∈R[x] such that f(A) is a positive matrix of rank one if and only if A is irreducible. Furthermore, we show that the lowest degree such polynomial f(x) with tr f(A) = n is unique. Thus, generalizing the well-known definition of the Hoffman polynomial of a strongly connected regular digraph, for any irreducible nonnegative n × n matrix A, we are led to define its Hoffman polynomial to be the polynomial f(x) of minimum degree satisfying that f(A) is positive and has rank 1 and trace n. The Hoffman polynomial of a strongly connected digraph is defined to be the Hoffman polynomial of its adjacency matrix. We collect in this paper some basic results and open problems related to the concept of Hoffman polynomials.     

Distributions vectorielles homogènes sur une algèbre de Jordan

We study distributions on a Euclidean Jordan algebra V with values in a finite dimensional representation space for the identity component G of the structure group of V and homogeneous equivariance condition. We show that such distributions exist if and only if the representation is spherical, and that then the dimension of the space of these distributions is r+1 (where r is the rank of V). We give also construction of these distributions and of those that are invariant under the semi-simple part of G.     

Primitive and measure-preserving systems of elements on the varieties of metabelian and metabelian profinite groups

Given a free metabelian group S of finite rank r, r ≥ 2, we prove that a system of elements g ₁, ..., g _n ∈ S for n = 1 or n = r preserves measure on the variety of all metabelian groups if and only if the system is primitive. Similar results hold for a free profinite group $\hat S$ and the variety of finite metabelian groups for each n, 1 ≤ n ≤ r. Some corollaries to these theorems are derived.