Simultaneous identification of diffusion coefficient,spacewise dependent source and initial value for one‐dimensional heat equation

This paper deals with an inverse problem of determining the diffusion coefficient, spacewise dependent source term, and the initial value simultaneously for a one‐dimensional heat equation based on the boundary control, boundary measurement, and temperature distribution at a given single instant in time. By a Dirichlet series representation for the boundary observation, the identification of the diffusion coefficient and initial value can be transformed into a spectral estimation problem of an exponential series with measurement error, which is solved by the matrix pencil method. For the identification of the source term, a finite difference approximation method in conjunction with the truncated singular value decomposition is adopted, where the regularization parameter is determined by the generalized cross‐validation criterion. Numerical simulations are performed to verify the result of the proposed algorithm. Copyright © 2016 John Wiley & Sons, Ltd.     

Sparse principal component analysis via regularized low rank matrix approximation

Principal component analysis (PCA) is a widely used tool for data analysis and dimension reduction in applications throughout science and engineering. However, the principal components (PCs) can sometimes be difficult to interpret, because they are linear combinations of all the original variables. To facilitate interpretation, sparse PCA produces modified PCs with sparse loadings, i.e. loadings with very few non-zero elements. In this paper, we propose a new sparse PCA method, namely sparse PCA via regularized SVD (sPCA-rSVD). We use the connection of PCA with singular value decomposition (SVD) of the data matrix and extract the PCs through solving a low rank matrix approximation problem. Regularization penalties are introduced to the corresponding minimization problem to promote sparsity in PC loadings. An efficient iterative algorithm is proposed for computation. Two tuning parameter selection methods are discussed. Some theoretical results are established to justify the use of sPCA-rSVD when only the data covariance matrix is available. In addition, we give a modified definition of variance explained by the sparse PCs. The sPCA-rSVD provides a uniform treatment of both classical multivariate data and high-dimension-low-sample-size (HDLSS) data. Further understanding of sPCA-rSVD and some existing alternatives is gained through simulation studies and real data examples, which suggests that sPCA-rSVD provides competitive results.     

Symmetric orthogonal approximation to symmetric tensors with applications to image reconstruction

The main objective of this paper is to study an approximation of symmetric tensors by symmetric orthogonal decomposition. We propose and study an iterative algorithm to determine a symmetric orthogonal approximation and analyze the convergence of the proposed algorithm. Numerical examples are reported to demonstrate the effectiveness of the proposed algorithm. We also apply the proposed algorithm to represent correlated face images. We demonstrate better face image reconstruction results by combining principal components and symmetric orthogonal approximation instead of combining principal components and higher‐order SVD results.     

快速幂法子空间跟踪

子空间跟踪算法是许多工程计算问题的核心.Hua等人将计算特征值问题的幂法扩展为自然幂法子空间跟踪算法.在指出基于秩1矩阵更新的自然幂法的快速实现方案NP3不收敛的同时,应用矩阵求逆引理给出了一种新的快速子空间跟踪算法:快速幂法子空间跟踪算法.仿真实验表明,所提算法是收敛与稳定的,其性能优于或相当于几种常见的快速子空间跟踪算法.     

Recursive approximation of the dominant eigenspace of an indefinite matrix

We consider here the problem of tracking the dominant eigenspace of an indefinite matrix by updating recursively a rank k$k$ approximation of the given matrix. The tracking uses a window of the given matrix, which increases at every step of the algorithm. Therefore, the rank of the approximation increases also, and hence a rank reduction of the approximation is needed to retrieve an approximation of rank k$k$. In order to perform the window adaptation and the rank reduction in an efficient manner, we make use of a new anti-triangular decomposition for indefinite matrices. All steps of the algorithm only make use of orthogonal transformations, which guarantees the stability of the intermediate steps. We also show some numerical experiments to illustrate the performance of the tracking algorithm.     

Low Tucker rank tensor completion using a symmetric block coordinate descent method

Low Tucker rank tensor completion has wide applications in science and engineering. Many existing approaches dealt with the Tucker rank by unfolding matrix rank. However, unfolding a tensor to a matrix would destroy the data's original multi-way structure, resulting in vital information loss and degraded performance. In this article, we establish a relationship between the Tucker ranks and the ranks of the factor matrices in Tucker decomposition. Then, we reformulate the low Tucker rank tensor completion problem as a multilinear low rank matrix completion problem. For the reformulated problem, a symmetric block coordinate descent method is customized. For each matrix rank minimization subproblem, the classical truncated nuclear norm minimization is adopted. Furthermore, temporal characteristics in image and video data are introduced to such a model, which benefits the performance of the method. Numerical simulations illustrate the efficiency of our proposed models and methods.     

基于信息压缩矩阵特征值映射的时变系统UD分解辨识算法

在系统辨识领域遗忘因子UD分解算法(一种通过对系统数据矩阵进行UD分解的在线辨识算法)具有对时变系统阶次和参数同步估计的优异性能,但传统的遗忘策略不能从根本上解决信息压缩矩阵数据过饱和问题,为了拓展现有UD分解算法在时变系统的适用范围,同时针对数据空间分布不均匀性,提出一种基于信息压缩矩阵特征值映射的UD分解辨识算法....     

On the Nyström and Column-Sampling Methods for the Approximate Principal Components Analysis of Large Datasets

In this article, we analyze approximate methods for undertaking a principal components analysis (PCA) on large datasets. PCA is a classical dimension reduction method that involves the projection of the data onto the subspace spanned by the leading eigenvectors of the covariance matrix. This projection can be used either for exploratory purposes or as an input for further analysis, for example, regression. If the data have billions of entries or more, the computational and storage requirements for saving and manipulating the design matrix in fast memory are prohibitive. Recently, the Nyström and column-sampling methods have appeared in the numerical linear algebra community for the randomized approximation of the singular value decomposition of large matrices. However, their utility for statistical applications remains unclear. We compare these approximations theoretically by bounding the distance between the induced subspaces and the desired, but computationally infeasible, PCA subspace. Additionally we show empirically, through simulations and a real data example involving a corpus of emails, the trade-off of approximation accuracy and computational complexity.     

Best two‐parameter rational approximation algorithm of given order: a variation of adaptive Fourier decomposition

The concept of mono‐component is widely used in nonstationary signal processing and time‐frequency analysis. In this paper, we construct several classes of complete rational function systems in the Hardy space, whose boundary values are mono‐components. Then, we propose a best approximation algorithm (BAA) based on optimal selections of two parameters in the orthonormal bases according to the approximation error. Effectiveness of BAA is evaluated by comparison experiments with the classical Fourier decomposition algorithm. It is also shown that BAA has promising results for filtering out noises and dealing with real‐world signals. Copyright © 2014 John Wiley & Sons, Ltd.     

On the successive supersymmetric rank‐1 decomposition of higher‐order supersymmetric tensors

In this paper, a successive supersymmetric rank‐1 decomposition of a real higher‐order supersymmetric tensor is considered. To obtain such a decomposition, we design a greedy method based on iteratively computing the best supersymmetric rank‐1 approximation of the residual tensors. We further show that a supersymmetric canonical decomposition could be obtained when the method is applied to an orthogonally diagonalizable supersymmetric tensor, and in particular, when the order is 2, this method generates the eigenvalue decomposition for symmetric matrices. Details of the algorithm designed and the numerical results are reported in this paper. Copyright © 2007 John Wiley & Sons, Ltd.     

Mixtures of truncated basis functions

In this paper we propose a framework, called mixtures of truncated basis functions (MoTBFs), for representing general hybrid Bayesian networks. The proposed framework generalizes both the mixture of truncated exponentials (MTEs) framework and the Mixture of Polynomials (MoPs) framework. Similar to MTEs and MoPs, MoTBFs are defined so that the potentials are closed under combination and marginalization, which ensures that inference in MoTBF networks can be performed efficiently using the Shafer-Shenoy architecture.Based on a generalized Fourier series approximation, we devise a method for efficiently approximating an arbitrary density function using the MoTBF framework. The translation method is more flexible than existing MTE or MoP-based methods, and it supports an online/anytime tradeoff between the accuracy and the complexity of the approximation. Experimental results show that the approximations obtained are either comparable or significantly better than the approximations obtained using existing methods.     

Near-optimal stochastic approximation for online principal component estimation

Principal component analysis (PCA) has been a prominent tool for high-dimensional data analysis. Online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests. Despite its rich applications, theoretical convergence analysis remains largely open. In this paper, we cast online PCA into a stochastic nonconvex optimization problem, and we analyze the online PCA algorithm as a stochastic approximation iteration. The stochastic approximation iteration processes data points incrementally and maintains a running estimate of the principal component. We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm. Under the subgaussian assumption, we show that the finite-sample error bound closely matches the minimax information lower bound.     

Truncated Newton method for sparse unconstrained optimization using automatic differentiation

When solving large complex optimization problems, the user is faced with three major problems. These are (i) the cost in human time in obtaining accurate expressions for the derivatives involved; (ii) the need to store second derivative information; and (iii), of lessening importance, the time taken to solve the problem on the computer. For many problems, a significant part of the latter can be attributed to solving Newton-like equations. In the algorithm described, the equations are solved using a conjugate direction method that only needs the Hessian at the current point when it is multiplied by a trial vector. In this paper, we present a method that finds this product using automatic differentiation while only requiring vector storage. The method takes advantage of any sparsity in the Hessian matrix and computes exact derivatives. It avoids the complexity of symbolic differentiation, the inaccuracy of numerical differentiation, the labor of finding analytic derivatives, and the need for matrix store. When far from a minimum, an accurate solution to the Newton equations is not justified, so an approximate solution is obtained by using a version of Dembo and Steihaug's truncated Newton algorithm (Ref. 1).This paper was presented at the SIAM National Meeting, Boston, Massachusetts, 1986.     

Precision Matrix Estimation by Inverse Principal Orthogonal Decomposition

We investigate the structure of a large precision matrix in Gaussian graphical models by decomposing it into a low rank component and a remainder part with sparse precision matrix.Based on the decomposition,we propose to estimate the large precision matrix by inverting a principal orthogonal decomposition(IPOD).The IPOD approach has appealing practical interpretations in conditional graphical models given the low rank component,and it connects to Gaussian graphical models with latent variables.Specifically,we show that the low rank component in the decomposition of the large precision matrix can be viewed as the contribution from the latent variables in a Gaussian graphical model.Compared with existing approaches for latent variable graphical models,the IPOD is conveniently feasible in practice where only inverting a low-dimensional matrix is required.To identify the number of latent variables,which is an objective of its own interest,we investigate and justify an approach by examining the ratios of adjacent eigenvalues of the sample covariance matrix?Theoretical properties,numerical examples,and a real data application demonstrate the merits of the IPOD approach in its convenience,performance,and interpretability.     

On the purely algebraic data‐sparse approximation of the inverse and the triangular factors of sparse matrices

The approximation of the inverse and the factors of the LU decomposition of general sparse matrices by hierarchical matrices is investigated. In this first approach, we present and motivate a new matrix partitioning algorithm which is based on the matrix graph by proving logarithmic‐linear complexity of the approximant in the case of bounded condition numbers. In contrast to the usual partitioning, the new algorithm allows to treat general grids if the origin of the sparse matrix is the finite element discretization of differential operators. Numerical examples indicate that the restriction to bounded condition numbers has only technical reasons. Copyright © 2010 John Wiley & Sons, Ltd.     

Principal component histograms from interval-valued observations

The focus of this paper is to propose an approach to construct histogram values for the principal components of interval-valued observations. Le-Rademacher and Billard (J Comput Graph Stat 21:413–432, 2012) show that for a principal component analysis on interval-valued observations, the resulting observations in principal component space are polytopes formed by the convex hulls of linearly transformed vertices of the observed hyper-rectangles. In this paper, we propose an algorithm to translate these polytopes into histogram-valued data to provide numerical values for the principal components to be used as input in further analysis. Other existing methods of principal component analysis for interval-valued data construct the principal components, themselves, as intervals which implicitly assume that all values within an observation are uniformly distributed along the principal components axes. However, this assumption is only true in special cases where the variables in the dataset are mutually uncorrelated. Representation of the principal components as histogram values proposed herein more accurately reflects the variation in the internal structure of the observations in a principal component space. As a consequence, subsequent analyses using histogram-valued principal components as input result in improved accuracy.     

Randomized algorithms for total least squares problems

Motivated by the recently popular probabilistic methods for low‐rank approximations and randomized algorithms for the least squares problems, we develop randomized algorithms for the total least squares problem with a single right‐hand side. We present the Nyström method for the medium‐sized problems. For the large‐scale and ill‐conditioned cases, we introduce the randomized truncated total least squares with the known or estimated rank as the regularization parameter. We analyze the accuracy of the algorithm randomized truncated total least squares and perform numerical experiments to demonstrate the efficiency of our randomized algorithms. The randomized algorithms can greatly reduce the computational time and still maintain good accuracy with very high probability.     

The Adjoint Newton Algorithm for Large-Scale Unconstrained Optimization in Meteorology Applications

A new algorithm is presented for carrying out large-scale unconstrained optimization required in variational data assimilation using the Newton method. The algorithm is referred to as the adjoint Newton algorithm. The adjoint Newton algorithm is based on the first- and second-order adjoint techniques allowing us to obtain the Newton line search direction by integrating a tangent linear equations model backwards in time (starting from a final condition with negative time steps). The error present in approximating the Hessian (the matrix of second-order derivatives) of the cost function with respect to the control variables in the quasi-Newton type algorithm is thus completely eliminated, while the storage problem related to the Hessian no longer exists since the explicit Hessian is not required in this algorithm. The adjoint Newton algorithm is applied to three one-dimensional models and to a two-dimensional limited-area shallow water equations model with both model generated and First Global Geophysical Experiment data. We compare the performance of the adjoint Newton algorithm with that of truncated Newton, adjoint truncated Newton, and LBFGS methods. Our numerical tests indicate that the adjoint Newton algorithm is very efficient and could find the minima within three or four iterations for problems tested here. In the case of the two-dimensional shallow water equations model, the adjoint Newton algorithm improves upon the efficiencies of the truncated Newton and LBFGS methods by a factor of at least 14 in terms of the CPU time required to satisfy the same convergence criterion.The Newton, truncated Newton and LBFGS methods are general purpose unconstrained minimization methods. The adjoint Newton algorithm is only useful for optimal control problems where the model equations serve as strong constraints and their corresponding tangent linear model may be integrated backwards in time. When the backwards integration of the tangent linear model is ill-posed in the sense of Hadamard, the adjoint Newton algorithm may not work. Thus, the adjoint Newton algorithm must be used with some caution. A possible solution to avoid the current weakness of the adjoint Newton algorithm is proposed.     

A method of matrix inverse triangular decomposition based on contiguous principal submatrices

An algorithm is presented which performs the triangular decomposition of the inverse of a given matrix. The method is applicable to any matrix all contiguous principal submatrices of which are nonsingular. The algorithm is particularly efficient when the matrix has certain partial symmetries exhibited by the Toeplitz structure.     

Randomized algorithms for distributed computation of principal component analysis and singular value decomposition

Randomized algorithms provide solutions to two ubiquitous problems: (1) the distributed calculation of a principal component analysis or singular value decomposition of a highly rectangular matrix, and (2) the distributed calculation of a low-rank approximation (in the form of a singular value decomposition) to an arbitrary matrix. Carefully honed algorithms yield results that are uniformly superior to those of the stock, deterministic implementations in Spark (the popular platform for distributed computation); in particular, whereas the stock software will without warning return left singular vectors that are far from numerically orthonormal, a significantly burnished randomized implementation generates left singular vectors that are numerically orthonormal to nearly the machine precision.