Goodness-of-fit indices for partial least squares path modeling

This paper discusses a recent development in partial least squares (PLS) path modeling, namely goodness-of-fit indices. In order to illustrate the behavior of the goodness-of-fit index (GoF) and the relative goodness-of-fit index (GoF_rel), we estimate PLS path models with simulated data, and contrast their values with fit indices commonly used in covariance-based structural equation modeling. The simulation shows that the GoF and the GoF_rel are not suitable for model validation. However, the GoF can be useful to assess how well a PLS path model can explain different sets of data.     

The partial total least squares algorithm

In this paper, an improved algorithm PTLS for solving total least squares (TLS) problems AX ≈ B is presented. As only a basis of the right singular subspace associated with the smallest singular values of the data [A; B] is needed, the computational cost can be reduced considerably by using the partial SVD algorithm. This algorithm computes in an efficient way a basis for the left and/or right singular subspace of a matrix associated with its smallest singular values.An analysis of the operation counts, as well as computational results, show the relative efficiency of PTLS with respect to the classical TLS algorithm. Typically, PTLS reduces the computation time with a factor 2.     

Two-stage least squares based iterative estimation algorithm for CARARMA system modeling

For stochastic systems described by the controlled autoregressive autoregressive moving average (CARARMA) models, a new-type two-stage least squares based iterative algorithm is proposed for identifying the system model parameters and the noise model parameters. The basic idea is based on the interactive estimation theory and to estimate the parameter vectors of the system model and the noise model, respectively. The simulation results indicate that the proposed algorithm is effective.     

On the convergence of gauss' alternating procedure in the method of the least squares

Sunto Gauss ha proposto un procedimento iterativo per risolvere problemi di minimi quadrati. L'autore dà una dimostrazione rigorosa della convergenza di questo metodo, riducendola a un altro teorema dimostrato nella sua Memoria pubblicata nei Rendiconti di Matematica pura ed applicata, serie V, vol. XIII, 1954, pp. 140–163.

---

---

A Giovanni Sansone per il suo 70° compleanno.

---

This work was performed on a National Bureau of Standards contract with The American University, Washington, D. C.     

Iterative weighted partial spline least squares estimation in semiparametric modeling of longitudinal data

In this paper we consider the estimating problem of a semiparametric regression modelling whenthe data are longitudinal.An iterative weighted partial spline least squares estimator(IWPSLSE)for the para-metric component is proposed which is more efficient than the weighted partial spline least squares estimator(WPSLSE)with weights constructed by using the within-group partial spline least squares residuals in the sense     

A partial proximal point algorithm for nuclear norm regularized matrix least squares problems

We introduce a partial proximal point algorithm for solving nuclear norm regularized matrix least squares problems with equality and inequality constraints. The inner subproblems, reformulated as a system of semismooth equations, are solved by an inexact smoothing Newton method, which is proved to be quadratically convergent under a constraint non-degeneracy condition, together with the strong semi-smoothness property of the singular value thresholding operator. Numerical experiments on a variety of problems including those arising from low-rank approximations of transition matrices show that our algorithm is efficient and robust.     

Deep partial least squares for instrumental variable regression

In this paper, we propose deep partial least squares for the estimation of high-dimensional nonlinear instrumental variable regression. As a precursor to a flexible deep neural network architecture, our methodology uses partial least squares for dimension reduction and feature selection from the set of instruments and covariates. A central theoretical result, due to Brillinger (2012) Selected Works of Daving Brillinger. 589-606, shows that the feature selection provided by partial least squares is consistent and the weights are estimated up to a proportionality constant. We illustrate our methodology with synthetic datasets with a sparse and correlated network structure and draw applications to the effect of childbearing on the mother's labor supply based on classic data of Chernozhukov et al. Ann Rev Econ. (2015b):649–688. The results on synthetic data as well as applications show that the deep partial least squares method significantly outperforms other related methods. Finally, we conclude with directions for future research.     

Influence function analysis applied to partial least squares

In this work we present the empirical influence functions for the covariances (eigenvalues) and directions (eigenvectors) of partial least squares under the constraint of uncorrelated components. We apply the results to several data sets and provide advice for using these tools in practice.     

A new algorithm for nonlinear least squares

A new algorithm for the nonlinear least-squares problems is introduced and illustrated in this article. It is shown that the new algorithm is relatively more efficient compared to the other algorithms in current use and it works for problems where other methods fail. The new method is illustrated by solving a number of classical test problems. In light of the present method, improvements for some other methods in current use are also suggested in this article. Bibliography: 22 titles. Published inZapiski Nauchnykh Seminarov POMI, Vol. 207, pp. 143–157, 1993.     

Solution of linear least squares via the ABS algorithm

The ABS class for linear and nonlinear systems has been recently introduced by Abaffy, Broyden, Galantai and Spedicato. Here we consider various ways of applying these algorithms to the determination of the minimal euclidean norm solution of over-determined linear systems in the least squares sense. Extensive numerical experiments show that the proposed algorithms are efficient and that one of them usually gives better accuracy than standard implementations of the QR orthogonalization algorithm with Householder reflections.     

On the modified Gram-Schmidt algorithm for weighted and constrained linear least squares problems

A framework and an algorithm for using modified Gram-Schmidt for constrained and weighted linear least squares problems is presented. It is shown that a direct implementation of a weighted modified Gram-Schmidt algorithm is unstable for heavily weighted problems. It is shown that, in most cases it is possible to get a stable algorithm by a simple modification free from any extra computational costs. In particular, it is not necessary to perform reorthogonalization.Solving the weighted and constrained linear least squares problem with the presented weighted modified Gram-Schmidt algorithm is seen to be numerically equivalent to an algorithm based on a weighted Householder-likeQR factorization applied to a slightly larger problem. This equivalence is used to explain the instability of the weighted modified Gram-Schmidt algorithm. If orthogonality, with respect to a weighted inner product, of the columns inQ is important then reorthogonalization can be used. One way of performing such reorthogonalization is described.Computational tests are given to show the main features of the algorithm.     

An overview on the shrinkage properties of partial least squares regression

The aim of this paper is twofold. In the first part, we recapitulate the main results regarding the shrinkage properties of partial least squares (PLS) regression. In particular, we give an alternative proof of the shape of the PLS shrinkage factors. It is well known that some of the factors are >1. We discuss in detail the effect of shrinkage factors for the mean squared error of linear estimators and argue that we cannot extend the results to PLS directly, as it is nonlinear. In the second part, we investigate the effect of shrinkage factors empirically. In particular, we point out that experiments on simulated and real world data show that bounding the absolute value of the PLS shrinkage factors by 1 seems to leads to a lower mean squared error.     

On discrete rational least squares approximation

Summary The paper deals with the finite rational least squares approximation to a discrete function. An approximation without poles and depending on a parameter is defined which tends to the least squares approximation for 0. It gives an acceptable approximation when the least squares approximation does not exist. Further it is shown that, if the discrete function to be fitted is sufficiently close to a rational function, then the least squares approximation exists.     

On the stability the least squares Monte Carlo

Consider least squares Monte Carlo (LSM) algorithm, which is proposed by Longstaff and Schwartz (Rev Financial Studies 14:113–147, 2001) for pricing American style securities. This algorithm is based on the projection of the value of continuation onto a certain set of basis functions via the least squares problem. We analyze the stability of the algorithm when the number of exercise dates increases and prove that, if the underlying process for the stock price is continuous, then the regression problem is ill-conditioned for small values of the time parameter.     

Exponential convergence in a Galerkin least squares hp-FEM for Stokes flow

A stabilized hp-finite element method (FEM) of Galerkin leastsquares (GLS) type is analysed for the Stokes equations in polygonaldomains. Contrary to the standard Galerkin FEM, the GLSFEM admitsthe implementationally attractive equal-order interpolationin the velocity and the pressure. In conjunction with geometricallyrefined meshes and linearly increasing approximation ordersit is shown that the hp-GLSFEM leads to exponential rates ofconvergence for solutions exhibiting singularities near corners.To obtain this result a novel hp-interpolation result is provedthat allows the approximation of pressure functions in certainweighted Sobolev spaces in a conforming way and at exponentialrates of convergence on geometric meshes. Received 6 June 1999. Accepted 14 March 2000.     

An algorithm for least squares analysis of spectroscopic data

This paper describes a variant of the Gauss-Newton-Hartley algorithm for nonlinear least squares, in which aQR implementation is used to solve the linear least squares problem. We follow Grey's idea of updating variables at intermediate stages of the orthogonalization. This technique, applied in partitions identified with known or suspected spectral lines, appears to be especially suited to the analysis of spectroscopic data. We suggest that this algorithm is an attractive candidate for the optimization role in Ekenberg's interactive computer graphics curve fitting program.     

The limit of the partial sums process of spatial least squares residuals

We establish a functional central limit theorem for a sequence of least squares residuals of spatial data from a linear regression model. Under mild assumptions on the model we explicitly determine the limit process in the case where the assumed linear model is true. Moreover, in the case where the assumed linear model is not true we explicitly establish the limit process for the localized true regression function under mild conditions. These results can be used to develop non-parametric model checks for linear regression. Our proofs generalize ideas of a univariate geometrical approach due to Bischoff [W. Bischoff, The structure of residual partial sums limit processes of linear regression models, Theory Stoch. Process. 8 (24) (2002) 23–28] which is different to that proposed by MacNeill and Jandhyala [I.B. MacNeill, V.K. Jandhyala, Change-point methods for spatial data, in: G.P. Patil, et al. (Eds.), Multivariate Environmental Statistics. Papers Presented at the 7th International Conference on Multivariate Analysis held at Pennsylvania State University, University Park, PA, USA, May 5–9 1992, in: Ser. Stat. Probab., vol. 6, North-Holland, Amsterdam, 1993, pp. 289–306 (in English)]. Moreover, Xie and MacNeill [L. Xie, I.B. MacNeill, Spatial residual processes and boundary detection, South African Statist. J. 40 (1) (2006) 33–53] established the limit process of set indexed partial sums of regression residuals. In our framework we get that result as an immediate consequence of a result of Alexander and Pyke [K.S. Alexander, R. Pyke, A uniform central limit theorem for set-indexed partial-sum processes with finite variance, Ann. Probab. 14 (1986) 582–597]. The reason for that is that by our geometrical approach we recognize the structure of the limit process: it is a projection of the Brownian sheet onto a certain subspace of the reproducing kernel Hilbert space of the Brownian sheet. Several examples are discussed.     

Local convergence of a secant type method for solving least squares problems

Local convergence of a secant type iterative method for approximating a solution of nonlinear least squares problems is investigated in this paper. The radius of convergence is determined as well as usable error estimates. Numerical examples are also provided.     

Algebraic connections between the least squares and total least squares problems

Summary In this paper the closeness of the total least squares (TLS) and the classical least squares (LS) problem is studied algebraically. Interesting algebraic connections between their solutions, their residuals, their corrections applied to data fitting and their approximate subspaces are proven.All these relationships point out the parameters which mainly determine the equivalences and differences between the two techniques. These parameters also lead to a better understanding of the differences in sensitivity between both approaches with respect to perturbations of the data.In particular, it is shown how the differences between both approaches increase when the equationsAXB become less compatible, when the length ofB orX is growing or whenA tends to be rank-deficient. They are maximal whenB is parallel with the singular vector ofA associated with its smallest singular value. Furthermore, it is shown how TLS leads to a weighted LS problem, and assumptions about the underlying perturbation model of both techniques are deduced. It is shown that many perturbation models correspond with the same TLS solution.Senior Research Assistant of the Belgian N.F.W.O. (National Fund of Scientific Research)