Regularized Principal Component Analysis for Spatial Data

In many atmospheric and earth sciences, it is of interest to identify dominant spatial patterns of variation based on data observed at p locations and n time points with the possibility that p > n. While principal component analysis (PCA) is commonly applied to find the dominant patterns, the eigenimages produced from PCA may exhibit patterns that are too noisy to be physically meaningful when p is large relative to n. To obtain more precise estimates of eigenimages, we propose a regularization approach incorporating smoothness and sparseness of eigenimages, while accounting for their orthogonality. Our method allows data taken at irregularly spaced or sparse locations. In addition, the resulting optimization problem can be solved using the alternating direction method of multipliers, which is easy to implement, and applicable to a large spatial dataset. Furthermore, the estimated eigenfunctions provide a natural basis for representing the underlying spatial process in a spatial random-effects model, from which spatial covariance function estimation and spatial prediction can be efficiently performed using a regularized fixed-rank kriging method. Finally, the effectiveness of the proposed method is demonstrated by several numerical examples.     

Prediction of an outcome using NETwork Clusters (NET-C)

Birth weight is a key consequence of environmental exposures and metabolic alterations and can influence lifelong health. While a number of methods have been used to examine associations of trace element (including essential nutrients and toxic metals) concentrations or metabolite concentrations with a health outcome, birth weight, studies evaluating how the coexistence of these factors impacts birth weight are extremely limited. Here, we present a novel algorithm NETwork Clusters (NET-C), to improve the prediction of outcome by considering the interactions of features in the network and then apply this method to predict birth weight by jointly modelling trace element and cord blood metabolite data. Specifically, by using trace element and/or metabolite subnetworks as groups, we apply group lasso to estimate birth weight. We conducted statistical simulation studies to examine how both sample size and correlations between grouped features and the outcome affect prediction performance. We showed that in terms of prediction error, our proposed method outperformed other methods such as (a) group lasso with groups defined by hierarchical clustering, (b) random forest regression and (c) neural networks. We applied our method to data ascertained as part of the New Hampshire Birth Cohort Study on trace elements, metabolites and birth outcomes, adjusting for other covariates such as maternal body mass index (BMI) and enrollment age. Our proposed method can be applied to a variety of similarly structured high-dimensional datasets to predict health outcomes.     

Isolation and structure determination of a new lasso peptide subterisin from Sphingomonas subterranea

A new lasso peptide named subterisin was isolated from the culture broth of Sphingomonas subterranea NBRC 16086^T. The molecular formula of subterisin was established as C₇₈H₁₂₁O₂₂N₂₁ based on accurate mass analysis. The chemical structure of subterisin was determined by 2D NMR experiments. The presence of macrolactam ring of Gly1–Glu8 was indicated by NOESY experiment and MS/MS analysis. The three-dimensional structure of subterisin in solution was established by calculation based on NMR data. The proposed biosynthetic gene cluster of subterisin was found on the genome of S. subterranea.     

Graphical Models for Ordinal Data

This article considers a graphical model for ordinal variables, where it is assumed that the data are generated by discretizing the marginal distributions of a latent multivariate Gaussian distribution. The relationships between these ordinal variables are then described by the underlying Gaussian graphical model and can be inferred by estimating the corresponding concentration matrix. Direct estimation of the model is computationally expensive, but an approximate EM-like algorithm is developed to provide an accurate estimate of the parameters at a fraction of the computational cost. Numerical evidence based on simulation studies shows the strong performance of the algorithm, which is also illustrated on datasets on movie ratings and an educational survey.     

Estimation Stability With Cross-Validation (ESCV)

Cross-validation (CV) is often used to select the regularization parameter in high-dimensional problems. However, when applied to the sparse modeling method Lasso, CV leads to models that are unstable in high-dimensions, and consequently not suited for reliable interpretation. In this article, we propose a model-free criterion ESCV based on a new estimation stability (ES) metric and CV. Our proposed ESCV finds a smaller and locally ES-optimal model smaller than the CV choice so that it fits the data and also enjoys estimation stability property. We demonstrate that ESCV is an effective alternative to CV at a similar easily parallelizable computational cost. In particular, we compare the two approaches with respect to several performance measures when applied to the Lasso on both simulated and real datasets. For dependent predictors common in practice, our main finding is that ESCV cuts down false positive rates often by a large margin, while sacrificing little of true positive rates. ESCV usually outperforms CV in terms of parameter estimation while giving similar performance as CV in terms of prediction. For the two real datasets from neuroscience and cell biology, the models found by ESCV are less than half of the model sizes by CV, but preserves CV's predictive performance and corroborates with subject knowledge and independent work. We also discuss some regularization parameter alignment issues that come up in both approaches. Supplementary materials are available online.     

Adaptive Lasso estimators for ultrahigh dimensional generalized linear models

Generalized linear models have been more widely used than linear models which exclude categorical variables. The penalized method becomes an effective tool to study ultrahigh dimensional generalized linear models. In this paper, we study theoretical results of the adaptive Lasso for generalized linear models in terms of diverging number of parameters and ultrahigh dimensionality. The asymptotic results are examined by several simulation studies.     

ConvexLAR: An Extension of Least Angle Regression

The least angle regression (LAR) was proposed by Efron, Hastie, Johnstone and Tibshirani in the year 2004 for continuous model selection in linear regression. It is motivated by a geometric argument and tracks a path along which the predictors enter successively and the active predictors always maintain the same absolute correlation (angle) with the residual vector. Although it gains popularity quickly, its extensions seem rare compared to the penalty methods. In this expository article, we show that the powerful geometric idea of LAR can be generalized in a fruitful way. We propose a ConvexLAR algorithm that works for any convex loss function and naturally extends to group selection and data adaptive variable selection. After simple modification, it also yields new exact path algorithms for certain penalty methods such as a convex loss function with lasso or group lasso penalty. Variable selection in recurrent event and panel count data analysis, Ada-Boost, and Gaussian graphical model is reconsidered from the ConvexLAR angle. Supplementary materials for this article are available online.     

Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEPs). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR), and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA, and SDR. Supplementary materials for this article are available online.     

Automatic Feature Selection via Weighted Kernels and Regularization

Selecting important features in nonlinear kernel spaces is a difficult challenge in both classification and regression problems. This article proposes to achieve feature selection by optimizing a simple criterion: a feature-regularized loss function. Features within the kernel are weighted, and a lasso penalty is placed on these weights to encourage sparsity. This feature-regularized loss function is minimized by estimating the weights in conjunction with the coefficients of the original classification or regression problem, thereby automatically procuring a subset of important features. The algorithm, KerNel Iterative Feature Extraction (KNIFE), is applicable to a wide variety of kernels and high-dimensional kernel problems. In addition, a modification of KNIFE gives a computationally attractive method for graphically depicting nonlinear relationships between features by estimating their feature weights over a range of regularization parameters. The utility of KNIFE in selecting features through simulations and examples for both kernel regression and support vector machines is demonstrated. Feature path realizations also give graphical representations of important features and the nonlinear relationships among variables. Supplementary materials with computer code and an appendix on convergence analysis are available online.     

Extending the Iterative Convex Minorant Algorithm to the Cox Model for Interval-Censored Data

Abstract

The iterative convex minorant (ICM) algorithm proposed by Groeneboom and Wellner is fast in computing the NPMLE of the distribution function for interval censored data without covariates. We reformulate the ICM as a generalized gradient projection method (GGP), which leads to a natural extension to the Cox model. It is also easily extended to support Tibshirani's Lasso method. Some simulation results are also shown. For illustration we reanalyze two real datasets.