On Adaptive Combination of Regression Estimators

Consider the problem of choosing between two estimators of the regression function, where one estimator is based on stronger assumptions than the other and thus the rates of convergence are different. We propose a linear combination of the estimators where the weights are estimated by Mallows' C _L . The adaptive estimator retains the optimal rates of convergence and is an extension of Stein-type estimators considered by Li and Hwang (1984, Ann. Statist., 12, 887-897) and related to an estimator in Burman and Chaudhuri (1999, Ann. Inst. Statist. Math. (to appear)).     

One-Step Estimator Paths for Concave Regularization

The statistics literature of the past 15 years has established many favorable properties for sparse diminishing-bias regularization: techniques that can roughly be understood as providing estimation under penalty functions spanning the range of concavity between ?₀ and ?₁ norms. However, lasso ?₁-regularized estimation remains the standard tool for industrial Big Data applications because of its minimal computational cost and the presence of easy-to-apply rules for penalty selection. In response, this article proposes a simple new algorithm framework that requires no more computation than a lasso path: the path of one-step estimators (POSE) does ?₁ penalized regression estimation on a grid of decreasing penalties, but adapts coefficient-specific weights to decrease as a function of the coefficient estimated in the previous path step. This provides sparse diminishing-bias regularization at no extra cost over the fastest lasso algorithms. Moreover, our gamma lasso implementation of POSE is accompanied by a reliable heuristic for the fit degrees of freedom, so that standard information criteria can be applied in penalty selection. We also provide novel results on the distance between weighted-?₁ and ?₀ penalized predictors; this allows us to build intuition about POSE and other diminishing-bias regularization schemes. The methods and results are illustrated in extensive simulations and in application of logistic regression to evaluating the performance of hockey players. Supplementary materials for this article are available online.     

Robust goodness-of-fit tests for AR(p) models based onL 1-norm fitting

A robustified residual autocorrelation is defined based onL ₁-regression. Under very general conditions, the asymptotic distribution of the robust residual autocorrelation is obtained. A robustified portmanteau statistic is then constructed which can be used in checking the goodness-of-fit of AR(p) models when usingL ₁-norm fitting. Empirical results show thatL ₁-norm estimators and the proposed portmanteau statistic are robust against outliers, error distributions, and accuracy for a given finite sample. Project supported by the Foundation of State Educational Commission and a research grant from the Doctoral Program Foundation of China (#97000139).     

Geometric dual formulation for first-derivative-based univariate cubic <Emphasis Type="Italic">L</Emphasis><Subscript>1</Subscript> splines

With the objective of generating “shape-preserving” smooth interpolating curves that represent data with abrupt changes in magnitude and/or knot spacing, we study a class of first-derivative-based -smooth univariate cubic L ₁ splines. An L ₁ spline minimizes the L ₁ norm of the difference between the first-order derivative of the spline and the local divided difference of the data. Calculating the coefficients of an L ₁ spline is a nonsmooth non-linear convex program. Via Fenchel’s conjugate transformation, the geometric dual program is a smooth convex program with a linear objective function and convex cubic constraints. The dual-to-primal transformation is accomplished by solving a linear program.     

Lp-Lq DECAY ESTIMATES FOR HYPERBOLIC EQUATIONS WITH OSCILLATIONS IN COEFFICIENTS

This work is concerned with the proof of Lp -Lq decay estimates for solutions of the Cauchy problem for utt -λ2(t)b2(t)/Δu =0. The coefficient consists of an increasing smooth function λ and an oscillating smooth and bounded function b which are uniformly separated from zero. The authors‘ main interest is devoted to the critical case where one has an interesting interplay between the growing and the oscillating part.     

<Emphasis Type="Italic">L</Emphasis><Subscript><Emphasis Type="Italic">p</Emphasis></Subscript>-mixed intersection bodies

In this paper the author first introduce a new concept of L _p -dual mixed volumes of star bodies which extends the classical dual mixed volumes. Moreover, we extend the notions of L _p intersection body to L _p -mixed intersection body. Inequalities for L _p -dual mixed volumes of L _p -mixed intersection bodies are established and the results established here provide new estimates for these type of inequalities. This work was supported by the Natural Science Foundation of Zhejiang Province of China (Grant No. Y605065) and the Foundation of the Education Department of Zhejiang Province of China (Grant No. 20050392)     

On Closability of Directional Gradients

Let be a centred Gaussian measure on a separable real Banach space E, and let H be a Hilbert subspace of E. We provide necessary and sufficient conditions for closability in L ^p (E,) of the gradient D _H in the direction of H. These conditions are further elaborated in case when the gradient D _H corresponds to a bilinear form associated with a certain nonsymmetric Ornstein–Uhlenbeck operator. Some natural examples of closability and nonclosability are presented.     

Integral Formulas for the r-Mean Curvature Linearized Operator of a Hypersurface

For a normal variation of a hypersurface M ⁿ in a space form Q _c ⁿ⁺¹ by a normal vector field fN, R. Reilly proved:

Approximating term structure of interest rates using cubic L1 splines

Classical spline fitting methods for estimating the term structure of interest rates have been criticized for generating highly fluctuating fitting curves for bond price and discount function. In addition, the performance of these methods usually relies heavily on parameter tuning involving human judgement. To overcome these drawbacks, a recently developed cubic L₁ spline model is proposed for term structure analysis. Cubic L₁ splines preserve the shape of the data, exhibit no extraneous oscillation and have small fitting errors. Cubic L₁ splines are tested using a set of real financial data and compared with the widely used B-splines.     

Model selection via adaptive shrinkage with t priors

We discuss a model selection procedure, the adaptive ridge selector, derived from a hierarchical Bayes argument, which results in a simple and efficient fitting algorithm. The hierarchical model utilized resembles an un-replicated variance components model and leads to weighting of the covariates. We discuss the intuition behind this type estimator and investigate its behavior as a regularized least squares procedure. While related alternatives were recently exploited to simultaneously fit and select variablses/features in regression models (Tipping in J Mach Learn Res 1:211–244, 2001; Figueiredo in IEEE Trans Pattern Anal Mach Intell 25:1150–1159, 2003), the extension presented here shows considerable improvement in model selection accuracy in several important cases. We also compare this estimator’s model selection performance to those offered by the lasso and adaptive lasso solution paths. Under randomized experimentation, we show that a fixed choice of tuning parameter leads to results in terms of model selection accuracy which are superior to the entire solution paths of lasso and adaptive lasso when the underlying model is a sparse one. We provide a robust version of the algorithm which is suitable in cases where outliers may exist.     

Some complete metrics on spaces of fuzzy subsets

The classes of the L_p,∞- and L_p-metrics play an important role to develop a probability theory in fuzzy sample spaces. All of these metrics are known to be separable, but not complete. The classes are closely related as for each L_p,∞-metric there exists some L_p-metric which induces the same topology. This paper deals with the completion of the L_p,∞- and L_p-metrics. We can also show that the relationship between the classes of L_p,∞- and L_p-metrics still holds for the obtained respective classes of their completions.     

L p c -concentration of blow-up solutions for the biharmonic nonlinear Schrödinger equation

This article is concerned with the blow-up solutions of the biharmonic Schrödinger equation with L ²-super critical nonlinearity. We obtain the nonexistence of strong limit of L ^{p _c} -norm and L ^{p _c} -concentration properties of the radially symmetric blow-up solutions, where L ^{p _c} is the invariant Lebesgue space.     

L1 regression estimate and its bootstrap

We study the asymptotic distribution of the L ₁ regression estimator under general conditions with matrix norming and possibly non i.i.d. errors. We then introduce an appropriate bootstrap procedure to estimate the distribution of this estimator and study its asymptotic properties. It is shown that this bootstrap is consistent under suitable conditions and in other situations the bootstrap limit is a random distribution. This work was supported by J.C. Bose National Fellowship, Government of India     

Lp-Approximable Sequences of Vectors and Limit Distribution of Quadratic Forms of Random Variables

The properties of L₂-approximable sequences established here form a complete toolkit for statistical results concerning weighted sums of random variables, where the weights are nonstochastic sequences approximated in some sense by square-integrable functions and the random variables are “two-wing” averages of martingale differences. The results constitute the first significant advancement in the theory of L₂-approximable sequences since 1976 when Moussatat introduced a narrower notion of L₂-generated sequences. The method relies on a study of certain linear operators in the spaces L_p and l_p. A criterion of L_p-approximability is given. The results are new even when the weight generating function is identically 1. A central limit theorem for quadratic forms of random variables illustrates the method.     

The L p -curvature images of convex bodies and L p -projection bodies

Associated with the L _p -curvature image defined by Lutwak, some inequalities for extended mixed p-affine surface areas of convex bodies and the support functions of L _p -projection bodies are established. As a natural extension of a result due to Lutwak, an L _p -type affine isoperimetric inequality, whose special cases are L _p -Busemann-Petty centroid inequality and L _p -affine projection inequality, respectively, is established. Some L _p -mixed volume inequalities involving L _p -projection bodies are also established.     

Point sets with low <Emphasis Type="Italic">L</Emphasis><Subscript><Emphasis Type="Italic">p</Emphasis></Subscript>-discrepancy

In this paper we study the L _p -discrepancy of digitally shifted Hammersley point sets. While it is known that the (unshifted) Hammersley point set (which is also known as Roth net) with N points has L _p -discrepancy (p an integer) of order (log N)/N, we show that there always exists a shift such that the digitally shifted Hammersley point set has L _p -discrepancy (p an even integer) of order which is best possible by a result of W. Schmidt. Further we concentrate on the case p = 2. We give very tight lower and upper bounds for the L ₂-discrepancy of digitally shifted Hammersley point sets which show that the value of the L ₂-discrepancy of such a point set mostly depends on the number of zero coordinates of the shift and not so much on the position of these. This work is supported by the Austrian Research Fund (FWF), Project P17022-N12 and Project S8305.     

Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error

In many problems involving generalized linear models, the covariates are subject to measurement error. When the number of covariates p exceeds the sample size n, regularized methods like the lasso or Dantzig selector are required. Several recent papers have studied methods which correct for measurement error in the lasso or Dantzig selector for linear models in the p > n setting. We study a correction for generalized linear models, based on Rosenbaum and Tsybakov’s matrix uncertainty selector. By not requiring an estimate of the measurement error covariance matrix, this generalized matrix uncertainty selector has a great practical advantage in problems involving high-dimensional data. We further derive an alternative method based on the lasso, and develop efficient algorithms for both methods. In our simulation studies of logistic and Poisson regression with measurement error, the proposed methods outperform the standard lasso and Dantzig selector with respect to covariate selection, by reducing the number of false positives considerably. We also consider classification of patients on the basis of gene expression data with noisy measurements. Supplementary materials for this article are available online.     

Exact L 2-norm plane separation

We consider the problem of separating two sets of points in an n-dimensional real space with a (hyper)plane that minimizes the sum of L _p -norm distances to the plane of points lying on the wrong side of it. Despite recent progress, practical techniques for the exact solution of cases other than the L ₁ and L _∞-norm were unavailable. We propose and implement a new approach, based on non-convex quadratic programming, for the exact solution of the L ₂-norm case. We solve in reasonable computing times artificial problems of up to 20000 points (in 6 dimensions) and 13 dimensions (with 2000 points). We also observe that, for difficult real-life instances from the UCI Repository, computation times are substantially reduced by incorporating heuristic results in the exact solution process. Finally, we compare the classification performance of the planes obtained for the L ₁, L ₂ and L _∞ formulations. It appears that, despite the fact that L ₂ formulation is computationally more expensive, it does not give significantly better results than the L ₁ and L _∞ formulations.     

H2 optimal model order reduction on the Stiefel manifold for the MIMO discrete system by the cross Gramian

ABSTRACT

In this paper, the H₂ optimal model order reduction method for the large-scale multiple-input multiple-output (MIMO) discrete system is investigated. First, the MIMO discrete system is resolved into a number of single-input single-output (SISO) subsystems, and the H₂ norm of the original MIMO discrete system is expressed by the cross Gramian of each subsystem. Then, the retraction and the vector transport on the Stiefel manifold are introduced, and the geometric conjugate gradient model order reduction method is proposed. The reduced system of the original MIMO discrete system is generated by using the proposed method. Finally, two numerical examples show the efficiency of the proposed method.     

Automatic Smoothing Spline Projection Pursuit

Abstract

A highly flexible nonparametric regression model for predicting a response y given covariates {x_k}^d _k=1 is the projection pursuit regression (PPR) model ? = h(x) = β₀ + Σ_jβ_jf_j(α^T _jx) where the f_j , are general smooth functions with mean 0 and norm 1, and Σ^d _k=1α² _kj=1. The standard PPR algorithm of Friedman and Stuetzle (1981) estimates the smooth functions f_j using the supersmoother nonparametric scatterplot smoother. Friedman's algorithm constructs a model with M _max linear combinations, then prunes back to a simpler model of size M ≤ M _max, where M and M _max are specified by the user. This article discusses an alternative algorithm in which the smooth functions are estimated using smoothing splines. The direction coefficients α_j, the amount of smoothing in each direction, and the number of terms M and M _max are determined to optimize a single generalized cross-validation measure.