Adaptive Local Linear Quantile Regression

In this paper we propose a new method of local linear adaptive smoothing for nonparametric conditional quantile regression. Some theoretical properties of the procedure are investigated. Then we demonstrate the performance of the method on a simulated example and compare it with other methods. The simulation results demonstrate a reasonable performance of our method proposed especially in situations when the underlying image is piecewise linear or can be approximated by such images. Generally speaking, our ...     

半参数空间变系数模型的局部线性二阶段估计

针对半参数空间变系数模型,利用二阶段估计方法借助于局部线性拟合技术给出了模型中常值系数估计量的精确表达式;同时给出了实验设计方法,用于验证所提出的估计方法对估计常值系数具有满意的精度和稳定性.     

����Lasso�ͷ��ļ�����ʧ������άЭ����������ϡ�辫�Ⱦ������

??This paper develops a covariate-adjusted precision matrix estimation using a two-stage estimation procedure. Firstly, we identify the relevant covariates that affect the means by a joint l_1 penalization. Then, the estimated regression coefficients are used to estimate the mean values in a multivariate sub-Gaussian model in order to estimate the sparse precision matrix through a Lasso penalized D-trace loss. Under some assumptions, we establish the convergence rate of the precision matrix estimation under different norms and demonstrate the sparse recovery property with probability converging to one. Simulation shows that our methods have the finite-sample performance compared with other methods.     

Adjusting for High-Dimensional Covariates in Sparse Precision Matrix Estimation by Lasso Penalized D-Trace Loss

This paper develops a covariate-adjusted precision matrix estimation using a two-stage estimation procedure. Firstly, we identify the relevant covariates that affect the means by a joint l_1 penalization. Then, the estimated regression coefficients are used to estimate the mean values in a multivariate sub-Gaussian model in order to estimate the sparse precision matrix through a Lasso penalized D-trace loss. Under some assumptions, we establish the convergence rate of the precision matrix estimation under different norms and demonstrate the sparse recovery property with probability converging to one. Simulation shows that our methods have the finite-sample performance compared with other methods.     

Multiclass Probability Estimation With Support Vector Machines

Multiclass classification and probability estimation have important applications in data analytics. Support vector machines (SVMs) have shown great success in various real-world problems due to their high classification accuracy. However, one main limitation of standard SVMs is that they do not provide class probability estimates, and thus fail to offer uncertainty measure about class prediction. In this article, we propose a simple yet effective framework to endow kernel SVMs with the feature of multiclass probability estimation. The new probability estimator does not rely on any parametric assumption on the data distribution, therefore, it is flexible and robust. Theoretically, we show that the proposed estimator is asymptotically consistent. Computationally, the new procedure can be conveniently implemented using standard SVM softwares. Our extensive numerical studies demonstrate competitive performance of the new estimator when compared with existing methods such as multiple logistic regression, linear discrimination analysis, tree-based methods, and random forest, under various classification settings. Supplementary materials for this article are available online.     

Wavelet-Based Weighted LASSO and Screening Approaches in Functional Linear Regression

One useful approach for fitting linear models with scalar outcomes and functional predictors involves transforming the functional data to wavelet domain and converting the data-fitting problem to a variable selection problem. Applying the LASSO procedure in this situation has been shown to be efficient and powerful. In this article, we explore two potential directions for improvements to this method: techniques for prescreening and methods for weighting the LASSO-type penalty. We consider several strategies for each of these directions which have never been investigated, either numerically or theoretically, in a functional linear regression context. We compare the finite-sample performance of the proposed methods through both simulations and real-data applications with both 1D signals and 2D image predictors. We also discuss asymptotic aspects. We show that applying these procedures can lead to improved estimation and prediction as well as better stability. Supplementary materials for this article are available online.     

部分线性单指标模型的复合分位数回归及变量选择

本文提出复合最小化平均分位数损失估计方法 (composite minimizing average check loss estimation,CMACLE)用于实现部分线性单指标模型(partial linear single-index models,PLSIM)的复合分位数回归(composite quantile regression,CQR).首先基于高维核函数构造参数部分的复合分位数回归意义下的相合估计,在此相合估计的基础上,通过采用指标核函数进一步得到参数和非参数函数的可达最优收敛速度的估计,并建立所得估计的渐近正态性,比较PLSIM的CQR估计和最小平均方差估计(MAVE)的相对渐近效率.进一步地,本文提出CQR框架下PLSIM的变量选择方法,证明所提变量选择方法的oracle性质.随机模拟和实例分析验证了所提方法在有限样本时的表现,证实了所提方法的优良性.     

Local linear regression for data with AR errors

In many statistical applications, data are collected over time, and they are likely correlated. In this paper, we investigate how to incorporate the correlation information into the local linear regression. Under the assumption that the error process is an auto-regressive process, a new estimation procedure is proposed for the nonparametric regression by using local linear regression method and the profile least squares techniques. We further propose the SCAD penalized profile least squares method to determine the order of auto-regressive process. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed procedure, and to compare the performance of the proposed procedures with the existing one. From our empirical studies, the newly proposed procedures can dramatically improve the accuracy of naive local linear regression with working-independent error structure. We illustrate the proposed methodology by an analysis of real data set.     

Bagging算法在混合适应性样条中的应用

本文通过分析Bagging算法在混合适应性样条估计中的作用，提出了一种新的空间适应性非参数函数估计算法．模拟研究表明该算法在空间适应性上具有明显优势，在估计精度和稳定性方面也可与其他同类方法相媲美，并且易于推广到多元情形．     

Interaction Model and Model Selection for Function-on-Function Regression

Regression models with interaction effects have been widely used in multivariate analysis to improve model flexibility and prediction accuracy. In functional data analysis, however, due to the challenges of estimating three-dimensional coefficient functions, interaction effects have not been considered for function-on-function linear regression. In this article, we propose function-on-function regression models with interaction and quadratic effects. For a model with specified main and interaction effects, we propose an efficient estimation method that enjoys a minimum prediction error property and has good predictive performance in practice. Moreover, converting the estimation of three-dimensional coefficient functions of the interaction effects to the estimation of two- and one-dimensional functions separately, our method is computationally efficient. We also propose adaptive penalties to account for varying magnitudes and roughness levels of coefficient functions. In practice, the forms of the models are usually unspecified. We propose a stepwise procedure for model selection based on a predictive criterion. This method is implemented in our R package FRegSigComp. Supplemental materials are available online.     

Estimation and test procedures for composite quantile regression with covariates missing at random

In this paper, we study the weighted composite quantile regression (WCQR) for general linear model with missing covariates. We propose the WCQR estimation and bootstrap test procedures for unknown parameters. Simulation studies and a real data analysis are conducted to examine the finite performance of our proposed methods.     

随机缺失数据下样本分位数估计

分位数的估计在生物医学、社会经济调查等领域有着广泛的应用,然而在实际问题的研究中,往往由于各种人为或不可控因素造成数据收集不完全.本文在随机缺失(MAR)假设条件下,利用非参数核补法和局部多重插补法给出了响应变量缺失时样本分位数的估计,并利用经验过程等理论证明了由这两种方法得到的分位数估计的大样本性质,同时,使用重抽样方法给出了估计的渐近方差的估计,模拟结果验证了这两种方法的有效性.文章所提两种方法的优点在于:首先,所提出的缺失修正方法不需要对缺失概率的模型做任何假设;其次,方法亦适用于其他有关参数不可微的估计目标函数;最后,方法很容易地推广到一般M估计的情况,并可以对多个分位数同时进行估计.     

The Fast RODEO for Local Polynomial Regression

An open challenge in nonparametric regression is finding fast, computationally efficient approaches to estimating local bandwidths for large datasets, in particular in two or more dimensions. In the work presented here, we introduce a novel local bandwidth estimation procedure for local polynomial regression, which combines the greedy search of the regularization of the derivative expectation operator (RODEO) algorithm with linear binning. The result is a fast, computationally efficient algorithm, which we refer to as the fast RODEO. We motivate the development of our algorithm by using a novel scale-space approach to derive the RODEO. We conclude with a toy example and a real-world example using data from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite validation study, where we show the fast RODEO’s improvement in accuracy and computational speed over two other standard approaches.     

Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data

A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123–139, 2002)’s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.     

Stochastic Generalized Method of Moments

The generalized method of moments (GMM) is a very popular estimation and inference procedure based on moment conditions. When likelihood-based methods are difficult to implement, one can often derive various moment conditions and construct the GMM objective function. However, minimization of the objective function in the GMM may be challenging, especially over a large parameter space. Due to the special structure of the GMM, we propose a new sampling-based algorithm, the stochastic GMM sampler, which replaces the multivariate minimization problem by a series of conditional sampling procedures. We develop the theoretical properties of the proposed iterative Monte Carlo method, and demonstrate its superior performance over other GMM estimation procedures in simulation studies. As an illustration, we apply the stochastic GMM sampler to a Medfly life longevity study. Supplemental materials for the article are available online.     

Chebyshev center based column generation

The classical column generation approach often shows a very slow convergence. Many different acceleration techniques have been proposed recently to improve the convergence. Here, we briefly survey these methods and propose a novel algorithm based on the Chebyshev center of the dual polyhedron. The Chebyshev center can be obtained by solving a linear program; consequently, the proposed method can be applied with small modifications on the classical column generation procedure. We also show that the performance of our algorithm can be enhanced by introducing proximity parameters which enable the position of the Chebyshev center to be adjusted. Numerical experiments are conducted on the binpacking, vehicle routing problem with time windows, and the generalized assignment problem. The computational results of these experiments demonstrate the effectiveness of our proposed method.     

Indirect assessment of the bivariate survival function

Estimating the bivariate survival function has been a major goal of many researchers. For that purpose many methods and techniques have been published. However, most of these techniques and methods rely heavily on bivariate failure data. There are situations in which failure time data are difficult to obtain and thus there is a growing need to assess the bivariate survival function for such cases. In this paper we propose two techniques for generating families of bivariate processes for describing several variables that can be used to indirectly assess the bivariate survival function. An estimation procedure is provided and a simulation study is conducted to evaluate the performance of our proposed estimator.     

部分线性混合效应模型中方差分量的稳健估计

部分线性混合效应模型中方差分量是我们感兴趣的参数, 文献中已经给出许多估计方法. 但是其中很多方法都可以归结为广义估计方程方法(GEE), 如: 最大似然估计(MLE), 约束最大似然估计(REMLE)等, 而GEE方法对异常点很敏感. 本文提出一组关于部分线性混合效应模型(PLMM)中均值和方差分量的稳健估计方程, 对均值和方差分量同时进行稳健估计; 并进行了随机模拟考察所提出稳健估计的有效性, 最后通过两个实例, 说明了所提方法的可行性.     

Interquantile Shrinkage in Regression Models

Conventional analysis using quantile regression typically focuses on fitting the regression model at different quantiles separately. However, in situations where the quantile coefficients share some common feature, joint modeling of multiple quantiles to accommodate the commonality often leads to more efficient estimation. One example of common features is that a predictor may have a constant effect over one region of quantile levels but varying effects in other regions. To automatically perform estimation and detection of the interquantile commonality, we develop two penalization methods. When the quantile slope coefficients indeed do not change across quantile levels, the proposed methods will shrink the slopes toward constant and thus improve the estimation efficiency. We establish the oracle properties of the two proposed penalization methods. Through numerical investigations, we demonstrate that the proposed methods lead to estimations with competitive or higher efficiency than the standard quantile regression estimation in finite samples. Supplementary materials for the article are available online.     

Jump-Preserving Regression and Smoothing using Local Linear Fitting: A Compromise

This paper deals with nonparametric estimation of a regression curve, where the estimation method should preserve possible jumps in the curve. At each point x at which one wants to estimate the regression function, the method chooses in an adaptive way among three estimates: a local linear estimate using only datapoints to the left of x, a local linear estimate based on only datapoints to the right of x, and finally a local linear estimate using data in a two-sided neighbourhood around x. The choice among these three estimates is made by looking at differences of the weighted residual mean squares of the three fits. The resulting estimate preserves the jumps well and in addition gives smooth estimates of the continuity parts of the curve. This property of compromise between local smoothing and jump-preserving is what distinguishes our method from most previously proposed methods, that mainly focused on local smoothing and consequently blurred possible jumps, or mainly focused on jump-preserving and hence led to rather noisy estimates in continuity regions of the underlying regression curve. Strong consistency of the estimator is established and its performance is tested via a simulation study. This study also compares the current method with some existing methods. The current method is illustrated in analyzing a real dataset.