单调缺失机制下高维纵向线性回归模型的变量选择

在响应变量带有单调缺失的情形下考虑高维纵向线性回归模型的变量选择.主要基于逆概率加权广义估计方程提出了一种自动的变量选择方法,该方法不使用现有的惩罚函数,不涉及惩罚函数非凸最优化的问题,并且可以自动地剔除零回归系数,同时得到非零回归系数的估计.在一定正则条件下,证明了该变量选择方法具有Oracle性质.最后,通过模拟研究验证了所提出方法的有限样本性质.     

一类非参数回归模型核估计的渐近性质

研究一类新的非参数回归模型回归函数的核估计问题,其中误差项为一阶非参数自回归方程.通过重复利用Watson-Nadaraya核估计方法,构造了回归函数及误差回归函数的估计量分别为m(.)和ρ(.),在适当的条件下,证明了估计量m(.)和ρ(.)的渐近正态性.     

部分线性自回归模型中回归函数的半参数估计（英文）

本文主要研究具有一阶自回归误差的三阶部分线性自回归模型中回归函数的半参数估计问题.假定回归函数来自某个参数分布族,利用条件最小二乘法得到参数估计量,再结合非参数核函数进行调整,给出回归函数的半参数估计量.并在一定条件下,证明了估计量具有相合性.最后,通过模拟研究验证了此方法的有效性.     

动态变系数回归模型的识别与非参数GMM估计

主要研究关于面板数据的有限阶固定效应的动态变系数回归模型(简称FDVCM)的统计推断问题.基于B-样条函数和广义矩估计(简称GMM)方法,首先建立了未知系数函数的非参数GMM估计,并证明大样本情形下该估计达到最优非参数收敛速度且具有渐近正态性质.然而实际问题中模型的动态阶数完全未知,也可能存在其它冗余的回归变量,文中借助文[Fan J,Li R.Variable selection via penalized likelihood and its oracle properties.Journal of the American Statistical Association,2001,96(456):1348-1360]中的smoothly clipped absolute deviation(简称SCAD)惩罚函数同时识别真实的动态阶数和显著的外生回归变量.同时建立了压缩估计的Oracle性质,即所识别的模型与真实模型中的参数估计具有相同的渐近分布.最后,无论是数值试验还是实例数据分析都验证了本文方法的合理性和可行性.     

一种惩罚最大似然方法估计混合回归模型

本文考虑具有正态误差假设下混合回归模型的参数估计问题.由于似然函数的无界性,混合回归模型普通的最大似然估计不存在.本文提出一种惩罚最大似然方法来估计混合回归模型的参数,证明惩罚最大似然估计量(penalized maximum likelihood estimation, PMLE)具有强相合和渐近正态性.通过深入模拟研究,从估计精确性角度看,惩罚最大似然估计量有很好的表现.本文还给出一个音调感知的例子来说明理论结果的应用.     

一类具有连续变量的全局最优化问题的凸化方法

本文研究了一类非凸最优化问题的凸化方法与最优性条件的问题.利用构造含有参数的函数变换方法,将具有次正定性质的目标函数凸化,并获得了这一类非凸优化问题全局最优解的充要条件,推广了凸化方法在求解全局最优化问题方面的应用.     

函数型部分线性复合分位数回归模型的估计（英文）

本文研究函数型部分线性复合分位数回归模型的估计问题.我们采用函数型主成分分析方法分析斜率函数,回归样条逼近非参数函数.在相当宽松的条件下给出斜率函数和非参数函数的收敛速度.最后通过理论模拟和实例分析来评价我们提出的方法.     

中位数回归的贝叶斯变量选择方法

当数据呈现厚尾特征或含有异常值时,基于惩罚最小二乘或似然函数的传统变量选择方法往往表现不佳.本文基于中位数回归和贝叶斯推断方法,研究线性模型的贝叶斯变量选择问题.通过选取回归系数的Spike and Slab先验,利用贝叶斯模型选择理论提出了中位数回归的贝叶斯估计方法,并提出了有效的后验Gibbs抽样程序.大量数值模拟和波士顿房价数据分析充分说明了所提方法的有效性.     

广义非参数回归的B样本贝叶斯估计

本文主要研究广义非参数模型B样条Bayes估计 .将回归函数按照B样条基展开 ,我们不具体选择节点的个数 ,而是节点个数取均匀的无信息先验 ,样条函数系数取正态先验 ,用B样条函数的后验均值估计回归函数 .并给出了回归函数B样条Bayes估计的MCMC的模拟计算方法 .通过对Logistic非参数回归的模拟研究 ,表明B样条Bayes估计得到了很好的估计效果     

固定设计下半参数回归模型核估计的渐近性质

研究一类新的半参数回归模型回归函数的核估计问题,其中误差项为一阶非参数自回归过程.通过重复利用Watson-Nadaraya核估计方法,构造了回归函数及误差回归函数的估计量分别为β,g(·)和ρ(·),在适当的条件下,证明了估计量β,g(·)和ρ(·)的渐近正态性.     

A Path Algorithm for Constrained Estimation

Many least-square problems involve affine equality and inequality constraints. Although there are a variety of methods for solving such problems, most statisticians find constrained estimation challenging. The current article proposes a new path-following algorithm for quadratic programming that replaces hard constraints by what are called exact penalties. Similar penalties arise in l ₁ regularization in model selection. In the regularization setting, penalties encapsulate prior knowledge, and penalized parameter estimates represent a trade-off between the observed data and the prior knowledge. Classical penalty methods of optimization, such as the quadratic penalty method, solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to ∞, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. The exact path-following method starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. Path following in Lasso penalized regression, in contrast, starts with a large value of the penalty constant and works its way downward. In both settings, inspection of the entire solution path is revealing. Just as with the Lasso and generalized Lasso, it is possible to plot the effective degrees of freedom along the solution path. For a strictly convex quadratic program, the exact penalty algorithm can be framed entirely in terms of the sweep operator of regression analysis. A few well-chosen examples illustrate the mechanics and potential of path following. This article has supplementary materials available online.     

A uniform framework for the combination of penalties in generalized structured models

Penalized estimation has become an established tool for regularization and model selection in regression models. A variety of penalties with specific features are available and effective algorithms for specific penalties have been proposed. But not much is available to fit models with a combination of different penalties. When modeling the rent data of Munich as in our application, various types of predictors call for a combination of a Ridge, a group Lasso and a Lasso-type penalty within one model. We propose to approximate penalties that are (semi-)norms of scalar linear transformations of the coefficient vector in generalized structured models—such that penalties of various kinds can be combined in one model. The approach is very general such that the Lasso, the fused Lasso, the Ridge, the smoothly clipped absolute deviation penalty, the elastic net and many more penalties are embedded. The computation is based on conventional penalized iteratively re-weighted least squares algorithms and hence, easy to implement. New penalties can be incorporated quickly. The approach is extended to penalties with vector based arguments. There are several possibilities to choose the penalty parameter(s). A software implementation is available. Some illustrative examples show promising results.     

Variable Selection via A Combination of the L0 and L1 Penalties

Variable selection is an important aspect of high-dimensional statistical modeling, particularly in regression and classification. In the regularization framework, various penalty functions are used to perform variable selection by putting relatively large penalties on small coefficients. The L₁ penalty is a popular choice because of its convexity, but it produces biased estimates for the large coefficients. The L₀ penalty is attractive for variable selection because it directly penalizes the number of non zero coefficients. However, the optimization involved is discontinuous and non convex, and therefore it is very challenging to implement. Moreover, its solution may not be stable. In this article, we propose a new penalty that combines the L₀ and L₁ penalties. We implement this new penalty by developing a global optimization algorithm using mixed integer programming (MIP). We compare this combined penalty with several other penalties via simulated examples as well as real applications. The results show that the new penalty outperforms both the L₀ and L₁ penalties in terms of variable selection while maintaining good prediction accuracy.     

Penalized likelihood regression for generalized linear models with non-quadratic penalties

One of the popular method for fitting a regression function is regularization: minimizing an objective function which enforces a roughness penalty in addition to coherence with the data. This is the case when formulating penalized likelihood regression for exponential families. Most of the smoothing methods employ quadratic penalties, leading to linear estimates, and are in general incapable of recovering discontinuities or other important attributes in the regression function. In contrast, non-linear estimates are generally more accurate. In this paper, we focus on non-parametric penalized likelihood regression methods using splines and a variety of non-quadratic penalties, pointing out common basic principles. We present an asymptotic analysis of convergence rates that justifies the approach. We report on a simulation study including comparisons between our method and some existing ones. We illustrate our approach with an application to Poisson non-parametric regression modeling of frequency counts of reported acquired immune deficiency syndrome (AIDS) cases in the UK.     

Penalized Nonparametric Scalar-on-Function Regression via Principal Coordinates

A number of classical approaches to nonparametric regression have recently been extended to the case of functional predictors. This article introduces a new method of this type, which extends intermediate-rank penalized smoothing to scalar-on-function regression. In the proposed method, which we call principal coordinate ridge regression, one regresses the response on leading principal coordinates defined by a relevant distance among the functional predictors, while applying a ridge penalty. Our publicly available implementation, based on generalized additive modeling software, allows for fast optimal tuning parameter selection and for extensions to multiple functional predictors, exponential family-valued responses, and mixed-effects models. In an application to signature verification data, principal coordinate ridge regression, with dynamic time warping distance used to define the principal coordinates, is shown to outperform a functional generalized linear model. Supplementary materials for this article are available online.     

Numerical Comparison of Some Penalty-Based Constraint Handling Techniques in Genetic Algorithms

We study five penalty function-based constraint handling techniques to be used with genetic algorithms in global optimization. Three of them, the method of superiority of feasible points, the method of parameter free penalties and the method of adaptive penalties have already been considered in the literature. In addition, we introduce two new modifications of these methods. We compare all the five methods numerically in 33 test problems and report and analyze the results obtained in terms of accuracy, efficiency and reliability. The method of adaptive penalties turned out to be most efficient while the method of parameter free penalties was the most reliable.     

Penalized Regressions: The Bridge versus the Lasso

Abstract

Bridge regression, a special family of penalized regressions of a penalty function Σ|β_j|^γ with γ ≤ 1, considered. A general approach to solve for the bridge estimator is developed. A new algorithm for the lasso (γ = 1) is obtained by studying the structure of the bridge estimators. The shrinkage parameter γ and the tuning parameter λ are selected via generalized cross-validation (GCV). Comparison between the bridge model (γ ≤ 1) and several other shrinkage models, namely the ordinary least squares regression (λ = 0), the lasso (γ = 1) and ridge regression (γ = 2), is made through a simulation study. It is shown that the bridge regression performs well compared to the lasso and ridge regression. These methods are demonstrated through an analysis of a prostate cancer data. Some computational advantages and limitations are discussed.     

A MODIFIED VARIABLE-PENALTY ALTERNATING DIRECTIONS METHOD FOR MONOTONE VARIATIONAL INEQUALITIES

Alternating directions method is one of the approaches for solving linearly constrained separate monotone variational inequalities. Experience on applications has shown that the number of iteration significantly depends on the penalty for the system of linearly constrained equations and therefore the method with variable penalties is advantageous in practice. In this paper, we extend the Kontogiorgis and Meyer method [12] by removing the monotonicity assumption on the variable penalty matrices. Moreover, we introduce a self-adaptive rule that leads the method to be more efficient and insensitive for various initial penalties. Numerical results for a class of Fermat-Weber problems show that the modified method and its self-adaptive technique are proper and necessary in practice.     

Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models

In this article, for Lasso penalized linear regression models in high-dimensional settings, we propose a modified cross-validation (CV) method for selecting the penalty parameter. The methodology is extended to other penalties, such as Elastic Net. We conduct extensive simulation studies and real data analysis to compare the performance of the modified CV method with other methods. It is shown that the popular K-fold CV method includes many noise variables in the selected model, while the modified CV works well in a wide range of coefficient and correlation settings. Supplementary materials containing the computer code are available online.     

Martingale methods for pricing inventory penalties under continuous replenishment and compound renewal demands

This paper addresses the problem of inventory penalty pricing under the risk-neutral valuation principle. The underlying production-inventory system has a constant replenishment rate and a compound renewal demand stream (i.e., iid demand interarrival times are independent of iid demand sizes), and is subject to underage and overage penalties. Our pricing approach treats the penalties as a series of perpetual American options, and constructs auxiliary martingale processes in term of the inventory process. We provide a necessary and sufficient martingale condition for general compound renewal demands. Explicit expressions of penalty functions for underage and overage are obtained for the case where demand arrivals follow a Poisson process.