Model selection for regression on a fixed design

We deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points. For this end, we consider some collection of finite dimensional linear spaces (models) and the least-squares estimator built on a data driven selected model among this collection. This data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on Mallows' C _p . We provide non asymptotic risk bounds for the so-defined estimator from which we deduce adaptivity properties. Our results hold under mild moment conditions on the errors. The statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach. Received: 2 July 1997 / Revised version: 20 September 1999 / Published online: 6 July 2000     

Stahel-Donoho kernel estimation for fixed design nonparametric regression models

This paper reports a robust kernel estimation for fixed design nonparametric regression models. A Stahel-Donoho kernel estimation is introduced, in which the weight functions depend on both the depths of data and the distances between the design points and the estimation points. Based on a local approximation, a computational technique is given to approximate to the incomputable depths of the errors. As a result the new estimator is computationally efficient. The proposed estimator attains a high breakdown point and has perfect asymptotic behaviors such as the asymptotic normality and convergence in the mean squared error. Unlike the depth-weighted estimator for parametric regression models, this depth-weighted nonparametric estimator has a simple variance structure and then we can compare its efficiency with the original one. Some simulations show that the new method can smooth the regression estimation and achieve some desirable balances between robustness and efficiency.     

On asymptotically efficient estimation for a semiparametric regression model

1.IntroductionConsiderthemodelY=X"0 g(T) E,(1'1)whereX"~(xl,',xo)areexplanatoryvariablesthatenterlinearly,Pisakx1vectorofunknownparameters,Tisanotherexplanatoryvariablesthatentersinanonlinearfashion,g')isanunknownsmoothfunctionofTinR',(X,T)andeareindependent,andeistheerrorwithmean0andvariancea2.Trangesoveranondegeneratecompact1-dimensionalilltervalC*;withoutlossofgenerality,C*=[0,1].Chenl2]discussedasymptoticnormalityofestimatorsP.of0byusingpiecewisepolynthacaltoapproximateg.Speckmanls…     

A new algorithm for total variation based image denoising

We propose a new algorithm for the total variation based on image denoising problem. The split Bregman method is used to convert an unconstrained minimization denoising problem to a linear system in the outer iteration. An algebraic multi-grid method is applied to solve the linear system in the inner iteration. Furthermore, Krylov subspace acceleration is adopted to improve convergence in the outer iteration. Numerical experiments demonstrate that this algorithm is efficient even for images with large signal-to-noise ratio.     

NA样本固定设计下半参数回归模型中估计的强相合性

对于半参数回归模型yi=xiβ g(ti) ei，i=1，2,…，n(xi，ti)为已知的固定设计点列．本在误差{e．1≤n≤n)为NA序列时，对g(t)和σ的估计量gn~(t)和σn^2的逐点强相合。以及gn~(t)的一致强相合作了研究，得到比较理想的结果。     

基于半参数回归模型的最小一乘局部线性算法

根据最小一乘准则,推导出最小一乘局部线性估计的计算方法,并通过对模拟数据的计算和分析,对比最小一乘核算法和最小二乘局部线性算法,验证了最小一乘局部线性算法是一种有效的,稳健的估计方法,并且有降低边界效应的作用．     

Plug-in method for nonparametric regression

The problem of bandwidth selection for non-parametric kernel regression is considered. We will follow the Nadaraya–Watson and local linear estimator especially. The circular design is assumed in this work to avoid the difficulties caused by boundary effects. Most of bandwidth selectors are based on the residual sum of squares (RSS). It is often observed in simulation studies that these selectors are biased toward undersmoothing. This leads to consideration of a procedure which stabilizes the RSS by modifying the periodogram of the observations. As a result of this procedure, we obtain an estimation of unknown parameters of average mean square error function (AMSE). This process is known as a plug-in method. Simulation studies suggest that the plug-in method could have preferable properties to the classical one. Supported by the MSMT: LC 06024.     

A class of fractional-order multi-scale variational models and alternating projection algorithm for image denoising

The total variation model proposed by Rudin, Osher and Fatemi performs very well for removing noise while preserving edges. However, it favors a piecewise constant solution in BV space which often leads to the staircase effect, and small details such as textures are often filtered out with noise in the process of denoising. To preserve the textures and eliminate the staircase effect, we improve the total variation model in this paper. This is accomplished by the following steps: (1) we define a new space of functions of fractional-order bounded variation called the BV_α space by using the Grünwald–Letnikov definition of fractional-order derivative; (2) we model the structure of the image as a function belonging to the BV_α space, and the textures in different scales as functions belonging to different negative Sobolev spaces. Thus, we propose a class of fractional-order multi-scale variational models for image denoising. (3) We analyze some properties of the fraction-order total variation operator and its conjugate operator. By using these properties, we develop an alternation projection algorithm for the new model and propose an efficient condition of the convergence of the algorithm. The numerical results show that the fractional-order multi-scale variational model can improve the peak signal to noise ratio of image, preserve textures and eliminate the staircase effect efficiently in the process of denoising.     

A finite algorithm for monotone regression and the application of its principle to the aggregation of ranks and indifference curve fitting

Monotone regression makes optimal consistent adjusted value assignments to ordinal dependent data, or monotonically adjusts ratio-level-dependent variable data to achieve the best possible agreement with an explanatory linear (in the sense of parameters) model. Thus, for example, as a tool for building group or individual multi-attribute value (MAV) functions, it partially obviates the need to prespecify a particular MAV function class. The height of the unknown MAV function at each comparison bundle is determined along with the other model parameters.In this paper, the Kruskal stress criterion and iterative computational methodology are shown to be reducible to one of three simpler convex quadratic programming problems. The fundamental idea underlying the method is not restricted to least-squares objectives. An application to the problem of aggregation of ranks is shown. Here, the minimax data-fitting criterion is employed.     

删失分位数回归模型基于扩展兴趣信息准则的平均估计

本文结合分位数回归技术,基于删失回归模型,把Claeskens和Hjort的传统兴趣信息准侧(focused information criterion,FIC)扩展到兴趣向量的情形,提出扩展的兴趣信息准则(extended focused information criterion,E-FIC),有效解决了同时针对多个兴趣参数的平均估计问题,并且对删失响应变量的不同水平分位数进行建模,以全面反映响应变量分布特征,有效克服异常值和厚尾模型误差的影响.基于扩展的兴趣信息准则给出参数的平均估计方法,证明估计的渐近性质.通过Monte Carlo随机模拟试验比较所提估计方法和最小二乘方法在有限样本量下的表现,用所提方法对原发性胆汁性肝硬化数据集进行数据分析.     

A new wavelet-based denoising algorithm for high-frequency financial data mining

Denoising analysis imposes new challenge for mining high-frequency financial data due to its irregularities and roughness. Inefficient decomposition of the systematic pattern (the trend) and noises of high-frequency data will lead to erroneous conclusion as the irregularities and roughness of the data make the application of traditional methods difficult. In this paper, we propose the local linear scaling approximation (in short, LLSA) algorithm, a new nonlinear filtering algorithm based on the linear maximal overlap discrete wavelet transform (MODWT) to decompose the systematic pattern and noises. We show several unique properties of this brand-new algorithm, that are, the local linearity, computational complexity, and consistency. We conduct a simulation study to confirm these properties we have analytically shown and compare the performance of LLSA with MODWT. We then apply our new algorithm with the real high-frequency data from German equity market to investigate its implementation in forecasting. We show the superior performance of LLSA and conclude that it can be applied with flexible settings and suitable for high-frequency data mining.     

部分线性单指标模型的复合分位数回归及变量选择

本文提出复合最小化平均分位数损失估计方法 (composite minimizing average check loss estimation,CMACLE)用于实现部分线性单指标模型(partial linear single-index models,PLSIM)的复合分位数回归(composite quantile regression,CQR).首先基于高维核函数构造参数部分的复合分位数回归意义下的相合估计,在此相合估计的基础上,通过采用指标核函数进一步得到参数和非参数函数的可达最优收敛速度的估计,并建立所得估计的渐近正态性,比较PLSIM的CQR估计和最小平均方差估计(MAVE)的相对渐近效率.进一步地,本文提出CQR框架下PLSIM的变量选择方法,证明所提变量选择方法的oracle性质.随机模拟和实例分析验证了所提方法在有限样本时的表现,证实了所提方法的优良性.     

A multivariate adaptive regression splines cutting plane approach for solving a two-stage stochastic programming fleet assignment model

The fleet assignment model assigns a fleet of aircraft types to the scheduled flight legs in an airline timetable published six to twelve weeks prior to the departure of the aircraft. The objective is to maximize profit. While costs associated with assigning a particular fleet type to a leg are easy to estimate, the revenues are based upon demand, which is realized close to departure. The uncertainty in demand makes it challenging to assign the right type of aircraft to each flight leg based on forecasts taken six to twelve weeks prior to departure. Therefore, in this paper, a two-stage stochastic programming framework has been developed to model the uncertainty in demand, along with the Boeing concept of demand driven dispatch to reallocate aircraft closer to the departure of the aircraft. Traditionally, two-stage stochastic programming problems are solved using the L-shaped method. Due to the slow convergence of the L-shaped method, a novel multivariate adaptive regression splines cutting plane method has been developed. The results obtained from our approach are compared to that of the L-shaped method, and the value of demand-driven dispatch is estimated.     

A recursive algorithm for the infinity-norm fixed point problem

We present the PFix algorithm for the fixed point problem f(x)=x on a nonempty domain [a,b], where d1, , and f is a Lipschitz continuous function with respect to the infinity norm, with constant q1. The computed approximation satisfies the residual criterion , where >0. In general, the algorithm requires no more than ∑_i=1^dsⁱ function component evaluations, where s≡max(1,log₂(||b−a||_∞/))+1. This upper bound has order as →0. For the domain [0,1]^d with <0.5 we prove a stronger result, i.e., an upper bound on the number of function component evaluations is , where r≡log₂(1/). This bound approaches as r→∞ (→0) and as d→∞. We show that when q<1 the algorithm can also compute an approximation satisfying the absolute criterion , where x^* is the unique fixed point of f. The complexity in this case resembles the complexity of the residual criterion problem, but with tolerance (1−q) instead of . We show that when q>1 the absolute criterion problem has infinite worst-case complexity when information consists of function evaluations. Finally, we report several numerical tests in which the actual number of evaluations is usually much smaller than the upper complexity bound.     

On tests for selection of variables and independence under multivariate regression models

The authors consider various procedures for testing the hypotheses of independence of two sets of variables and certain regression coefficients are zero under multivariate regression model. Various properties of these procedures and the asymptotic distributions associated with these procedures are also considered.     

Tests for independence of two multivariate regression equations with different design matrices

In this paper, the authors considered various procedures for testing for the independence of two multivariate regression equations with different design matrices. Asymptotic null distributions as well as nonnull distributions under local alternatives of the test statistics associated with the above procedures are also derived.     

A two-phase ant colony algorithm for multi-echelon defective supply chain network design

Supply chain system is an integrated production system of a product. In the past researches, this system was often assumed to be an equilibrium structure, but in real production process, some members in this system usually cannot effectively complete their production task because of the losses of production, which will reduce the performance of the whole supply chain production system. This supply chain with the losses of production is called the defective supply chain (DSC) system. This research will discuss the partner selection and the production–distribution planning in this DSC network system. Besides the cost of production and transportation, the reliability of the structure and the unbalance of this system caused by the losses of production are considered. Then a germane mathematical programming model is developed for solving this problem. Due to the complex problem and in order to get a satisfactory near-optimal solution with great speed, this research proposes seeking the solution with the solving model based on ant colony algorithm. The application results in real cases show that the solving model presented by this research can quickly and effectively plan the most suitable type of the DSC network and decision-making of the production–distribution. Finally, a comparative numerical experiment is performed by using the proposed approach and the common single-phase ant colony algorithm (SAC) to demonstrate the performance of the proposed approach. The analysis results show that the proposed approach can outperform the SAC in partner selection and production–distribution planning for DSC network design.     

A variable selection method based on Tabu search for logistic regression models

A Tabu search method is proposed and analysed for selecting variables that are subsequently used in Logistic Regression Models. The aim is to find from among a set of m variables a smaller subset which enables the efficient classification of cases. Reducing dimensionality has some very well-known advantages that are summarized in literature. The specific problem consists in finding, for a small integer value of p, a subset of size p of the original set of variables that yields the greatest percentage of hits in Logistic Regression. The proposed Tabu search method performs a deep search in the solution space that alternates between a basic phase (that uses simple moves) and a diversification phase (to explore regions not previously visited). Testing shows that it obtains significantly better results than the Stepwise, Backward or Forward methods used by classic statistical packages. Some results of applying these methods are presented.     

Robust model selection for a semimartingale continuous time regression from discrete data

The paper considers the problem of estimating a periodic function in a continuous time regression model observed under a general semimartingale noise with an unknown distribution in the case when continuous observation cannot be provided and only discrete time measurements are available. Two specific types of noises are studied in detail: a non-Gaussian Ornstein–Uhlenbeck process and a time-varying linear combination of a Brownian motion and compound Poisson process. We develop new analytical tools to treat the adaptive estimation problems from discrete data. A lower bound for the frequency sampling, needed for the efficiency of the procedure constructed by discrete observations, has been found. Sharp non-asymptotic oracle inequalities for the robust quadratic risk have been derived. New convergence rates for the efficient procedures have been obtained. An example of the regression with a martingale noise exhibits that the minimax robust convergence rate may be both higher or lower as compared with the minimax rate for the “white noise” model. The results of Monte-Carlo simulations are given.