Optimal prediction for linear regression with infinitely many parameters

The problem of optimal prediction in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that outperforms asymptotically the ordinary least squares predictor. Moreover, if the random errors are Gaussian, the method is asymptotically minimax over ellipsoids in ?₂. The method is based on a regularized least squares estimator with weights of the Pinsker filter. We also consider the case of dynamic linear regression, which is important in the context of transfer function modeling.     

Global dynamics of the Kirschner–Panetta model for the tumor immunotherapy

In this paper we examine the global dynamics of the Kirschner–Panetta model describing the tumor immunotherapy. We give upper and lower ultimate bounds for densities of cell populations involved in this model. We demonstrate for this dynamics that there is a positively invariant polytope in the positive orthant. We present sufficient conditions on model parameters and treatment parameters under which all trajectories in the positive orthant tend to the tumor-free equilibrium point. We compare our results with Kirschner–Tsygvintsev results and concern biological implications of our assertions.     

Asymptotic Property of <Emphasis Type="Italic">M</Emphasis> Estimator in Classical Linear Models Under Dependent Random Errors

In this paper, we first establish a useful result on strong convergence for weighted sums of widely orthant dependent (WOD, in short) random variables. Based on the strong convergence that we established and the Bernstein type inequality, we investigate the strong consistency of M estimators of the regression parameters in linear models based on WOD random errors under some more mild moment conditions. The results obtained in the paper improve and extend the corresponding ones for negatively orthant dependent random variables and negatively superadditive dependent random variables. Finally, the simulation study is provided to illustrate the feasibility of the theoretical result that we established.     

正交回归和一般最小二乘回归的几何误差分析

线性回归分析中,一般最小二乘回归的目标函数只考虑一个方向的扰动,采用基于几何距离的正交回归能克服固定单方向最优带来的拟合稳定性差的弊端。本文分析和比较了正交回归和一般最小二乘回归的误差,并定量地给出了两者的几何误差与原始数据的方差、相关系数之间的关系,指出正交回归的几何误差小于一般最小二乘回归,并且正交回归具有旋转不变性。最后,以平面直线拟合为例验证了这个结论。     

生态系统的参数确定问题

通过不同观测数据研究捕食者—被捕食者生态系统参数确定问题.研究了四种情形1.观察数据无误差,并已知一个参数值.这种情况下,参数可由其相轨线和最小二乘法精确确定.2.观察数据无误差,但所有参数未知.此时仅靠相轨线的研究,无论给出多少组精确数据,都无法精确确定这些参数.通过原非线性模型的数值计算和网格搜索法,至少需要4组数据,同样得到了精度较高的参数值.3.当观测数据有误差时,根据解的周期性,引入标准周期的概念,在一个标准周期里讨论参数的确定问题,并利用标准周期内的捕食者与被捕食者的数量均值与系统的平衡点的关系对参数进行修正,然后使用网格法进行搜索,进一步提高了参数的精度.4.当观测时间也有误差时,先选取相对最优的随机正态数对观测时刻进行修正,然后再利用3.的方法估计参数.     

An implicit least squares algorithm for nonlinear rational model parameter estimation

In this study a new insight into least squares regression is identified and immediately applied to estimating the parameters of nonlinear rational models. From the beginning the ordinary explicit expression for linear in the parameters model is expanded into an implicit expression. Then a generic algorithm in terms of least squares error is developed for the model parameter estimation. It has been proved that a nonlinear rational model can be expressed as an implicit linear in the parameters model, therefore, the developed algorithm can be comfortably revised for estimating the parameters of the rational models. The major advancement of the generic algorithm is its conciseness and efficiency in dealing with the parameter estimation problems associated with nonlinear in the parameters models. Further, the algorithm can be used to deal with those regression terms which are subject to noise. The algorithm is reduced to an ordinary least square algorithm in the case of linear or linear in the parameters models. Three simulated examples plus a realistic case study are used to test and illustrate the performance of the algorithm.     

A semiparametric method for estimating nonlinear autoregressive model with dependent errors

The first-order nonlinear autoregressive model is considered and a semiparametric method is proposed to estimate regression function. In the presented model, dependent errors are defined as first-order autoregressive AR(1). The conditional least squares method is used for parametric estimation and the nonparametric kernel approach is applied to estimate regression adjustment. In this case, some asymptotic behaviors and simulated results for the semiparametric method are presented. Furthermore, the method is applied for the financial data in Iran’s Tejarat-Bank.     

A comparative assessment of different fuzzy regression methods for volatility forecasting

The aim of this paper is to compare different fuzzy regression methods in the assessment of the information content on future realised volatility of option-based volatility forecasts. These methods offer a suitable tool to handle both imprecision of measurements and fuzziness of the relationship among variables. Therefore, they are particularly useful for volatility forecasting, since the variable of interest (realised volatility) is unobservable and a proxy for it is used. Moreover, measurement errors in both realised volatility and volatility forecasts may affect the regression results. We compare both the possibilistic regression method of Tanaka et al. (IEEE Trans Syst Man Cybern 12:903–907, 1982) and the least squares fuzzy regression method of Savic and Pedrycz (Fuzzy Sets Syst 39:51–63, 1991). In our case study, based on intra-daily data of the DAX-index options market, both methods have proved to have advantages and disadvantages. Overall, among the two methods, we prefer the Savic and Pedrycz (Fuzzy Sets Syst 39:51–63, 1991) method, since it contains as special case (the central line) the ordinary least squares regression, is robust to the analysis of the variables in logarithmic terms or in levels, and provides sharper results than the Tanaka et al. (IEEE Trans Syst Man Cybern 12:903–907, 1982) method.     

Estimation non paramétrique de la régression avec variable explicative dans un espace métrique

We study a nonparametric regression estimator when the explanatory variable takes its values in a semi-metric space. We establish some asymptotic results and give upper bounds of the p-mean and the almost sure estimation errors under general conditions. We end by an application to the discrimination in a semi-metric space and illustrate the results by the example of Wiener process as an explanatory variable. To cite this article: S. Dabo-Niang, N. Rhomari, C. R. Acad. Sci. Paris, Ser. I 336 (2003).     

Dimension Asymptotics for Generalised Bootstrap in Linear Regression

We prove consistency of a class of generalised bootstrap techniques for the distribution of the least squares parameter estimator in linear regression, when the number of parameters tend to infinity with data size and the regressors are random. We show that best results are obtainable with resampling techniques that have not been considered earlier in the literature.     

High-dimensional variable screening and bias in subsequent inference,with an empirical comparison

We review variable selection and variable screening in high-dimensional linear models. Thereby, a major focus is an empirical comparison of various estimation methods with respect to true and false positive selection rates based on 128 different sparse scenarios from semi-real data (real data covariables but synthetic regression coefficients and noise). Furthermore, we present some theoretical bounds for the bias in subsequent least squares estimation, using the selected variables from the first stage, which have direct implications for construction of p-values for regression coefficients.     

稳健矩阵回归模型和方法研究

随着大数据时代的到来,我们面临的数据越来越复杂,其中待估系数为矩阵的模型亟待构造和求解.无论在统计还是优化领域,许多专家学者都致力于矩阵模型的统计性质分析及寻找其最优解的算法设计.当随机误差期望为0且同方差时,采用基于最小二乘的模型可以很好地解决问题.当随机误差异方差,分布为重尾分布(如双指数分布,t-分布等)或数据含有异常值时,需要考虑稳健的方法来求解问题.常用的稳健方法有最小一乘,分位数,Huber等.目前稳健方法的研究大多集中于线性回归问题,对于矩阵回归问题的研究比较缺乏.本文从最小二乘模型讲起,对矩阵回归问题进行了总结和评述,同时列出了一些文献和简要介绍了我们的近期的部分工作.最后对于稳健矩阵回归,我们提出了一些展望和设想.     

Derivative-free optimization and neural networks for robust regression

Large outliers break down linear and nonlinear regression models. Robust regression methods allow one to filter out the outliers when building a model. By replacing the traditional least squares criterion with the least trimmed squares (LTS) criterion, in which half of data is treated as potential outliers, one can fit accurate regression models to strongly contaminated data. High-breakdown methods have become very well established in linear regression, but have started being applied for non-linear regression only recently. In this work, we examine the problem of fitting artificial neural networks (ANNs) to contaminated data using LTS criterion. We introduce a penalized LTS criterion which prevents unnecessary removal of valid data. Training of ANNs leads to a challenging non-smooth global optimization problem. We compare the efficiency of several derivative-free optimization methods in solving it, and show that our approach identifies the outliers correctly when ANNs are used for nonlinear regression.     

A note on optimality conditions for the Euclidean. Multifacility location problem

Thekey problem of the Euclidean multifacility location (EMFL) problem is to decide whether a givendead point is optimal. If it is not optimal, we wish to compute a descent direction. This paper extends the optimality conditions of Calamai and Conn and Overton to the case when the rows of the active constraints matrix are linearly dependent. We show that linear dependence occurs wheneverG, the graph of the coinciding facilities, has a cycle. In this case the key problem is formulated as a linear least squares problem with bounds on the Euclidean norms of certain subvectors.     

Second-order nonlinear least squares estimation

The ordinary least squares estimation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. We extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. It is shown that this “second-order” least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. Simulation studies show that the variance reduction of the new estimator can be as high as 50% for sample sizes lower than 100. As a by-product, the joint asymptotic covariance matrix of the ordinary least squares estimators for the regression parameter and for the random error variance is also derived, which is only available in the literature for very special cases, e.g. that random error has a normal distribution. The results apply to both linear and nonlinear regression models, where the random error distributions are not necessarily known.     

Local Walsh-average-based estimation and variable selection for single-index models

We propose a robust estimation procedure based on local Walsh-average regression(LWR) for single-index models. Our novel method provides a root-n consistent estimate of the single-index parameter under some mild regularity conditions; the estimate of the unknown link function converges at the usual rate for the nonparametric estimation of a univariate covariate. We theoretically demonstrate that the new estimators show significant efficiency gain across a wide spectrum of non-normal error distributions and have almost no loss of efficiency for the normal error. Even in the worst case, the asymptotic relative efficiency(ARE) has a lower bound compared with the least squares(LS) estimates; the lower bounds of the AREs are 0.864 and 0.8896 for the single-index parameter and nonparametric function, respectively. Moreover, the ARE of the proposed LWR-based approach versus the ARE of the LS-based method has an expression that is closely related to the ARE of the signed-rank Wilcoxon test as compared with the t-test. In addition, to obtain a sparse estimate of the single-index parameter, we develop a variable selection procedure by combining the estimation method with smoothly clipped absolute deviation penalty; this procedure is shown to possess the oracle property. We also propose a Bayes information criterion(BIC)-type criterion for selecting the tuning parameter and further prove its ability to consistently identify the true model. We conduct some Monte Carlo simulations and a real data analysis to illustrate the finite sample performance of the proposed methods.     

A Minmax Regret Linear Regression Model Under Uncertainty in the Dependent Variable

This paper analyzes the simple linear regression model corresponding to a sample affected by errors from a non-probabilistic viewpoint. We consider the simplest case where the errors just affect the dependent variable and there only exists one explanatory variable. Moreover, we assume the errors affecting each observation can be bounded. In this context the minmax regret criterion is used in order to obtain a regression line with nearly optimal goodness of fit for any true values of the dependent variable. Theoretical results as well as numerical methods are stated in order to solve the optimization problem under different residual cost functions.     

Detection of outliers in weighted least squares regression

In multiple linear regression model, we have presupposed assumptions (independence, normality, variance homogeneity and so on) on error term. When case weights are given because of variance heterogeneity, we can estimate efficiently regression parameter using weighted least squares estimator. Unfortunately, this estimator is sensitive to outliers like ordinary least squares estimator. Thus, in this paper, we proposed some statistics for detection of outliers in weighted least squares regression.     

Asymptotics of estimators for nonparametric multivariate regression models with long memory

In this paper, a nonparametric multivariate regression model with long memory covariates and long memory errors is considered. We approximate the nonparametric multivariate regression function by the weighted additive one-dimensional functions. The local linear smoothing and least squares method are proposed for the one-dimensional regression estimation and the weight parameters estimation, respectively. The asymptotic behaviors of the proposed estimators are investigated.