Robust model selection for a semimartingale continuous time regression from discrete data

The paper considers the problem of estimating a periodic function in a continuous time regression model observed under a general semimartingale noise with an unknown distribution in the case when continuous observation cannot be provided and only discrete time measurements are available. Two specific types of noises are studied in detail: a non-Gaussian Ornstein–Uhlenbeck process and a time-varying linear combination of a Brownian motion and compound Poisson process. We develop new analytical tools to treat the adaptive estimation problems from discrete data. A lower bound for the frequency sampling, needed for the efficiency of the procedure constructed by discrete observations, has been found. Sharp non-asymptotic oracle inequalities for the robust quadratic risk have been derived. New convergence rates for the efficient procedures have been obtained. An example of the regression with a martingale noise exhibits that the minimax robust convergence rate may be both higher or lower as compared with the minimax rate for the “white noise” model. The results of Monte-Carlo simulations are given.     

General model selection estimation of a periodic regression with a Gaussian noise

This paper considers the problem of estimating a periodic function in a continuous time regression model with an additive stationary Gaussian noise having unknown correlation function. A general model selection procedure on the basis of arbitrary projective estimates, which does not need the knowledge of the noise correlation function, is proposed. A non-asymptotic upper bound for L₂{\mathcal{L}_2} -risk (oracle inequality) has been derived under mild conditions on the noise. For the Ornstein–Uhlenbeck noise the risk upper bound is shown to be uniform in the nuisance parameter. In the case of Gaussian white noise the constructed procedure has some advantages as compared with the procedure based on the least squares estimates (LSE). The asymptotic minimaxity of the estimates has been proved. The proposed model selection scheme is extended also to the estimation problem based on the discrete data applicably to the situation when high frequency sampling can not be provided.     

Estimation for regression with infinite variance errors

This paper addresses the problem of modelling time series with nonstationarity from a finite number of observations. Problems encountered with the time varying parameters in regression type models led to the smoothing techniques. The smoothing methods basically rely on the finiteness of the error variance, and thus, when this requirement fails, particularly when the error distribution is heavy tailed, the existing smoothing methods due to [1], are no longer optimal. In this paper, we propose a penalized minimum dispersion method for time varying parameter estimation when a regression model generated by an infinite variance stable process with characteristic exponent α ε (1, 2). Recursive estimates are evaluated and it is shown that these estimates for a nonstationary process with normal errors is a special case.     

Asymptotic distributions of two “synthetic data” estimators for censored single-index models

The censored single-index model provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored and the link function is unknown. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure time models for survival analysis. This paper proposes two methods for estimation of single-index models with randomly censored samples. We first transform the censored data into synthetic data or pseudo-responses unbiasedly, then obtain estimates of the index coefficients by the rOPG or rMAVE procedures of Xia (2006) [1]. Finally, we estimate the unknown nonparametric link function using techniques for univariate censored nonparametric regression. The estimators for the index coefficients are shown to be root-n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodologies.     

Autoregressive frequency detection using Regularized Least Squares

Tracking of an unknown frequency embedded in noise is widely applied in a variety of applications. Unknown frequencies can be obtained by approximating generalized spectral density of a periodic process by an autoregressive (AR) model. The advantage is that an AR model has a simple structure and its parameters can be easily estimated iteratively, which is crucial for online (real-time) applications. Typically, the order of the AR approximation is chosen by information criteria. However, with an increase of a sample size, model order may change, which leads to re-estimation of all model parameters. We propose a new iterative procedure for frequency detection based on a regularization of an empirical information matrix. The suggested method enables to avoid the repeated model selection as well as parameter estimation steps and therefore optimize computational costs. The asymptotic properties of the proposed regularized AR (RAR) frequency estimates are derived and performance of RAR is evaluated by numerical examples.     

Randomly censored partially linear single-index models

This paper proposes a method for estimation of a class of partially linear single-index models with randomly censored samples. The method provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure-time models for survival analysis. The estimation procedure involves three stages: first, transform the censored data into synthetic data or pseudo-responses unbiasedly; second, obtain quasi-likelihood estimates of the regression coefficients in both linear and single-index components by an iteratively algorithm; finally, estimate the unknown nonparametric regression function using techniques for univariate censored nonparametric regression. The estimators for the regression coefficients are shown to be jointly root-n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as all the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodology.     

On estimation of surrogate models for multivariate computer experiments

Estimation of surrogate models for computer experiments leads to nonparametric regression estimation problems without noise in the dependent variable. In this paper, we propose an empirical maximal deviation minimization principle to construct estimates in this context and analyze the rate of convergence of corresponding quantile estimates. As an application, we consider estimation of computer experiments with moderately high dimension by neural networks and show that here we can circumvent the so-called curse of dimensionality by imposing rather general assumptions on the structure of the regression function. The estimates are illustrated by applying them to simulated data and to a simulation model in mechanical engineering.

     

Local influence in multilevel regression for growth curves

Influence analysis is important in modelling and identification of special patterns in the data. It is well established in ordinary regression. However, analogous diagnostics are generally not available for the multilevel regression model, in which estimation involves a complex iterative algorithm. This paper studies the local influence of small perturbations on the parameter estimates in the multilevel regression model with application to growth curves. The estimation is based on the iterative generalized least-squares (IGLS) method suggested by Goldstein (Biometrika 73 (1986) 43). The generalized influence function and generalized Cook statistic (Biometrika 84(1) (1997) 175) of IGLS of unknown parameters under some specific simultaneous perturbations are derived to study the joint influence of subject units on parameter estimators. The perturbation scheme is introduced through a variance–covariance matrix of error variables. A one-step approximation formula is suggested for simplifying the computations. The method is examined on growth-curve data.     

Classical and Bayesian inference in two parameter exponential distribution with randomly censored data

This paper deal with the classical and Bayesian estimation for two parameter exponential distribution having scale and location parameters with randomly censored data. The censoring time is also assumed to follow a two parameter exponential distribution with different scale but same location parameter. The main stress is on the location parameter in this paper. This parameter has not yet been studied with random censoring in literature. Fitting and using exponential distribution on the range \((0, \infty )\), specially when the minimum observation in the data set is significantly large, will give estimates far from accurate. First we obtain the maximum likelihood estimates of the unknown parameters with their variances and asymptotic confidence intervals. Some other classical methods of estimation such as method of moment, L-moments and least squares are also employed. Next, we discuss the Bayesian estimation of the unknown parameters using Gibbs sampling procedures under generalized entropy loss function with inverted gamma priors and Highest Posterior Density credible intervals. We also consider some reliability and experimental characteristics and their estimates. A Monte Carlo simulation study is performed to compare the proposed estimates. Two real data examples are given to illustrate the importance of the location parameter.     

Efficiency,effectiveness and productivity of marketing in services

In this paper we consider complex deterministic problems, where there are two models that can be used to predict the performance for a given design. One of the models can give a precise estimation, but is complex and time consuming. The other model is simple and fast, but can only give a very crude estimation. We have proposed a learning-based ordinal optimization approach to tackle this problem. In this approach, we first run a simple model for all the designs and a complex model for a few designs, and then, through regression analysis, we estimate the noise trend, and this noise trend together with the crude estimates from the simple model will be used to screen the designs. The proposed approach is applied to solve an integrally bladed rotor (IBR) manufacturing problem where the production sequence and the production parameters need to be determined in order to minimize the overall manufacturing cost while satisfying the manufacturing constraints. The results indicate that, by using a very crude and simple model, we are able to identify good designs with a high degree of confidence.     

Spatial Regression Models,Response Surfaces,and Process Optimization

Abstract

Spatial regression models are developed as a complementary alternative to second-order polynomial response surfaces in the context of process optimization. These models provide estimates of design variable effects and smooth, data-faithful approximations to the unknown response function over the design space. The predicted response surfaces are driven by the covariance structures of the models. Several structures, isotropic and anisotropic, are considered and connections with thin plate splines are reviewed. Estimation of covariance parameters is achieved via maximum likelihood and residual maximum likelihood. A feature of the spatial regression approach is the visually appealing graphical summaries that are produced. These allow rapid and intuitive identification of process windows on the design space for which the response achieves target performance. Relevant design issues are briefly discussed and spatial designs, such as the packing designs available in Gosset, are suggested as a suitable design complement. The spatial regression models also perform well with no global design, for example with data obtained from series of designs on the same space of design variables. The approach is illustrated with an example involving the optimization of components in a DNA amplification assay. A Monte Carlo comparison of the spatial models with both thin plate splines and second-order polynomial response surfaces for a scenario motivated by the example is also given. This shows superior performance of the spatial models to the second-order polynomials with respect to both prediction over the complete design space and for cross-validation prediction error in the region of the optimum. An anisotropic spatial regression model performs best for a high noise case and both this model and the thin plate spline for a low noise case. Spatial regression is recommended for construction of response surfaces in all process optimization applications.     

Sequential Estimation of the Parameters in a Trigonometric Regression Model with the Gaussian Coloured Noise

This paper considers the estimation problem for a trigonometric regression model with the noise specified by the Ornstein–Uhlenbeck process with unknown parameter. We propose a sequential procedure which ensures a prescribed mean square precision uniformly in the nuisance parameter. The asymptotic behaviour of the procedure duration mean has been studied. This revised version was published online in August 2006 with corrections to the Cover Date.     

对左截断数据估计平稳序列

设平稳信号过程$\{X_t\}$被白噪声序列$\{Y_t\}$干扰. 只有$X_t>Y_t$时可以观测到信号过程$X_t$, 否则只能观测到白噪声$Y_t$. 这种数据模型被称为左截断数据模型. 本文在左截断数据模型下估计平稳信号过程的$\{X_t\}$均值, 自协方差函数, 和自相关系数. 证明所给的估计量是强相合估计. 当$X_t$是自回归序列时, 本文给出自回归模型的强相合的参数估计.     

Estimating structural credit risk models when market prices are contaminated with noise

In this paper, sequential estimation on hidden asset value and model parameter estimation is implemented under the Black–Cox model. To capture short‐term autocorrelation in the stock market, we assume that market noise follows a mean reverting process. For estimation, Bayesian methods are applied in this paper: the particle filter algorithm for sequential estimation of asset value and the generalized Gibbs and multivariate adapted Metropolis methods for model parameters estimation. The first simulation study shows that sequential hidden asset value estimation using both option price and equity price is more efficient than estimation using equity price alone. The second simulation study shows that, by applying the generalized Gibbs sampling and multivariate adapted Metropolis methods, model parameters can be estimated successfully. In an empirical analysis, the stock market noise for firms with more liquid stock is estimated as having smaller volatility. Copyright © 2015 John Wiley & Sons, Ltd.     

Estimated stochastic programs with chance constraints

We will consider a non-parametric estimation procedure for chance-constrained stochastic programs where the random parameters appear on the right-hand side of linear constraints for the decision variable. The assumed independence of the components of the random right-hand side data results in stochastic programs with a separability structure in the constraints. We estimate the unknown probability distribution of the random right-hand side data via isotonic regression estimates of increasing hazard rates. Our choice of the estimates was motivated by the relationship between logarithmic concave measures and increasing hazard rate distributions. We establish large deviation results for optimal values and optimal solution sets of the estimated programs. Finally, we discuss the numerical treatment of the estimated chance-constrained programs and report on a test run.     

Parameter and state estimation algorithm for single-input single-output linear systems using the canonical state space models

This paper presents a new parameter and state estimation algorithm for single-input single-output systems based on canonical state space models from the given input–output data. Difficulties of identification for state space models lie in that there exist unknown noise terms in the formation vector and unknown state variables. By means of the hierarchical identification principle, those noise terms in the information vector are replaced with the estimated residuals and a new least squares algorithm is proposed for parameter estimation and the system states are computed by using the estimated parameters. Finally, an example is provided.     

Inference for an anisotropic diffusion model

The vector sum of a white noise in an unknown hyperspace and an Ornstein-Uhlenbeck process in an unknown line is observed through sharp linear test functions over a finite time span. The parameters associated with the white noise (including the hyperplane) are determinable with precision and index the measure-equivalence classes in the relevant sample space. An intraclass relative density provides a basis for Bayesian inference of the remaining parameters.     

Bayesian Methods for Wavelet Series in Single-Index Models

Single-index models have found applications in econometrics and biometrics, where multidimensional regression models are often encountered. This article proposes a nonparametric estimation approach that combines wavelet methods for nonequispaced designs with Bayesian models. We consider a wavelet series expansion of the unknown regression function and set prior distributions for the wavelet coefficients and the other model parameters. To ensure model identifiability, the direction parameter is represented via its polar coordinates. We employ ad hoc hierarchical mixture priors that perform shrinkage on wavelet coefficients and use Markov chain Monte Carlo methods for a posteriori inference. We investigate an independence-type Metropolis-Hastings algorithm to produce samples for the direction parameter. Our method leads to simultaneous estimates of the link function and of the index parameters. We present results on both simulated and real data, where we look at comparisons with other methods.     

Consistency and asymptotic normality of least squares estimates used in linear systems identification

Least squares estimation of the parameters of a single input-single output linear autonomous system is considered where both plant noise and observation noise are present. It is shown that under fairly general conditions the estimates converge almost surely to the true system parameters and that the estimates are asymptotically normal.     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.