Bayesian time series regression with nonparametric modeling of autocorrelation

Series models have several functions: comprehending the functional dependence of variable of interest on covariates, forecasting the dependent variable for future values of covariates and estimating variance disintegration, co-integration and steady-state relations. Although the regression function in a time series model has been extensively modeled both parametrically and nonparametrically, modeling of the error autocorrelation is mainly restricted to the parametric setup. A proper modeling of autocorrelation not only helps to reduce the bias in regression function estimate, but also enriches forecasting via a better forecast of the error term. In this article, we present a nonparametric modeling of autocorrelation function under a Bayesian framework. Moving into the frequency domain from the time domain, we introduce a Gaussian process prior to the log of the spectral density, which is then updated by using a Whittle approximation for the likelihood function (Whittle likelihood). The posterior computation is simplified due to the fact that Whittle likelihood is approximated by the likelihood of a normal mixture distribution with log-spectral density as a location shift parameter, where the mixture is of only five components with known means, variances, and mixture probabilities. The problem then becomes conjugate conditional on the mixture components, and a Gibbs sampler is used to initiate the unknown mixture components as latent variables. We present a simulation study for performance comparison, and apply our method to the two real data examples.     

Bayesian analysis of constant elasticity of variance models

This paper focuses on the estimation of some models in finance and in particular, in interest rates. We analyse discretized versions of the constant elasticity of variance (CEV) models where the normal law showing up in the usual discretization of the diffusion part is replaced by a range of heavy‐tailed distributions. A further extension of the model is to allow the elasticity of variance to be a parameter itself. This generalized model allows great flexibility in modelling and simplifies the model implementation considerably using the scale mixtures representation. The mixing parameters provide a means to identify possible outliers and protect inference by down‐weighting the distorting effects of these outliers. For parameter estimation, Bayesian approach is adopted and implemented using the software WinBUGS (Bayesian inference using Gibbs sampler). Results from a real data analysis show that an exponential power distribution with a random shape parameter, which is highly leptokurtic compared with the normal distribution, forms the best CEV model for the data. Copyright © 2006 John Wiley & Sons, Ltd.     

Reducing variance in nonparametric surface estimation

We suggest a method for reducing variance in nonparametric surface estimation. The technique is applicable to a wide range of inferential problems, including both density estimation and regression, and to a wide variety of estimator types. It is based on estimating the contours of a surface by minimising deviations of elementary surface estimates along a quadratic curve. Once a contour estimate has been obtained, the final surface estimate is computed by averaging conventional surface estimates along a portion of the contour. Theoretical and numerical properties of the technique are discussed.     

Bayesian nonparametric statistical inference for Poisson point processes

Summary A random measure is said to be selected by a weighted gamma prior probability if the values it assigns to disjoint sets are independent gamma random variables with positive multipliers. If the intensity measure of a nonhomogeneous Poisson point process is selected by a weighted gamma prior probability and if a sample is drawn from the Poisson point process having this intensity measure, then the posterior random intensity measure given the observations is also selected by a weighted gamma prior probability. If the measure space is Euclidean and if the true intensity measure is continuous and finite, the centered posterior process, rescaled by the square root of the sample size, will converge weakly in Skorohod topology to a Wiener process subject to a change of time scale.This research was supported in part by the National Science Foundation Grants MCS 77-10376 and MCS 75-14194     

Bayesian nonparametric regression with varying residual density

We consider the problem of robust Bayesian inference on the mean regression function allowing the residual density to change flexibly with predictors. The proposed class of models is based on a Gaussian process (GP) prior for the mean regression function and mixtures of Gaussians for the collection of residual densities indexed by predictors. Initially considering the homoscedastic case, we propose priors for the residual density based on probit stick-breaking mixtures. We provide sufficient conditions to ensure strong posterior consistency in estimating the regression function, generalizing existing theory focused on parametric residual distributions. The homoscedastic priors are generalized to allow residual densities to change nonparametrically with predictors through incorporating GP in the stick-breaking components. This leads to a robust Bayesian regression procedure that automatically down-weights outliers and influential observations in a locally adaptive manner. The methods are illustrated using simulated and real data applications.     

Bayesian nonparametric predictive inference and bootstrap techniques

We address the question as to whether a prior distribution on the space of distribution functions exists which generates the posterior produced by Efron's and Rubin's bootstrap techniques, emphasizing the connection with the Dirichlet process. We also introduce a new resampling plan which has two advantages: prior opinions are taken into account and the predictive distribution of the future observations is not forced to be concentrated on observed values.     

A Wald-type variance estimation for the nonparametric distribution estimators for doubly censored data

We discuss the variance estimation for the nonparametric distribution estimator for doubly censored data. We first provide another view of Kuhn–Tucker’s conditions to construct the profile likelihood, and lead a Newton–Raphson algorithm as an optimization technique unlike the EM algorithm. The main proposal is an iteration-free Wald-type variance estimate based on the chain rule of differentiating conditions to construct the profile likelihood, which generalizes the variance formula in only right- or left-censored data. In this estimation procedure, we overcome some difficulties caused in directly applying Turnbull’s formula to large samples and avoid a load with computationally heavy iterations, such as solving the Fredholm equations, computing the profile likelihood ratio or using the bootstrap. Also, we establish the consistency of the formulated Wald-type variance estimator. In addition, simulation studies are performed to investigate the properties of the Wald-type variance estimates in finite samples in comparison with those from the profile likelihood ratio.     

Objective Bayesian model selection approach to the two way analysis of variance

An objective Bayesian procedure for testing in the two way analysis of variance is proposed. In the classical methodology the main effects of the two factors and the interaction effect are formulated as linear contrasts between means of normal populations, and hypotheses of the existence of such effects are tested. In this paper, for the first time these hypotheses have been formulated as objective Bayesian model selection problems. Our development is under homoscedasticity and heteroscedasticity, providing exact solutions in both cases. Bayes factors are the key tool to choose between the models under comparison but for the usual default prior distributions they are not well defined. To avoid this difficulty Bayes factors for intrinsic priors are proposed and they are applied in this setting to test the existence of the main effects and the interaction effect. The method has been illustrated with an example and compared with the classical method. For this example, both approaches went in the same direction although the large P value for interaction (0.79) only prevents us against to reject the null, and the posterior probability of the null (0.95) was conclusive.     

Bayesian model selection approach to the one way analysis of variance under homoscedasticity

An objective Bayesian model selection procedure is proposed for the one way analysis of variance under homoscedasticity. Bayes factors for the usual default prior distributions are not well defined and thus Bayes factors for intrinsic priors are used instead. The intrinsic priors depend on a training sample which is typically a unique random vector. However, for the homoscedastic ANOVA it is not the case. Nevertheless, we are able to illustrate that the Bayes factors for the intrinsic priors are not sensitive to the minimal training sample chosen; furthermore, we propose an alternative pooled prior that yields similar Bayes factors. To compute these Bayes factors Bayesian computing methods are required when the sample sizes of the involved populations are large. Finally, a one to one relationship—which we call the calibration curve—between the posterior probability of the null hypothesis and the classical $p$ value is found, thus allowing comparisons between these two measures of evidence. The behavior of the calibration curve as a function of the sample size is studied and conclusions relating both procedures are stated.     

Two-sample Kolmogorov-Smirnov test using a Bayesian nonparametric approach

In this paper, a Bayesian nonparametric approach to the two-sample problem is proposed. Given two samples \(\text{X} = {X_1}, \ldots ,{X_{m1}}\;\mathop {\text~}\limits^{i.i.d.} F\) and \(Y = {Y_1}, \ldots ,{Y_{{m_2}}}\mathop {\text~}\limits^{i.i.d.} G\), with F and G being unknown continuous cumulative distribution functions, we wish to test the null hypothesis H ₀: F = G. The method is based on computing the Kolmogorov distance between two posterior Dirichlet processes and comparing the results with a reference distance. The parameters of the Dirichlet processes are selected so that any discrepancy between the posterior distance and the reference distance is related to the difference between the two samples. Relevant theoretical properties of the procedure are also developed. Through simulated examples, the approach is compared to the frequentist Kolmogorov–Smirnov test and a Bayesian nonparametric test in which it demonstrates excellent performance.     

M-estimation in nonparametric regression under strong dependence and infinite variance

A robust local linear regression smoothing estimator for a nonparametric regression model with heavy-tailed dependent errors is considered in this paper. Under certain regularity conditions, the weak consistency and asymptotic distribution of the proposed estimators are obtained. If the errors are short-range dependent, then the limiting distribution of the estimator is normal. If the data are long-range dependent, then the limiting distribution of the estimator is a stable distribution.     

A simple test for the parametric form of the variance function in nonparametric regression

In this paper a new test for the parametric form of the variance function in the common nonparametric regression model is proposed which is applicable under very weak smoothness assumptions. The new test is based on an empirical process formed from pseudo residuals, for which weak convergence to a Gaussian process can be established. In the special case of testing for homoscedasticity the limiting process is essentially a Brownian bridge, such that critical values are easily available. The new procedure has three main advantages. First, in contrast to many other methods proposed in the literature, it does not depend directly on a smoothing parameter. Secondly, it can detect local alternatives converging to the null hypothesis at a rate n ^−1/2. Thirdly, in contrast to most of the currently available tests, it does not require strong smoothness assumptions regarding the regression and variance function. We also present a simulation study and compare the tests with the procedures that are currently available for this problem and require the same minimal assumptions.     

Efficient nonparametric testing by functional estimation

Many nonparametric tests admit improvement by identifying a functional on a set of probability measures , of which the test statistic is an estimator. We call such a functional a gauge for the problem if it induces the partition of into null and alternative and enjoys certain invariance properties. Two nonparametric testing problems are explored here: a dependency problem and an equidistribution problem. In each a dual smoothing problem is posed and optimally solved in the estimation framework, and a corresponding testing procedure gives a consistency rate improvement over the original test.     

On the estimation of a monotone conditional variance in nonparametric regression

A monotone estimate of the conditional variance function in a heteroscedastic, nonparametric regression model is proposed. The method is based on the application of a kernel density estimate to an unconstrained estimate of the variance function and yields an estimate of the inverse variance function. The final monotone estimate of the variance function is obtained by an inversion of this function. The method is applicable to a broad class of nonparametric estimates of the conditional variance and particularly attractive to users of conventional kernel methods, because it does not require constrained optimization techniques. The approach is also illustrated by means of a simulation study.     

Computation of reference Bayesian inference for variance components in longitudinal studies

Generalized linear mixed models (GLMMs) have been applied widely in the analysis of longitudinal data. This model confers two important advantages, namely, the flexibility to include random effects and the ability to make inference about complex covariances. In practice, however, the inference of variance components can be a difficult task due to the complexity of the model itself and the dimensionality of the covariance matrix of random effects. Here we first discuss for GLMMs the relation between Bayesian posterior estimates and penalized quasi-likelihood (PQL) estimates, based on the generalization of Harville’s result for general linear models. Next, we perform fully Bayesian analyses for the random covariance matrix using three different reference priors, two with Jeffreys’ priors derived from approximate likelihoods and one with the approximate uniform shrinkage prior. Computations are carried out via the combination of asymptotic approximations and Markov chain Monte Carlo methods. Under the criterion of the squared Euclidean norm, we compare the performances of Bayesian estimates of variance components with that of PQL estimates when the responses are non-normal, and with that of the restricted maximum likelihood (REML) estimates when data are assumed normal. Three applications and simulations of binary, normal, and count responses with multiple random effects and of small sample sizes are illustrated. The analyses examine the differences in estimation performance when the covariance structure is complex, and demonstrate the equivalence between PQL and the posterior modes when the former can be derived. The results also show that the Bayesian approach, particularly under the approximate Jeffreys’ priors, outperforms other procedures.     

Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs

We study a Bayesian approach to nonparametric estimation of the periodic drift function of a one-dimensional diffusion from continuous-time data. Rewriting the likelihood in terms of local time of the process, and specifying a Gaussian prior with precision operator of differential form, we show that the posterior is also Gaussian with the precision operator also of differential form. The resulting expressions are explicit and lead to algorithms which are readily implementable. Using new functional limit theorems for the local time of diffusions on the circle, we bound the rate at which the posterior contracts around the true drift function.     

Heteroscedasticity check in nonlinear semiparametric models based on nonparametric variance function

The assumption of homoscedasticity has received much attention in classical analysis of regression. Heteroscedasticity tests have been well studied in parametric and nonparametric regressions. The aim of this paper is to present a test of heteroscedasticity for nonlinear semiparametric regression models with nonparametric variance function. The validity of the proposed test is illustrated by two simulated examples and a real data example.     

Optimal variance estimation based on lagged second-order difference in nonparametric regression

Differenced estimators of variance bypass the estimation of regression function and thus are simple to calculate. However, there exist two problems: most differenced estimators do not achieve the asymptotic optimal rate for the mean square error; for finite samples the estimation bias is also important and not further considered. In this paper, we estimate the variance as the intercept in a linear regression with the lagged Gasser-type variance estimator as dependent variable. For the equidistant design, our estimator is not only \(n^{1/2}\)-consistent and asymptotically normal, but also achieves the optimal bound in terms of estimation variance with less asymptotic bias. Simulation studies show that our estimator has less mean square error than some existing differenced estimators, especially in the cases of immense oscillation of regression function and small-sized sample.