Robust Bayesian prediction and estimation under a squared log error loss function

Robust Bayesian analysis is concerned with the problem of making decisions about some future observation or an unknown parameter, when the prior distribution belongs to a class Γ instead of being specified exactly. In this paper, the problem of robust Bayesian prediction and estimation under a squared log error loss function is considered. We find the posterior regret Γ-minimax predictor and estimator in a general class of distributions. Furthermore, we construct the conditional Γ-minimax, most stable and least sensitive prediction and estimation in a gamma model. A prequential analysis is carried out by using a simulation study to compare these predictors.     

Second-order nonlinear least squares estimation

The ordinary least squares estimation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. We extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. It is shown that this “second-order” least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. Simulation studies show that the variance reduction of the new estimator can be as high as 50% for sample sizes lower than 100. As a by-product, the joint asymptotic covariance matrix of the ordinary least squares estimators for the regression parameter and for the random error variance is also derived, which is only available in the literature for very special cases, e.g. that random error has a normal distribution. The results apply to both linear and nonlinear regression models, where the random error distributions are not necessarily known.     

Bayesian tail‐risk forecasting using realized GARCH

A realized generalized autoregressive conditional heteroskedastic (GARCH) model is developed within a Bayesian framework for the purpose of forecasting value at risk and conditional value at risk. Student‐t and skewed‐t return distributions are combined with Gaussian and student‐t distributions in the measurement equation to forecast tail risk in eight international equity index markets over a 4‐year period. Three realized measures are considered within this framework. A Bayesian estimator is developed that compares favourably, in simulations, with maximum likelihood, both in estimation and forecasting. The realized GARCH models show a marked improvement compared with ordinary GARCH for both value‐at‐risk and conditional value‐at‐risk forecasting. This improvement is consistent across a variety of data and choice of distributions. Realized GARCH models incorporating a skewed student‐t distribution for returns are favoured overall, with the choice of measurement equation error distribution and realized measure being of lesser importance. Copyright © 2017 John Wiley & Sons, Ltd.     

Estimating and Visualizing Conditional Densities

Abstract

We consider the kernel estimator of conditional density and derive its asymptotic bias, variance, and mean-square error. Optimal bandwidths (with respect to integrated mean-square error) are found and it is shown that the convergence rate of the density estimator is order n ^–2/3. We also note that the conditional mean function obtained from the estimator is equivalent to a kernel smoother. Given the undesirable bias properties of kernel smoothers, we seek a modified conditional density estimator that has mean equivalent to some other nonparametric regression smoother with better bias properties. It is also shown that our modified estimator has smaller mean square error than the standard estimator in some commonly occurring situations. Finally, three graphical methods for visualizing conditional density estimators are discussed and applied to a data set consisting of maximum daily temperatures in Melbourne, Australia.     

The distribution of a generalized least squares estimator with covariance adjustment

The distribution theory is developed for a generalized least squares estimator of the growth curve model. A special case of the estimator is the maximum likelihood estimator which is weighted by the sample covariance matrix. The distribution of two conditional forms of the estimator are derived and from these its density is obtained. Two general pivots and their distributions are derived from the conditional forms and special cases of these are investigated. The results obtained are linked to carlier work.     

时空模型的局部众数回归

时空数据经常含有奇异点或来自重尾分布,此时基于最小二乘的估计方法效果欠佳,需要更稳健的估计方法.本文提出时空模型的基于局部众数(local modal, LM)的局部线性估计方法.理论和数据分析结果都显示,若数据含有奇异点或来自重尾分布,基于局部众数的局部线性方法比基于最小二乘的局部线性方法有效;若数据无奇异点且来自正态分布,两种方法效率渐近一致.本文采用众数期望最大化(modal expectation-maximization, MEM)算法,并在数据相依情形下得出估计量的渐近正态性.     

A robust and efficient adaptive reweighted estimator of multivariate location and scatter

This article proposes a reweighted estimator of multivariate location and scatter, with weights adaptively computed from the data. Its breakdown point and asymptotic behavior under elliptical distributions are established. This adaptive estimator is able to attain simultaneously the maximum possible breakdown point for affine equivariant estimators and full asymptotic efficiency at the multivariate normal distribution. For the special case of hard-rejection weights and the MCD as initial estimator, it is shown to be more efficient than its non-adaptive counterpart for a broad range of heavy-tailed elliptical distributions. A Monte Carlo study shows that the adaptive estimator is as robust as its non-adaptive relative for several types of bias-inducing contaminations, while it is remarkably more efficient under normality for sample sizes as small as 200.     

Asymptotic distribution of the weighted least squares estimator

This paper derives the asymptotic distribution of the weighted least squares estimator (WLSE) in a heteroscedastic linear regression model. A consistent estimator of the asymptotic covariance matrix of the WLSE is also obtained. The results are obtained under weak conditions on the design matrix and some moment conditions on the error distributions. It is shown that most of the error distributions encountered in practice satisfy these moment conditions. Some examples of the asymptotic covariance matrices are also given.     

Bayesian point estimation and prediction

In the Bayesian viewpoint, point estimation and prediction are treated from a decision-making standpoint. If a loss function can be determined which associates a loss with every possible error of estimation or prediction, then the optimal estimator or predictor is that value which minimizes expected loss. In most applications, the loss function is assumed to be linear or quadratic in the error of estimation or prediction, although there are many practical situations in which these simple functions are quite inappropriate. In this paper, we investigate the properties of Bayesian point estimates under other loss functions; both the general case and two special cases (power and exponential loss functions) are considered. For the special cases, we also investigate the sensitivity of Bayesian point estimation and prediction to misspecification in the loss function and discuss the practical implications of the results.     

Improving density forecast by modeling asymmetric features: An application to S&P500 returns

A density forecast is an estimate of the probability distribution of the possible future values of a random variable. From the current literature, an economic time series may have three types of asymmetry: asymmetry in unconditional distribution, asymmetry in conditional distribution, volatility asymmetry. In this paper, we propose three density forecasting methods under two-piece normal assumption to capture these asymmetric features. A GARCH model with two-piece normal distribution is developed to capture asymmetries in the conditional distributions. In this approach, we first estimate parameters of a GARCH model by assuming normal innovations, and then fit a two-piece normal distribution to the empirical residuals. Block bootstrap procedure, and moving average method with two-piece normal distribution are presented for volatility asymmetry and asymmetry in the conditional distributions. Application of the developed methods to the weekly S&P500 returns illustrates that forecast quality can be significantly improved by modeling these asymmetric features.     

Arc refractor methods for adaptive importance sampling on large Bayesian networks under evidential reasoning

Approximate Bayesian inference by importance sampling derives probabilistic statements from a Bayesian network, an essential part of evidential reasoning with the network and an important aspect of many Bayesian methods. A critical problem in importance sampling on Bayesian networks is the selection of a good importance function to sample a network’s prior and posterior probability distribution. The initially optimal importance functions eventually start deviating from the optimal function when sampling a network’s posterior distribution given evidence, even when adaptive methods are used that adjust an importance function to the evidence by learning. In this article we propose a new family of Refractor Importance Sampling (RIS) algorithms for adaptive importance sampling under evidential reasoning. RIS applies “arc refractors” to a Bayesian network by adding new arcs and refining the conditional probability tables. The goal of RIS is to optimize the importance function for the posterior distribution and reduce the error variance of sampling. Our experimental results show a significant improvement of RIS over state-of-the-art adaptive importance sampling algorithms.     

Asymptotic properties of conditional maximum likelihood estimator in a certain exponential model

The conditional maximum likelihood estimator is suggested as an alternative to the maximum likelihood estimator and is favorable for an estimator of a dispersion parameter in the normal distribution, the inverse-Gaussian distribution, and so on. However, it is not clear whether the conditional maximum likelihood estimator is asymptotically efficient in general. Consider the case where it is asymptotically efficient and its asymptotic covariance depends only on an objective parameter in an exponential model. This remand implies that the exponential model possesses a certain parallel foliation. In this situation, this paper investigates asymptotic properties of the conditional maximum likelihood estimator and compares the conditional maximum likelihood estimator with the maximum likelihood estimator. We see that the bias of the former is more robust than that of the latter and that two estimators are very close, especially in the sense of bias-corrected version. The mean Pythagorean relation is also discussed.     

Pyramid Quantile Regression

We describe a Bayesian model for simultaneous linear quantile regression at several specified quantile levels. More specifically, we propose to model the conditional distributions by using random probability measures, known as quantile pyramids, introduced by Hjort and Walker. Unlike many existing approaches, this framework allows us to specify meaningful priors on the conditional distributions, while retaining the flexibility afforded by the nonparametric error distribution formulation. Simulation studies demonstrate the flexibility of the proposed approach in estimating diverse scenarios, generally outperforming other competitive methods. We also provide conditions for posterior consistency. The method is particularly promising for modeling the extremal quantiles. Applications to extreme value analysis and in higher dimensions are also explored through data examples. Supplemental material for this article is available online.     

条件密度近邻－核估计的逼近速度

本文首先研究了条件密度函数近邻－核估计的误差分布的正态逼近精度,然后利用随机加权法构造了近邻－核估计的随机加权统计量,获得了随机加权逼近精度。     

工序能力Bayes推断

本文从Ｂａｙｅｓ观点研究工序能力，对无信息先验和共轭先验，给出了Ｃｐ的后验分布、条件期望估计和最大后验估计、Ｂａｙｅｓ置信下限和判断工序是否有能力的临界值，适于对相似工序作统计推断。     

Simulation Estimation of Quantiles From a Distribution With Known Mean

It is common in practice to estimate the quantiles of a complicated distribution by using the order statistics of a simulated sample. If the distribution of interest has known population mean, then it is often possible to improve the mean square error of the standard quantile estimator substantially through the simple device of mean-correction: subtract off the sample mean and add on the known population mean. Asymptotic results for the meancorrected quantile estimator are derived and compared to the standard sample quantile. Simulation results for a variety of distributions and processes illustrate the asymptotic theory. Application to Markov chain Monte Carlo and to simulation-based uncertainty analysis is described.     

0-1分布的贝叶斯检验在医疗检查中的应用

本文给出了样本相互独立,但不同分布的情况下后验概率函数的表达式及其与序贯后验概率函数之间的关系。在此基础上,给出了先验分布和条件分布为0-1分布情况下贝叶斯后验概率大小的比较方法,结合贝叶斯检验分析法安排医疗检查,使其在不降低诊断准确率的前提下,节省检查费用,提出了合理安排医疗检查的建议。     

Penalized empirical likelihood estimation of semiparametric models

We propose an empirical likelihood-based estimation method for conditional estimating equations containing unknown functions, which can be applied for various semiparametric models. The proposed method is based on the methods of conditional empirical likelihood and penalization. Thus, our estimator is called the penalized empirical likelihood (PEL) estimator. For the whole parameter including infinite-dimensional unknown functions, we derive the consistency and a convergence rate of the PEL estimator. Furthermore, for the finite-dimensional parametric component, we show the asymptotic normality and efficiency of the PEL estimator. We illustrate the theory by three examples. Simulation results show reasonable finite sample properties of our estimator.     

Game-theoretic probability combination with applications to resolving conflicts between statistical methods

In the typical analysis of a data set, a single method is selected for statistical reporting even when equally applicable methods yield very different results. Examples of equally applicable methods can correspond to those of different ancillary statistics in frequentist inference and of different prior distributions in Bayesian inference. More broadly, choices are made between parametric and nonparametric methods and between frequentist and Bayesian methods. Rather than choosing a single method, it can be safer, in a game-theoretic sense, to combine those that are equally appropriate in light of the available information. Since methods of combining subjectively assessed probability distributions are not objective enough for that purpose, this paper introduces a method of distribution combination that does not require any assignment of distribution weights. It does so by formalizing a hedging strategy in terms of a game between three players: nature, a statistician combining distributions, and a statistician refusing to combine distributions. The optimal move of the first statistician reduces to the solution of a simpler problem of selecting an estimating distribution that minimizes the Kullback–Leibler loss maximized over the plausible distributions to be combined. The resulting combined distribution is a linear combination of the most extreme of the distributions to be combined that are scientifically plausible. The optimal weights are close enough to each other that no extreme distribution dominates the others. The new methodology is illustrated by combining conflicting empirical Bayes methods in the context of gene expression data analysis.     

Partition Weighted Approach For Estimating the Marginal Posterior Density With Applications

The computation of marginal posterior density in Bayesian analysis is essential in that it can provide complete information about parameters of interest. Furthermore, the marginal posterior density can be used for computing Bayes factors, posterior model probabilities, and diagnostic measures. The conditional marginal density estimator (CMDE) is theoretically the best for marginal density estimation but requires the closed-form expression of the conditional posterior density, which is often not available in many applications. We develop the partition weighted marginal density estimator (PWMDE) to realize the CMDE. This unbiased estimator requires only a single Markov chain Monte Carlo output from the joint posterior distribution and the known unnormalized posterior density. The theoretical properties and various applications of the PWMDE are examined in detail. The PWMDE method is also extended to the estimation of conditional posterior densities. We carry out simulation studies to investigate the empirical performance of the PWMDE and further demonstrate the desirable features of the proposed method with two real data sets from a study of dissociative identity disorder patients and a prostate cancer study, respectively. Supplementary materials for this article are available online.