Monte Carlo Integration With Acceptance-Rejection

This article considers Monte Carlo integration under rejection sampling or Metropolis-Hastings sampling. Each algorithm involves accepting or rejecting observations from proposal distributions other than a target distribution. While taking a likelihood approach, we basically treat the sampling scheme as a random design, and define a stratified estimator of the baseline measure. We establish that the likelihood estimator has no greater asymptotic variance than the crude Monte Carlo estimator under rejection sampling or independence Metropolis-Hastings sampling. We employ a subsampling technique to reduce the computational cost, and illustrate with three examples the computational effectiveness of the likelihood method under general Metropolis-Hastings sampling.     

Markov Chain Monte Carlo With Mixtures of Mutually Singular Distributions

Markov chain Monte Carlo (MCMC) methods for Bayesian computation are mostly used when the dominating measure is the Lebesgue measure, the counting measure, or a product of these. Many Bayesian problems give rise to distributions that are not dominated by the Lebesgue measure or the counting measure alone. In this article we introduce a simple framework for using MCMC algorithms in Bayesian computation with mixtures of mutually singular distributions. The idea is to find a common dominating measure that allows the use of traditional Metropolis-Hastings algorithms. In particular, using our formulation, the Gibbs sampler can be used whenever the full conditionals are available. We compare our formulation with the reversible jump approach and show that the two are closely related. We give results for three examples, involving testing a normal mean, variable selection in regression, and hypothesis testing for differential gene expression under multiple conditions. This allows us to compare the three methods considered: Metropolis-Hastings with mutually singular distributions, Gibbs sampler with mutually singular distributions, and reversible jump. In our examples, we found the Gibbs sampler to be more precise and to need considerably less computer time than the other methods. In addition, the full conditionals used in the Gibbs sampler can be used to further improve the estimates of the model posterior probabilities via Rao-Blackwellization, at no extra cost.     

Gradient Estimation Using Lagrange Interpolation Polynomials

We use Lagrange interpolation polynomials to obtain good gradient estimations. This is e.g. important for nonlinear programming solvers. As an error criterion, we take the mean squared error, which can be split up into a deterministic error and a stochastic error. We analyze these errors using N-times replicated Lagrange interpolation polynomials. We show that the mean squared error is of order if we replicate the Lagrange estimation procedure N times and use 2d evaluations in each replicate. As a result, the order of the mean squared error converges to N ⁻¹ if the number of evaluation points increases to infinity. Moreover, we show that our approach is also useful for deterministic functions in which numerical errors are involved. We provide also an optimal division between the number of gridpoints and replicates in case the number of evaluations is fixed. Further, it is shown that the estimation of the derivatives is more robust when the number of evaluation points is increased. Finally, test results show the practical use of the proposed method. We thank Jack Kleijnen, Gül Gürkan, and Peter Glynn for useful remarks on an earlier version of this paper. We thank Henk Norde for the proof of Lemma 2.2.     

An overview on the shrinkage properties of partial least squares regression

The aim of this paper is twofold. In the first part, we recapitulate the main results regarding the shrinkage properties of partial least squares (PLS) regression. In particular, we give an alternative proof of the shape of the PLS shrinkage factors. It is well known that some of the factors are >1. We discuss in detail the effect of shrinkage factors for the mean squared error of linear estimators and argue that we cannot extend the results to PLS directly, as it is nonlinear. In the second part, we investigate the effect of shrinkage factors empirically. In particular, we point out that experiments on simulated and real world data show that bounding the absolute value of the PLS shrinkage factors by 1 seems to leads to a lower mean squared error.     

加权再生核空间中信号的平均采样与重构

主要讨论L_v~p的加权再生核子空间中信号的平均采样与重构.首先,针对两种平均采样泛函建立了采样稳定性;其次,基于概率测度给出一个一般的迭代算法,将迭代逼近投影算法和迭代标架算法统一起来;最后,针对被白噪声污染的平均样本给出了信号重构的渐进点态误差估计.     

Mean Value Estimation Using Two-Phase Samples with Missing Data in Both Phases

The phenomenon of nonresponse in a sample survey reduces the precision of parameter estimates and causes the bias. Several methods have been developed to compensate for these effects. An important technique is the double sampling scheme introduced by Hansen and Hurwitz (J. Am. Stat. Assoc. 41, 517–529, 1946) which relies on subsampling of nonrespondents and repeating efforts to collect data from subsampled units. Several generalizations of this procedure have been proposed, including the application of arbitrary sampling designs considered by Särndal et al. (Model Assisted Survey Sampling, 1992). Under the assumption of complete response in the second phase, the population mean estimator constructed using data from both phases is unbiased. In this paper the properties of the mean value estimator under two-phase sampling are investigated for the case of the above assumption not being met. Expressions for bias and variance are obtained for general two-phase sampling procedure involving arbitrary sampling designs in both phases. Stochastic nonresponse governed by separate response distributions in both phases is assumed. Some special cases are discussed.     

Marginal density estimation for linear processes with cyclical long memory

Some convergence results on the kernel density estimator are proven for a class of linear processes with cyclic effects. In particular, we extend the results of Ho and Hsing (1996), Mielniczuk (1997) and Hall and Hart (1990) to the stationary processes for which the singularities of the spectral density are not limited to the origin. We show that the convergence rates and the limiting distribution may be different in this context.     

屏蔽数据下指数分布两部件串联系统多变点模型的贝叶斯估计

通过引入潜在变量得到了截尾情形屏蔽数据下指数分布两部件串联系统交点模型较简单的似然函数.利用Gibbs抽样与Metropolis-Hastings算法相结合的MCMC方法对各参数进行了抽样.基于Gibbs样本对参数进行估计.随机模拟的结果表明估计的精度较高.     

Structured Markov Chain Monte Carlo

Abstract

This article introduces a general method for Bayesian computing in richly parameterized models, structured Markov chain Monte Carlo (SMCMC), that is based on a blocked hybrid of the Gibbs sampling and Metropolis—Hastings algorithms. SMCMC speeds algorithm convergence by using the structure that is present in the problem to suggest an appropriate Metropolis—Hastings candidate distribution. Although the approach is easiest to describe for hierarchical normal linear models, we show that its extension to both nonnormal and nonlinear cases is straightforward. After describing the method in detail we compare its performance (in terms of run time and autocorrelation in the samples) to other existing methods, including the single-site updating Gibbs sampler available in the popular BUGS software package. Our results suggest significant improvements in convergence for many problems using SMCMC, as well as broad applicability of the method, including previously intractable hierarchical nonlinear model settings.     

Maximum integrated likelihood estimator of the interest parameter when the nuisance parameter is location or scale

The problem of estimation of an interest parameter in the presence of a nuisance parameter, which is either location or scale, is studied. Two estimators are considered: the usual maximum likelihood estimator and the estimator based on maximization of the integrated likelihood function. The estimators are compared, asymptotically, with respect to the bias and with respect to the mean squared error. The examples are given.     

On the estimation of optimal batch sizes in the analysis of simulation output

The estimation of the variance of point estimators is a classical problem of stochastic simulation. A more specific problem addresses the estimation of the variance of a sample mean from a steady-state autocorrelated process. Many proposed estimators of the variance of the sample mean are parameterized by batch size. A critical problem is to find an appropriate batch size that provides a good tradeoff between bias and variance. This paper proposes a procedure for determining the optimal batch size to minimize the mean squared error of estimators of the variance of the sample mean. This paper also presents the results of empirical studies of the procedure. The experiments involve symmetric two-state Markov chain models, first-order autoregressive processes, seasonal autoregressive processes, and queue-waiting times for several M/M/1 queueing models. The empirical results indicate that the estimation procedure works nearly as well as it would if the parameters of the processes were known.     

删失截断情形下失效率变点模型的Bayes参数估计

通过添加部分缺失寿命变量数据,得到了删失截断情形下失效率变点模型相对简单的似然函数.讨论了所添加缺失数据变量的概率分布和随机抽样方法.利用Monte Carlo EM算法对未知参数进行了迭代.结合Metropolis-Hastings算法对参数的满条件分布进行了Gibbs抽样,基于Gibbs样本对参数进行估计,详细介绍了MCMC方法的实施步骤.随机模拟试验的结果表明各参数Bayes估计的精度较高.     

纵向数据下半参数联合均值协方差模型的贝叶斯分析

基于改进的Cholesky分解,研究分析了纵向数据下半参数联合均值协方差模型的贝叶斯估计和贝叶斯统计诊断,其中非参数部分采用B样条逼近.主要通过应用Gibbs抽样和Metropolis-Hastings算法相结合的混合算法获得模型中未知参数的贝叶斯估计和贝叶斯数据删除影响诊断统计量.并利用诊断统计量的大小来识别数据的异常点.模拟研究和实例分析都表明提出的贝叶斯估计和诊断方法是可行有效的.     

删失截断情形下Weibull分布多变点模型的参数估计

通过添加缺失的寿命变量数据,得到了删失截断情形下Weibull分布多变点模型的完全数据似然函数,研究了变点位置参数和形状参数以及尺度参数的满条件分布.利用Gibbs抽样与Metropolis-Hastings算法相结合的MCMC方法得到了参数的Gibbs样本,把Gibbs样本的均值作为各参数的Bayes估计.详细介绍了MCMC方法的实施步骤.随机模拟试验的结果表明各参数Bayes估计的精度都较高.     

The Construction and Properties of Boundary Kernels for Smoothing Sparse Multinomials

In recent years several authors have investigated the use of smoothing methods for sparse multinomial data. In particular, Hall and Titterington (1987) studied kernel smoothing in detail. It is pointed out here that the bias of kernel estimates of probabilities for cells near the boundaries of the multinomial vector can dominate the mean sum of squared error of the estimator for most true probability vectors. Fortunately, boundary kernels devised to correct boundary effects for kernel regression estimators can achieve the same result for these estimators. Properties of estimates based on boundary kernels are investigated and compared to unmodified kernel estimates and maximum penalized likelihood estimates. Monte Carlo evidence indicates that the boundary-corrected kernel estimates usually outperform uncorrected kernel estimates and are quite competitive with penalized likelihood estimates.     

SOME RESULTS ON ESTIMATION OF THE TAIL INDEX OF A DISTRIBUTION

§１．ＩｎｔｒｏｄｕｃｔｉｏｎＳｕｐｐｏｓｅｔｈａｔＦｉｓａｄｉｓｔｒｉｂｕｔｉｏｎｆｕｎｃｔｉｏｎｓｕｃｈｔｈａｔ,ｆｏｒａｎｙｘ＞０,ｌｉｍｔ→∞１－Ｆ（ｔｘ）１－Ｆ（ｔ）＝ｘ－１γ,γ＞０．（１．１）ＷｅｃａｌγｔｈｅｔａｉｌｉｎｄｅｘｏｆＦ．...     

Fourier methods for smooth distribution function estimation

The limit behavior of the optimal bandwidth sequence for the kernel distribution function estimator is analyzed, in its greatest generality, by using Fourier transform methods. We show a class of distributions for which the kernel estimator achieves a first-order improvement in efficiency over the empirical estimator.     

IIRCT下泊松分布参数多变点模型的贝叶斯估计

通过添加缺损的寿命变量数据得到了带有不完全信息随机截尾试验下泊松分布参数多变点模型的完全数据似然函数,研究了变点位置参数和其它参数的满条件分布.利用Gibbs抽样与Metropolis-Hastings算法相结合的MCMC方法对各参数的满条件分布分别进行了抽样,把Gibbs样本的均值作为各参数的贝叶斯估计,并且详细介绍了MCMC方法的实施步骤.最后进行了随机模拟试验,试验结果表明各参数贝叶斯估计的精度都较高.     

Alternative proof and interpretations for a recent state-dependent importance sampling scheme

Recently, a state-dependent change of measure for simulating overflows in the two-node tandem queue was proposed by Dupuis et al. (Ann. Appl. Probab. 17(4):1306–1346, 2007), together with a proof of its asymptotic optimality. In the present paper, we present an alternative, shorter and simpler proof. As a side result, we obtain interpretations for several of the quantities involved in the change of measure in terms of likelihood ratios. Part of this research has been funded by the Dutch BSIK/BRICKS project; part of this research was done while the first author was visiting INRIA/IRISA, Rennes, France.     

Nonlinear and Non-Gaussian State Space Modeling Using Sampling Techniques

In this paper, the nonlinear non-Gaussian filters and smoothers are proposed using the joint density of the state variables, where the sampling techniques such as rejection sampling (RS), importance resampling (IR) and the Metropolis-Hastings independence sampling (MH) are utilized. Utilizing the random draws generated from the joint density, the density-based recursive algorithms on filtering and smoothing can be obtained. Furthermore, taking into account possibility of structural changes and outliers during the estimation period, the appropriately chosen sampling density is possibly introduced into the suggested nonlinear non-Gaussian filtering and smoothing procedures. Finally, through Monte Carlo simulation studies, the suggested filters and smoothers are examined.