Bayesian Inference with Wavelets: Density Estimation

Abstract

We propose a prior probability model in the wavelet coefficient space. The proposed model implements wavelet coefficient thresholding by full posterior inference in a coherent probability model. We introduce a prior probability model with mixture priors for the wavelet coefficients. The prior includes a positive prior probability mass at zero which leads to a posteriori thresholding and generally to a posteriori shrinkage on the coefficients. We discuss an efficient posterior simulation scheme to implement inference in the proposed model. The discussion is focused on the density estimation problem. However, the introduced prior probability model on the wavelet coefficient space and the Markov chain Monte Carlo scheme are general.     

On the correlation structure of exponential order statistics and some extensions

In this paper, we carry out a theoretical study of the correlation coefficients between exponential order statistics and their monotonicity properties. Then, aided by a Monte Carlo simulation study, we make some conjectures about the correlation structure of order statistics from a larger class of distributions.     

Wavelet improvement in turning point detection using a hidden Markov model: from the aspects of cyclical identification and outlier correction

The hidden Markov model (HMM) has been widely used in regime classification and turning point detection for econometric series after the decisive paper by Hamilton (Econometrica 57(2):357–384, 1989). The present paper will show that when using HMM to detect the turning point in cyclical series, the accuracy of the detection will be influenced when the data are exposed to high volatilities or combine multiple types of cycles that have different frequency bands. Moreover, outliers will be frequently misidentified as turning points. The present paper shows that these issues can be resolved by wavelet multi-resolution analysis based methods. By providing both frequency and time resolutions, the wavelet power spectrum can identify the process dynamics at various resolution levels. We apply a Monte Carlo experiment to show that the detection accuracy of HMMs is highly improved when combined with the wavelet approach. Further simulations demonstrate the excellent accuracy of this improved HMM method relative to another two change point detection algorithms. Two empirical examples illustrate how the wavelet method can be applied to improve turning point detection in practice.     

Bayesian Methods for Wavelet Series in Single-Index Models

Single-index models have found applications in econometrics and biometrics, where multidimensional regression models are often encountered. This article proposes a nonparametric estimation approach that combines wavelet methods for nonequispaced designs with Bayesian models. We consider a wavelet series expansion of the unknown regression function and set prior distributions for the wavelet coefficients and the other model parameters. To ensure model identifiability, the direction parameter is represented via its polar coordinates. We employ ad hoc hierarchical mixture priors that perform shrinkage on wavelet coefficients and use Markov chain Monte Carlo methods for a posteriori inference. We investigate an independence-type Metropolis-Hastings algorithm to produce samples for the direction parameter. Our method leads to simultaneous estimates of the link function and of the index parameters. We present results on both simulated and real data, where we look at comparisons with other methods.     

GMRES and the minimal polynomial

We present a qualitative model for the convergence behaviour of the Generalised Minimal Residual (GMRES) method for solving nonsingular systems of linear equationsAx =b in finite and infinite dimensional spaces. One application of our methods is the solution of discretised infinite dimensional problems, such as integral equations, where the constants in the asymptotic bounds are independent of the mesh size.Our model provides simple, general bounds that explain the convergence of GMRES as follows: If the eigenvalues ofA consist of a single cluster plus outliers then the convergence factor is bounded by the cluster radius, while the asymptotic error constant reflects the non-normality ofA and the distance of the outliers from the cluster. If the eigenvalues ofA consist of several close clusters, then GMRES treats the clusters as a single big cluster, and the convergence factor is the radius of this big cluster. We exhibit matrices for which these bounds are tight.Our bounds also lead to a simpler proof of existing r-superlinear convergence results in Hilbert space.This research was partially supported by National Science Foundation grants DMS-9122745, DMS-9423705, CCR-9102853, CCR-9400921, DMS-9321938, DMS-9020915, and DMS-9403224.     

A differential evolution-based regression framework for forecasting Bitcoin price

This research proposes a differential evolution-based regression framework for forecasting one day ahead price of Bitcoin. The maximal overlap discrete wavelet transformation first decomposes the original series into granular linear and nonlinear components. We then fit polynomial regression with interaction (PRI) and support vector regression (SVR) on linear and nonlinear components and obtain component-wise projections. The sum of these projections constitutes the final forecast. For accurate predictions, the PRI coefficients and tuning of the hyperparameters of SVR must be precisely estimated. Differential evolution, a metaheuristic optimization technique, helps to achieve these goals. We compare the forecast accuracy of the proposed regression framework with six advanced predictive modeling algorithms- multilayer perceptron neural network, random forest, adaptive neural fuzzy inference system, standalone SVR, multiple adaptive regression spline, and least absolute shrinkage and selection operator. Finally, we perform the numerical experimentation based on—(1) the daily closing prices of Bitcoin for January 10, 2013, to February 23, 2019, and (2) randomly generated surrogate time series through Monte Carlo analysis. The forecast accuracy of the proposed framework is higher than the other predictive modeling algorithms.

     

A computational study of a gradient-based log-barrier algorithm for a class of large-scale SDPs

 The authors of this paper recently introduced a transformation [4] that converts a class of semidefinite programs (SDPs) into nonlinear optimization problems free of matrix-valued constraints and variables. This transformation enables the application of nonlinear optimization techniques to the solution of certain SDPs that are too large for conventional interior-point methods to handle efficiently. Based on the transformation, we proposed a globally convergent, first-order (i.e., gradient-based) log-barrier algorithm for solving a class of linear SDPs. In this paper, we discuss an efficient implementation of the proposed algorithm and report computational results on semidefinite relaxations of three types of combinatorial optimization problems. Our results demonstrate that the proposed algorithm is indeed capable of solving large-scale SDPs and is particularly effective for problems with a large number of constraints. Received: June 22, 2001 / Accepted: January 20, 2002 Published online: December 9, 2002 RID="†" ID="†"Computational results reported in this paper were obtained on an SGI Origin2000 computer at Rice University acquired in part with support from NSF Grant DMS-9872009. RID="⋆" ID="⋆"This author was supported in part by NSF Grants CCR-9902010, INT-9910084 and CCR-0203426 RID="⋆⋆" ID="⋆⋆"This author was supported in part by NSF Grants CCR-9902010, INT-9910084 and CCR-0203113 RID="⋆⋆⋆" ID="⋆⋆⋆"This author was supported in part by DOE Grant DE-FG03-97ER25331, DOE/LANL Contract 03891-99-23 and NSF Grant DMS-9973339. Key Words. semidefinite program – semidefinite relaxation – nonlinear programming – interior-point methods – limited memory quasi-Newton methods. Mathematics Subject Classification (1991): 90C06, 90C27, 90C30.     

Conditioning of convex piecewise linear stochastic programs

 In this paper we consider stochastic programming problems where the objective function is given as an expected value of a convex piecewise linear random function. With an optimal solution of such a problem we associate a condition number which characterizes well or ill conditioning of the problem. Using theory of Large Deviations we show that the sample size needed to calculate the optimal solution of such problem with a given probability is approximately proportional to the condition number. Received: May 2000 / Accepted: May 2002-07-16 Published online: September 5, 2002 RID="★" The research of this author was supported, in part, by grant DMS-0073770 from the National Science Foundation Key Words. stochastic programming – Monte Carlo simulation – large deviations theory – ill-conditioned problems     

Rotor dynamics behavior of turbo-expander involving droplet impact

ABSTRACT

This paper dedicates on the rotor dynamics behaviour research on the turbo-expander rotor system involving droplet impact. A stochastic model based on Beta distribution and Bernoulli distribution of droplet generation is established and the formulations of droplet impact forces are deduced, which is applied on the rotor dynamics equations of the tilting pad bearing supported turbo-expander considering the temperature gradient for a further analysis. A time domain research is carried out and a conclusion that continuously droplet impact will perturb the steady vibration of the turbo-expander rotor system is obtained. Monte Carlo method is implemented for a statistics dynamics research and the results suggests that in the design of expander impellers, in order to decrease the uncertainty brought by droplet impact, the number of channels should be as few as possible, the droplet impact should be controlled to occur uniformly, and the collision, entrainment of the primary droplets and the stripping of the liquid film on the blade should be strictly restrained.     

On Markov Chain Monte Carlo Acceleration

Abstract

Markov chain Monte Carlo (MCMC) methods are currently enjoying a surge of interest within the statistical community. The goal of this work is to formalize and support two distinct adaptive strategies that typically accelerate the convergence of an MCMC algorithm. One approach is through resampling; the other incorporates adaptive switching of the transition kernel. Support is both by analytic arguments and simulation study. Application is envisioned in low-dimensional but nontrivial problems. Two pathological illustrations are presented. Connections with reparameterization are discussed as well as possible difficulties with infinitely often adaptation.