Extensions of the conjugate prior through the Kullback-Leibler separators

The conjugate prior for the exponential family, referred to also as the natural conjugate prior, is represented in terms of the Kullback-Leibler separator. This representation permits us to extend the conjugate prior to that for a general family of sampling distributions. Further, by replacing the Kullback-Leibler separator with its dual form, we define another form of a prior, which will be called the mean conjugate prior. Various results on duality between the two conjugate priors are shown. Implications of this approach include richer families of prior distributions induced by a sampling distribution and the empirical Bayes estimation of a high-dimensional mean parameter.     

Locally best rotation-invariant rank tests for modal location

For a general class of unipolar, rotationally symmetric distributions on the multi-dimensional unit spherical surface, a characterization of locally best rotation-invariant test statistics is exploited in the construction of locally best rotation-invariant rank tests for modal location. Allied statistical distributional problems are appraised, and in the light of these assessments, asymptotic relative efficiency of a class of rotation-invariant rank tests (with respect to some of their parametric counterparts) is studied. Finite sample permutational distributional perspectives are also appraised.     

Bayesian analysis of finite mixture models of distributions from exponential families

This paper deals with the Bayesian analysis of finite mixture models with a fixed number of component distributions from natural exponential families with quadratic variance function (NEF-QVF). A unified Bayesian framework addressing the two main difficulties in this context is presented, i.e., the prior distribution choice and the parameter unidentifiability problem. In order to deal with the first issue, conjugate prior distributions are used. An algorithm to calculate the parameters in the prior distribution to obtain the least informative one into the class of conjugate distributions is developed. Regarding the second issue, a general algorithm to solve the label-switching problem is presented. These techniques are easily applied in practice as it is shown with an illustrative example.     

Grouped Dirichlet distribution: A new tool for incomplete categorical data analysis

Motivated by the likelihood functions of several incomplete categorical data, this article introduces a new family of distributions, grouped Dirichlet distributions (GDD), which includes the classical Dirichlet distribution (DD) as a special case. First, we develop distribution theory for the GDD in its own right. Second, we use this expanded family as a new tool for statistical analysis of incomplete categorical data. Starting with a GDD with two partitions, we derive its stochastic representation that provides a simple procedure for simulation. Other properties such as mixed moments, mode, marginal and conditional distributions are also derived. The general GDD with more than two partitions is considered in a parallel manner. Three data sets from a case-control study, a leprosy survey, and a neurological study are used to illustrate how the GDD can be used as a new tool for analyzing incomplete categorical data. Our approach based on GDD has at least two advantages over the commonly used approach based on the DD in both frequentist and conjugate Bayesian inference: (a) in some cases, both the maximum likelihood and Bayes estimates have closed-form expressions in the new approach, but not so when they are based on the commonly-used approach; and (b) even if a closed-form solution is not available, the EM and data augmentation algorithms in the new approach converge much faster than in the commonly-used approach.     

Weighted Poisson Distributions for Overdispersion and Underdispersion Situations

The main goal of this paper is to introduce new exponential families, that come from the concept of weighted distribution, that include and generalize the Poisson distribution. In these families there are distributions with index of dispersion greater than, equal to or smaller than one. This property makes them suitable to fit discrete data in overdispersion or underdispersion situations. We study the statistical properties of the families and we provide a useful interpretation of the parameters. Two classical examples are considered in order to compare the fits with some other distributions. To obtain the fits with the new family, the study of the profile log-likelihood is required.     

Optimally Adjusted Mixture Sampling and Locally Weighted Histogram Analysis

Consider the two problems of simulating observations and estimating expectations and normalizing constants for multiple distributions. First, we present a self-adjusted mixture sampling method, which accommodates both adaptive serial tempering and a generalized Wang–Landau algorithm. The set of distributions are combined into a labeled mixture, with the mixture weights depending on the initial estimates of log normalizing constants (or free energies). Then, observations are generated by Markov transitions, and free energy estimates are adjusted online by stochastic approximation. We propose two stochastic approximation schemes by Rao–Blackwellization of the scheme commonly used, and derive the optimal choice of a gain matrix, resulting in the minimum asymptotic variance for free energy estimation, in a simple and feasible form. Second, we develop an offline method, locally weighted histogram analysis, for estimating free energies and expectations, using all the simulated data from multiple distributions by either self-adjusted mixture sampling or other sampling algorithms. This method can be computationally much faster, with little sacrifice of statistical efficiency, than a global method currently used, especially when a large number of distributions are involved. We provide both theoretical results and numerical studies to demonstrate the advantages of the proposed methods.     

A posteriori error bounds for quasi-linear Dirichlet and Neumann problems in full divergent form

Second-order quasi-linear Dirichlet and Neumann problems in four-term divergent form on a simply connected domain with a Lipschitz-continuous boundary of finite length are considered. Derivatives and primitives of distributions on the boundary are defined in such a way that for sufficiently smooth boundary distributions, these derivatives and primitives coincide with derivatives and primitives with respect to arc length on the boundary. Using these concepts, conjugate problems, that is, a pair of one Dirichlet and one Neumann problem, the minima of the energies of which add to zero, are introduced. From the concept of conjugate problems, two-sided bounds for the energy of the exact solution of any given Dirichlet or Neumann problem are constructed. These two-sided bounds for the energy at the exact solution are in turn used to obtain a posteriori error bounds for the norm of the difference of the approximate and exact solutions of the problem. These a posteriori bounds consist of a constant times the sum of the energies of the approximate solutions of the conjugate Dirichlet and Neumann problems and are easily constructed numerically.     

Subregion-Adaptive Integration of Functions Having a Dominant Peak

Abstract

Many statistical multiple integration problems involve integrands that have a dominant peak. In applying numerical methods to solve these problems, statisticians have paid relatively little attention to existing quadrature methods and available software developed in the numerical analysis literature. One reason these methods have been largely overlooked, even though they are known to be more efficient than Monte Carlo for well-behaved problems of low dimensionality, may be that when applied naively they are poorly suited for peaked-integrand problems. In this article we use transformations based on “split t” distributions to allow the integrals to be efficiently computed using a subregion-adaptive numerical integration algorithm. Our split t distributions are modifications of those suggested by Geweke and may also be used to define Monte Carlo importance functions. We then compare our approach to Monte Carlo. In the several examples we examine here, we find subregion-adaptive integration to be substantially more efficient than importance sampling.     

Two families of scaled three-term conjugate gradient methods with sufficient descent property for nonconvex optimization

Numerical Algorithms - In this paper, we present two families of modified three-term conjugate gradient methods for solving unconstrained large-scale smooth optimization problems. We show that our...     

The optimality principle for discrete and first order partial differential inclusions

Necessary and sufficient conditions for optimality are derived for the problems under consideration on the basis of the apparatus of locally conjugate mappings and local tents.     

Stable convergence of the log-likelihood ratio to a mixture of infinitely divisible distributions

Some results concerning the asymptotic behavior of the log-likelihood ratio (LLR) and also of certain other random variables closely associated with the likelihood ratio are presented. More specifically, in the present paper we formulate the conditions for the stable convergence in distribution of the LLR for two sequences of the probability measures to a mixture of infinitely divisible distributions with finite variance. Moreover, the notion of a locally asymptotically mixed infinitely divisible (LAMID) sequence of parametric families of the probability measures is introduced, and it is shown that when a certain kind of differentiability-type regularity condition is satisfied, the given sequence of families satisfies the LAMID condition. These results extend and supplement the previous investigations of the author concerning non-Gaussian asymptotic distributions in statistics. Proceedings of the XVII Seminar on Stability Problems for Stochastic Models, Kazan, Russia, 1995, Part III.     

Reference priors for exponential families with simple quadratic variance function

Reference analysis is one of the most successful general methods to derive noninformative prior distributions. In practice, however, reference priors are often difficult to obtain. Recently developed theory for conditionally reducible natural exponential families identifies an attractive reparameterization which allows one, among other things, to construct an enriched conjugate prior. In this paper, under the assumption that the variance function is simple quadratic, the order-invariant group reference prior for the above parameter is found. Furthermore, group reference priors for the mean- and natural parameter of the families are obtained. A brief discussion of the frequentist coverage properties is also presented. The theory is illustrated for the multinomial and negative-multinomial family. Posterior computations are especially straightforward due to the fact that the resulting reference distributions belong to the corresponding enriched conjugate family. A substantive application of the theory relates to the construction of reference priors for the Bayesian analysis of two-way contingency tables with respect to two alternative parameterizations.     

A Gaussian Mixed Model for Learning Discrete Bayesian Networks

In this paper, we address the problem of learning discrete Bayesian networks from noisy data. A graphical model based on a mixture of Gaussian distributions with categorical mixing structure coming from a discrete Bayesian network is considered. The network learning is formulated as a maximum likelihood estimation problem and performed by employing an EM algorithm. The proposed approach is relevant to a variety of statistical problems for which Bayesian network models are suitable—from simple regression analysis to learning gene/protein regulatory networks from microarray data.     

A locally computed suboptimal control strategy for a class of interconnected systems

In this paper, a locally computed suboptimal control strategy for a class of interconnected systems is introduced. First, optimal statefeedback control equations are derived for a finite-horizon quadratic cost. Then, the control for each subsystem is separated into two portions. The first portion stabilizes the isolated subsystem, and the second portion corresponds to the interactions. To achieve a locally calculable control, an approximation to the optimal control equations is introduced, and two iterative suboptimal control algorithms are developed. In the first algorithm, the initial conditions of subsystems are assumed to be known; in the second algorithm, this information is replaced by statistical distributions. The orders of errors in the iterations of the algorithm and in the suboptimality are given in terms of interconnections. An example with comparisons is also included to show the performance of the approach.     

On duality in problems of optimal control described by convex differential inclusions of Goursat-Darboux type

Sufficient conditions of optimality are derived for convex and non-convex problems with state constraints on the basis of the apparatus of locally conjugate mappings. The duality theorem is formulated and the conditions under which the direct and dual problems are connected by the duality relation are searched for.     

局部β-凸空间的共轭锥与Hahn-Banach定理

由 [1 ],局部β-凸空间 X的共轭锥 X*β 取代共轭空间在局部β-凸分析中扮演核心角色 .本文第一部分在局部β-凸空间上给出β-次半范的 Hahn-Banach定理 ,第二部分通过共轭锥 ( X*β ,‖‖ )得到赋β-范空间 ( X,‖‖β)的可分性定理 ,第三部分给出局部 β-凸空间的共轭锥 X*β 在一致收敛拓扑下的完备性定理等 .     

Two effective hybrid conjugate gradient algorithms based on modified BFGS updates

Based on two modified secant equations proposed by Yuan, and Li and Fukushima, we extend the approach proposed by Andrei, and introduce two hybrid conjugate gradient methods for unconstrained optimization problems. Our methods are hybridizations of Hestenes-Stiefel and Dai-Yuan conjugate gradient methods. Under proper conditions, we show that one of the proposed algorithms is globally convergent for uniformly convex functions and the other is globally convergent for general functions. To enhance the performance of the line search procedure, we propose a new approach for computing the initial value of the steplength for initiating the line search procedure. We give a comparison of the implementations of our algorithms with two efficiently representative hybrid conjugate gradient methods proposed by Andrei using unconstrained optimization test problems from the CUTEr collection. Numerical results show that, in the sense of the performance profile introduced by Dolan and Moré, the proposed hybrid algorithms are competitive, and in some cases more efficient.     

局部凸空间上检验函数与广义函数空间的嵌入定理及其应用

设Y是局部凸向量空间,其上装配有GaussianRadon测度γ．A（Y）（或ε（Y）是Y上检验函数空间（或με（Y）是相应的分布函数空间·我们证明了：（或με（Y）,并由此得到μA（Y）（或με（Y））上的Fourier变换公式.其中“*”表示复共轭算子,“”表示连续稠线性嵌入.进一步还得到了A（Y）（或ε（Y））上无穷维伪微分算子A是L2（Y,γ）上连续的充要条件是其共轭算子A’满足A’（L2（Y,γ）L2（Y,γ）．     

Locally Desarguesian spaces

In Riemannian spaces, locally Desarguesian spaces have constant curvature and are therefore locally symmetric. This does not hold for Finsler spaces, so that locally Desarguesian spaces represent a generalization other than the obvious one we studied previously of (certain) Riemannian symmetric spaces. In this paper we discuss them in detail; as an example of the results obtained we mention that a simply connected locally Desarguesian space without conjugate points is globally Desarguesian. Applications are then given to spaces which are locally symmetric in a wider sense. We also study (and in Minkowski spaces determine exactly) the properties of functions which measure the distance of a point from those on a line.     

Bayesian demand updating in the lost sales newsvendor problem: A two-moment approximation

We consider Bayesian updating of demand in a lost sales newsvendor model with censored observations. In a lost sales environment, where the arrival process is not recorded, the exact demand is not observed if it exceeds the beginning stock level, resulting in censored observations. Adopting a Bayesian approach for updating the demand distribution, we develop expressions for the exact posteriors starting with conjugate priors, for negative binomial, gamma, Poisson and normal distributions. Having shown that non-informative priors result in degenerate predictive densities except for negative binomial demand, we propose an approximation within the conjugate family by matching the first two moments of the posterior distribution. The conjugacy property of the priors also ensure analytical tractability and ease of computation in successive updates. In our numerical study, we show that the posteriors and the predictive demand distributions obtained exactly and with the approximation are very close to each other, and that the approximation works very well from both probabilistic and operational perspectives in a sequential updating setting as well.