A Dynamic Mixture Model for Unsupervised Tail Estimation without Threshold Selection

Exceedances over high thresholds are often modeled by fitting a generalized Pareto distribution (GPD) on R⁺. It is difficult to select the threshold, above which the GPD assumption is enough solid and enough data is available for inference. We suggest a new dynamically weighted mixture model, where one term of the mixture is the GPD, and the other is a light-tailed density distribution. The weight function varies on R⁺ in such a way that for large values the GPD component is predominant and thus takes the role of threshold selection. The full data set is used for inference on the parameters present in the two component distributions and in the weight function. Maximum likelihood provides estimates with approximate standard deviations. Our approach has been successfully applied to simulated data and to the (previously studied) Danish fire loss data set. We compare the new dynamic mixture method to Dupuis' robust thresholding approach in peaks-over-threshold inference. We discuss robustness with respect to the choice of the light-tailed component and the form of the weight function. We present encouraging simulation results that indicate that the new approach can be useful in unsupervised tail estimation, especially in heavy tailed situations and for small percentiles.     

On special values of certain Dirichlet L-functions

Let r _k (n) denote the number of ways n can be expressed as a sum of k squares. Recently, S. Cooper (Ramanujan J. 6:469–490, [2002]), conjectured a formula for r ₉(t), t≡5 (mod 8), r ₁₁(t), t≡7 (mod 8), where t is a square-free positive integer. In this note we observe that these conjectures follow from the works of Lomadze (Akad. Nauk Gruz. Tr. Tbil. Mat. Inst. Razmadze 17:281–314, [1949]; Acta Arith. 68(3):245–253, [1994]). Further we express r ₉(t), r ₁₁(t) in terms of certain special values of Dirichlet L-functions. Combining these two results we get expressions for these special values of Dirichlet L-functions involving Jacobi symbols.      

Comparison of Methods for the Computation of Multivariate t Probabilities

This article compares methods for the numerical computation of multivariate t probabilities for hyper-rectangular integration regions. Methods based on acceptance-rejection, spherical-radial transformations, and separation-of-variables transformations are considered. Tests using randomly chosen problems show that the most efficient numerical methods use a transformation developed by Genz for multivariate normal probabilities. These methods allow moderately accurate multivariate t probabilities to be quickly computed for problems with as many as 20 variables. Methods for the noncentral multivariate t distribution are also described.     

Estimation Methods for the Multivariate <Emphasis Type="Italic">t</Emphasis> Distribution

The known estimation and simulation methods for multivariate t distributions are reviewed. A review of selected applications is also provided. We believe that this review will serve as an important reference and encourage further research activities in the area.     

Computing multiple-output regression quantile regions from projection quantiles

In the multiple-output regression context, Hallin et al. (Ann Statist 38:635–669, 2010) introduced a powerful data-analytical tool based on regression quantile regions. However, the computation of these regions, that are obtained by considering in all directions an original concept of directional regression quantiles, is a very challenging problem. Paindaveine and Šiman (Comput Stat Data Anal 2011b) described a first elegant solution relying on linear programming techniques. The present paper provides another solution based on the fact that the quantile regions can also be computed from a competing concept of projection regression quantiles, elaborated in Kong and Mizera (Quantile tomography: using quantiles with multivariate data 2008) and Paindaveine and Šiman (J Multivar Anal 2011a). As a by-product, this alternative solution further provides various characteristics useful for statistical inference. We describe in detail the algorithm solving the parametric programming problem involved, and illustrate the resulting procedure on simulated data. We show through simulations that the Matlab implementation of the algorithm proposed in this paper is faster than that from Paindaveine and Šiman (Comput Stat Data Anal 2011b) in various cases.     

Improved prediction for a multivariate normal distribution with unknown mean and variance

The prediction problem for a multivariate normal distribution is considered where both mean and variance are unknown. When the Kullback–Leibler loss is used, the Bayesian predictive density based on the right invariant prior, which turns out to be a density of a multivariate t-distribution, is the best invariant and minimax predictive density. In this paper, we introduce an improper shrinkage prior and show that the Bayesian predictive density against the shrinkage prior improves upon the best invariant predictive density when the dimension is greater than or equal to three.     

Exceedances over High Thresholds: A Guide to Threshold Selection

In this paper, we consider the modeling of exceedances over high thresholds.The natural distribution for such exceedances, the generalized Pareto distribution (GPD), is used and the problematic issue of threshold selection is addressed. We fit the GPD robustly to the data using techniques based on optimal bias-robust estimates. The robust procedure will assign weights between 0 and 1 to each data point. These weights are used to assess the validity of the GPD model for exceedances of the proposed threshold and thus can guide threshold selection. That is, we can initially consider a low threshold and increase it (thus reducing the number of data points) until all weights are close to one. The new approach is used to analyze two of the NERC data sets.     

Family of multivariate generalized t distributions

In this paper, we introduce a new family of multivariate distributions as the scale mixture of the multivariate power exponential distribution introduced by Gómez et al. (Comm. Statist. Theory Methods 27(3) (1998) 589) and the inverse generalized gamma distribution. Since the resulting family includes the multivariate t distribution and the multivariate generalization of the univariate GT distribution introduced by McDonald and Newey (Econometric Theory 18 (11) (1988) 4039) we call this family as the “multivariate generalized t-distributions family”, or MGT for short. We show that this family of distributions belongs to the elliptically contoured distributions family, and investigate the properties. We give the stochastic representation of a random variable distributed as a multivariate generalized t distribution. We give the marginal distribution, the conditional distribution and the distribution of the quadratic forms. We also investigate the other properties, such as, asymmetry, kurtosis and the characteristic function.     

Extremes of Lévy driven mixed MA processes with convolution equivalent distributions

We investigate the extremal behavior of stationary mixed MA processes for t ≥ 0, where f is a deterministic function and Λ is an infinitely divisible and independently scattered random measure. Particular examples of mixed MA processes are superpositions of Ornstein-Uhlenbeck processes, applied as stochastic volatility models in Barndorff-Nielsen and Shephard (2001a). We assume that the finite dimensional distributions of Λ are in the class of convolution equivalent tails and in the maximum domain of attraction of the Gumbel distribution. On the one hand, we compute the tail behavior of Y(0) and sup _{t ∈ [0,1]} Y(t). On the other hand, we study the extremes of Y by means of marked point processes based on maxima of Y in random intervals. A complementary result guarantees the convergence of the running maxima of Y to the Gumbel distribution. Financial support from the Deutsche Forschungsgemeinschaft through a research grant is gratefully acknowledged.     

A census of highly symmetric combinatorial designs

As a consequence of the classification of the finite simple groups, it has been possible in recent years to characterize Steiner t-designs, that is t-(v,k,1) designs, mainly for t=2, admitting groups of automorphisms with sufficiently strong symmetry properties. However, despite the finite simple group classification, for Steiner t-designs with t>2 most of these characterizations have remained long-standing challenging problems. Especially, the determination of all flag-transitive Steiner t-designs with 3≤t≤6 is of particular interest and has been open for about 40 years (cf. Delandtsheer (Geom. Dedicata 41, p. 147, 1992 and Handbook of Incidence Geometry, Elsevier Science, Amsterdam, 1995, p. 273), but presumably dating back to 1965). The present paper continues the author’s work (see Huber (J. Comb. Theory Ser. A 94, 180–190, 2001; Adv. Geom. 5, 195–221, 2005; J. Algebr. Comb., 2007, to appear)) of classifying all flag-transitive Steiner 3-designs and 4-designs. We give a complete classification of all flag-transitive Steiner 5-designs and prove furthermore that there are no non-trivial flag-transitive Steiner 6-designs. Both results rely on the classification of the finite 3-homogeneous permutation groups. Moreover, we survey some of the most general results on highly symmetric Steiner t-designs.      

Peaks-over-threshold stability of multivariate generalized Pareto distributions

It is well-known that the univariate generalized Pareto distributions (GPD) are characterized by their peaks-over-threshold (POT) stability. We extend this result to multivariate GPDs.It is also shown that this POT stability is asymptotically shared by distributions which are in a certain neighborhood of a multivariate GPD. A multivariate extreme value distribution is a typical example.The usefulness of the results is demonstrated by various applications. We immediately obtain, for example, that the excess distribution of a linear portfolio with positive weights a_i, i≤d, is independent of the weights, if (U₁,…,U_d) follows a multivariate GPD with identical univariate polynomial or Pareto margins, which was established by Macke [On the distribution of linear combinations of multivariate EVD and GPD distributed random vectors with an application to the expected shortfall of portfolios, Diploma Thesis, University of Würzburg, 2004, (in German)] and Falk and Michel [Testing for tail independence in extreme value models. Ann. Inst. Statist. Math. 58 (2006) 261-290]. This implies, for instance, that the expected shortfall as a measure of risk fails in this case.     

Stochastic Equations with Time-Dependent Drift Driven by Levy Processes

The stochastic equation dX _t =dS _t +a(t,X _t )dt, t≥0, is considered where S is a one-dimensional Levy process with the characteristic exponent ψ(ξ),ξ∈ℝ. We prove the existence of (weak) solutions for a bounded, measurable coefficient a and any initial value X ₀=x ₀∈ℝ when (ℛe  ψ(ξ))⁻¹=o(|ξ|⁻¹) as |ξ|→∞. These conditions coincide with those found by Tanaka, Tsuchiya and Watanabe (J. Math. Kyoto Univ. 14(1), 73–92, 1974) in the case of a(t,x)=a(x). Our approach is based on Krylov’s estimates for Levy processes with time-dependent drift. Some variants of those estimates are derived in this note.     

Accounting for threshold uncertainty in extreme value estimation

Tail data are often modelled by fitting a generalized Pareto distribution (GPD) to the exceedances over high thresholds. In practice, a threshold is fixed and a GPD is fitted to the data exceeding . A difficulty in this approach is the selection of the threshold above which the GPD assumption is appropriate. Moreover the estimates of the parameters of the GPD may depend significantly on the choice of the threshold selected. Sensitivity with respect to the threshold choice is normally studied but typically its effects on the properties of estimators are not accounted for. In this paper, to overcome the difficulties of the fixed-threshold approach, we propose to model extreme and non-extreme data with a distribution composed of a piecewise constant density from a low threshold up to an unknown end point and a GPD with threshold for the remaining tail part. Since we estimate the threshold together with the other parameters of the GPD we take naturally into account the threshold uncertainty. We will discuss this model from a Bayesian point of view and the method will be illustrated using simulated data and a real data set.     

Hypothesis testing for the common mean of two normal distributions in the presence of an indifference zone

Summary LetX ₁,...,X _m andY _t,...,Y be independent, random samples from populations which are N(θ,σ _x ² ) and N(θ,σ _y ² ), respectively, with all parameters unknown. In testingH ₀:θ=0 againstH ₁:θ≠0, thet-test based upon either sample is known to be admissible in the two-sample setting. If, however, one testsH ₀ againstH ₁:|θ|≧ε>0, with ε arbitrary, our main results show: (i) the construction of a test which is better than the particulart-test chosen, (ii) eacht-test is admissible under the invariance principle with respect to the group of scale changes, and (iii) there does not exist a test which simultaneously is better than botht-tests.     

Integral Representations and Approximations for Multivariate Gamma Distributions

Let R be a p×p-correlation matrix with an “m-factorial” inverse R ⁻¹ = D − BB′ with diagonal D minimizing the rank m of B. A new -variate integral representation is given for p-variate gamma distributions belonging to R, which is based on the above decomposition of R ⁻¹ without the restriction D > 0 required in former formulas. This extends the applicability of formulas with small m. For example, every p-variate gamma cdf can be computed by an at most -variate integral if p = 3 or p = 4. Since computation is only feasible for small m, a given R is approximated by an m-factorial R ₀. The cdf belonging to R is approximated by the cdf associated with R ₀ and some additional correction terms with the deviations between R and R ₀.     

An Optimal Wavelet Series Expansion of the Riemann–Liouville Process

Consider the Riemann–Liouville process R ^α ={R ^α (t)} _t∈[0,1] with parameter α>1/2. Depending on α, wavelet series representations for R ^α (t) of the form ∑ _k=1^∞ u _k (t)ε _k are given, where the u _k are deterministic functions, and {ε _k } _k≥1 is a sequence of i.i.d. standard normal random variables. The expansion is based on a modified Daubechies wavelet family, which was originally introduced in Meyer (Rev. Mat. Iberoam. 7:115–133, 1991). It is shown that these wavelet series representations are optimal in the sense of Kühn–Linde (Bernoulli 8:669–696, 2002) for all values of α>1/2.     

Deducing the multidimensional Szemerédi theorem from an infinitary removal lemma

We offer a new proof of the Furstenberg-Katznelson multiple recurrence theorem for several commuting probability-preserving transformations T ₁, T ₂, …, T _d : ℤ ↷ (X, ∑, μ) ([6]), and so, via the Furstenberg correspondence principle introduced in [5], a new proof of the multi-dimensional Szemerédi Theorem. We bypass the careful manipulation of certain towers of factors of a probability-preserving system that underlies the Furstenberg-Katznelson analysis, instead modifying an approach recently developed in [1] to pass to a large extension of our original system in which this analysis greatly simplifies. The proof is then completed using an adaptation of arguments developed by Tao in [13] for his study of an infinitary analog of the hypergraph removal lemma. In a sense, this addresses the difficulty, highlighted by Tao, of establishing a direct connection between his infinitary, probabilistic approach to the hypergraph removal lemma and the infinitary, ergodic-theoretic approach to Szemerédi’s Theorem set in motion by Furstenberg [5].     

A Little More on Coz-Unique Frames

Coz-unique frames were defined and characterized by Banaschewski and Gilmour (J Pure Appl Algebra 157:1–22, 2001). In this note we give further characterizations of these frames along the lines of characterizations of absolutely z-embedded spaces obtained by Blair and Hager (Math Z 136:41–52, 1974) on the one hand, and by Hager and Johnson (Canad J Math 20:389–393, 1968) on the other. We also extend to frames certain characterizations of z-embedded spaces; namely, we give a characterization of coz-onto frame homomorphisms in terms of normal covers.      

On multivariate Gaussian tails

Let {X _{n, n} ≥1} be a sequence of standard Gaussian random vectors in ℝ _d ,d ≥ 2. In this paper we derive lower and upper bounds for the tail probabilityP{X _n >t _n } witht _n ∈ ℝ _d some threshold. We improve for instance bounds on Mills ratio obtained by Savage (1962,J. Res. Nat. Bur. Standards Sect. B,66, 93–96). Furthermore, we prove exact asymptotics under fairly general conditions on bothX _n andt _n , as ‖t _n ‖→∞ where the correlation matrix Σ _n ofX _n may also depend onn.     

On the Number of i.i.d. Samples Required to Observe All of the Balls in an Urn

Suppose an urn contains m distinct balls, numbered 1,...,m, and let τ denote the number of i.i.d. samples required to observe all of the balls in the urn. We generalize the partial fraction expansion type arguments used by Pólya (Z Angew Math Mech 10:96–97, 1930) for approximating \mathbbE(t)\mathbb{E}(\tau) in the case of fixed sample sizes to obtain an approximation of \mathbbE(t)\mathbb{E}(\tau) when the sample sizes are i.i.d. random variables. The approximation agrees with that of Sellke (Ann Appl Probab 5(1):294–309, 1995), who made use of Wald’s equation and a Markov chain coupling argument. We also derive a new approximation of \mathbbV(t)\mathbb{V}(\tau), provide an (improved) bound on the error in these approximations, derive a recurrence for \mathbbE(t)\mathbb{E}(\tau), give a new large deviation type result for tail probabilities, and look at some special cases.