Review of testing issues in extremes: in honor of Professor Laurens de Haan

As a leading statistician in extreme value theory, Professor Laurens de Haan has made significant contribution in both probability and statistics of extremes. In honor of his 70th birthday, we review testing issues in extremes, which include research done by Professor Laurens de Haan and many others. In comparison with statistical estimation in extremes, research on testing has received less attention. So we also point out some practical questions in this direction.      

It was 30 years ago today when Laurens de Haan went the multivariate way

Since the publication of his masterpiece on regular variation and its application to the weak convergence of (univariate) sample extremes in 1970, Laurens de Haan (Thesis, Mathematical Centre Tract vol. 32, University of Amsterdam, 1970) is among the leading mathematicians in the world, with a particular focus on extreme value theory (EVT). On the occasion of his 70th birthday it is a great pleasure and a privilege to follow his route through multivariate EVT, which started only seven years later in 1977, when Laurens de Haan published his first paper on multivariate EVT, jointly with Sid Resnick.      

An interview with Laurens de Haan

Laurens de Haan was born January 15, 1937 in Rotterdam, The Netherlands. He graduated 1966 in mathematics and received a doctoral degree in 1970 from the University of Amsterdam, while working at the Mathematical center CWI in Amsterdam. Since 1973 he was Professor for probability and mathematical statistics at the Econometric Institute of the Economic Faculty at the Erasmus University Rotterdam, where he retired 1998. Since 2008 he is part-time professor at the Department of Econometrics and Operations Research of Tilburg University. Laurens de Haan has been active in research throughout his career. He has published more than 110 scientific papers. Among other distinctions, he was elected IMS fellow for his seminal contributions to extreme value theory in 1977, and he was appointed Honorary Doctor of the University of Lisbon in 2000.     

Some aspects of extreme value statistics under serial dependence

On the occasion of Laurens de Haan’s 70th birthday, we discuss two aspects of the statistical inference on the extreme value behavior of time series with a particular emphasis on his important contributions. First, the performance of a direct marginal tail analysis is compared with that of a model-based approach using an analysis of residuals. Second, the importance of the extremal index as a measure of the serial extremal dependence is discussed by the example of solutions of a stochastic recurrence equation.      

On testing extreme value conditions

Applications of univariate extreme value theory rely on certain as- sumptions. Recently, two methods for testing these extreme value conditions are derived by [Dietrich, D., de Haan, L., Hüsler, J., Extremes 5: 71–85, (2002)] and [Drees, H., de Haan, L., Li, D., J. Stat. Plan. Inference, 136: 3498–3538, (2006)]. In this paper we compare the two tests by simulations and investigate the effect of a possible weight function by choosing a parameter, the test error and the power of each test. The conclusions are useful for extreme value applications.     

Smoothing the Moment Estimator of the Extreme Value Parameter

Let {X _n be a sequence of i.i.d. random variables whose common distribution F belongs to the domain of attraction of an extreme value law. A semi-parametric estimator of the extreme value parameter is the Dekkers, Einmahl and de Haan [8] moment estimator. Practical use of this estimator requires the problematic choice of a number k=k(n) of upper order statistics and there are few reliable guidelines for this choice. An averaging or smoothing technique is proposed for this estimator yielding a less volatile function of k which in practice aids estimation.     

On Domains of Attraction of Multivariate Extreme Value Distributions under Absolute Continuity

The paper gives sufficient conditions for domains of attraction of multivariate extreme value distributions. Under the assumption of absolute continuity of a multivariate distribution, the criteria enable one to examine, by using limits of some rescaled conditional densities, whether the distribution belongs to the domain of attraction of some multivariate extreme value distribution. If this is the case, the criteria also determine how to construct such an extreme value distribution. Unlike the criterion given by de Haan and Resnick [1987,Stochastic Process. Appl.2583–93], the criteria are easily applicable even when the marginal tails are not Pareto-like.     

The expected payoff to Internet auctions

In an Internet auction, the expected payoff acts as a benchmark of the reasonableness of the price that is paid for the purchased item. Since the number of potential bidders is not observable, the expected payoff is difficult to estimate accurately. We approach this problem by considering the bids as a record and 2-record sequence of the potential bidder’s valuation and using the Extreme Value Theory models to model the tail distribution of the bidder’s valuation and study the expected payoff. Along the discussions for three different cases regarding the extreme value index γ, we show that the observed payoff does not act as an accurate estimation of the expected payoff in all the cases except a subclass of the case γ = 0. Within this subclass and under a second order condition, the observed payoff consistently converges to the expected payoff and the corresponding asymptotic normality holds.      

Bootstrap and empirical likelihood methods in extremes

One of the major interests in extreme-value statistics is to infer the tail properties of the distribution functions in the domain of attraction of an extreme-value distribution and to predict rare events. In recent years, much effort in developing new methodologies has been made by many researchers in this area so as to diminish the impact of the bias in the estimation and achieve some asymptotic optimality in inference problems such as estimating the optimal sample fractions and constructing confidence intervals of various quantities. In particular, bootstrap and empirical likelihood methods, which have been widely used in many areas of statistics, have drawn attention. This paper reviews some novel applications of the bootstrap and the empirical likelihood techniques in extreme-value statistics. Dedicated to Professor Laurens de Haan on the occasion of his 70th birthday.     

Copula parameter estimation by maximum-likelihood and minimum-distance estimators: a simulation study

The purpose of this paper is to present a comprehensive Monte Carlo simulation study on the performance of minimum-distance (MD) and maximum-likelihood (ML) estimators for bivariate parametric copulas. In particular, I consider Cramér-von-Mises-, Kolmogorov-Smirnov- and L ₁-variants of the CvM-statistic based on the empirical copula process, Kendall’s dependence function and Rosenblatt’s probability integral transform. The results presented in this paper show that regardless of the parametric form of the copula, the sample size or the location of the parameter, maximum-likelihood yields smaller estimation biases at less computational effort than any of the MD-estimators. The MD-estimators based on copula goodness-of-fit metrics, on the other hand, suffer from large biases especially when used for estimating the parameters of archimedean copulas. Moreover, the results show that the bias and efficiency of the minimum-distance estimators are strongly influenced by the location of the parameter. Conversely, the results for the maximum-likelihood estimator are relatively stable over the parameter interval of the respective parametric copula.     

Semi-Iterative Minimum Cross-Entropy Algorithms for Rare-Events,Counting, Combinatorial and Integer Programming

We present a new generic minimum cross-entropy method, called the semi-iterative MinxEnt, or simply SME, for rare-event probability estimation, counting, and approximation of the optimal solutions of a broad class of NP-hard linear integer and combinatorial optimization problems (COP’s). The main idea of our approach is to associate with each original problem an auxiliary single-constrained convex MinxEnt program of a special type, which has a closed-form solution. We prove that the optimal pdf obtained from the solution of such a specially designed MinxEnt program is a zero variance pdf, provided the “temperature” parameter is set to minus infinity. In addition we prove that the parametric pdf based on the product of marginals obtained from the optimal zero variance pdf coincides with the parametric pdf of the standard cross-entropy (CE) method. Thus, originally designed at the end of 1990s as a heuristics for estimation of rare-events and COP’s, CE has strong connection with MinxEnt, and thus, strong mathematical foundation. The crucial difference between the proposed SME method and the standard CE counterparts lies in their simulation-based versions: in the latter we always require to generate (via Monte Carlo) a sequence of tuples including the temperature parameter and the parameter vector in the optimal marginal pdf’s, while in the former we can fix in advance the temperature parameter (to be set to a large negative number) and then generate (via Monte Carlo) a sequence of parameter vectors of the optimal marginal pdf’s alone. In addition, in contrast to CE, neither the elite sample no the rarity parameter is needed in SME. As result, the proposed SME algorithm becomes simpler, faster and at least as accurate as the standard CE. Motivated by the SME method we introduce a new updating rule for the parameter vector in the parametric pdf of the CE program. We show that the CE algorithm based on the new updating rule, called the combined CE (CCE), is at least as fast and accurate as its standard CE and SME counterparts. We also found numerically that the variance minimization (VM)-based algorithms are the most robust ones. We, finally, demonstrate numerically that the proposed algorithms, and in particular the CCE one, allows accurate estimation of counting quantities up to the order of hundred of decision variables and hundreds of constraints. This research was supported by the Israel Science Foundation (grant No 191-565).     

A two-step estimator of the extreme value index

In this paper, we build a two-step estimator , which satisfies , where is the well-known maximum likelihood estimator of the extreme value index. Since the two-step estimator can be calculated easily as a function of the observations, it is much simpler to use in practice. By properly choosing the first step estimator, such as the Pickands estimator, we can even get a shift and scale invariant estimator with the above property. The author thanks Laurens de Haan for motivating this work and giving helpful comments. The author also thanks two anonymous referees for their useful comments.     

A note on the rigid core of affine surfaces

During recent decades, $$\mathbb {G}_a$$-actions, especially certain invariants of $$\mathbb {G}_a$$-actions, have been important tools in the study of affine varieties. The $$\mathbb {G}_a$$-actions are usually studied through locally nilpotent derivations in characteristic zero and exponential maps (see Definition 1.1) in arbitrary characteristic. The “Makar-Limanov invariant” of locally nilpotent derivations played a pivotal role in solving the linearization conjecture in the 1990s, while invariants of exponential maps were central to N. Gupta’s resolution of the Zariski cancellation problem in positive characteristic. In the study of locally nilpotent derivations on commutative algebras containing $$\mathbb {Q}$$, Freudenburg and Moser-Jauslin (Mich Math J 62:227–258, (2013), Theorem 6.1) have introduced a new invariant called “rigid core” and used it to formulate an alternative version of Mason’s theorem and to prove a well-known analogue of Fermat’s last theorem for rational functions (Freudenburg and Moser-Jauslin (2013), Corollary 6.1). In this note, we consider the concept of the rigid core in the framework of exponential maps on commutative algebras over an algebraically closed field k of arbitrary characteristic. We observe that for any factorial k-domain B with $${\text {tr.deg}}_k(B)=2$$, the concept of rigid core coincides with the Makar-Limanov invariant. We also show that over any affine two-dimensional normal k-domain B, its rigid core is a stable invariant.     

Estimation of Kullback–Leibler Divergence by Local Likelihood

Motivated from the bandwidth selection problem in local likelihood density estimation and from the problem of assessing a final model chosen by a certain model selection procedure, we consider estimation of the Kullback–Leibler divergence. It is known that the best bandwidth choice for the local likelihood density estimator depends on the distance between the true density and the ‘vehicle’ parametric model. Also, the Kullback–Leibler divergence may be a useful measure based on which one judges how far the true density is away from a parametric family. We propose two estimators of the Kullback-Leibler divergence. We derive their asymptotic distributions and compare finite sample properties. Research of Young Kyung Lee was supported by the Brain Korea 21 Projects in 2004. Byeong U. Park’s research was supported by KOSEF through Statistical Research Center for Complex Systems at Seoul National University.     

The mathematics of F. J. Almgren, Jr.

Frederick Justin Almgren, Jr, one of the world’s leading geometric analysts and a pioneer in the geometric calculus of variations, died on February 5, 1997 at the age of 63 as a result of myelodysplasia. Throughout his career, Almgren brought great geometric insight, technical power, and relentless determination to bear on a series of the most important and difficult problems in his field. He solved many of them and, in the process, discovered ideas which turned out to be useful for many other problems. This article is a more-or-less chronological survey of Almgren’s mathematical research. (Excerpts from this article appeared in the December 1997 issue of theNotices of the American Mathematical Society.) Almgren was also an outstanding educator, and he supervised the thesis work of nineteen PhD students; the 1997 volume 6 issue of the journalExperimental Mathematics is dedicated to Almgren and contains reminiscences by two of his PhD students and by various colleagues. A general article about Almgren’s life appeared in the October 1997Notices of the American Mathematical Society [MD]. See [T3]for a brief biography.     

Modal Tense and the Absolutely Unrestricted Quantifier

In this paper, I examine Takashi Yagisawa’s response to van Inwagen’s ontic objection against David Lewis. Van Inwagen criticizes Lewis’s commitment to the absolutely unrestricted sense of ‘there is,’ and Yagisawa claims that by adopting modal tenses he avoids commitment to absolutely unrestricted quantification. I argue that Yagisawa faces a problem parallel to the one Lewis faces. Although Yagisawa officially rejects the absolutely unrestricted sense of a quantifying expression, he is still committed to the absolutely unrestricted sense of ‘is _a real.’     

Cohomology of discrete groups in harmonic cochains on buildings

Modules of harmonic cochains on the Bruhat-Tits building of the projective general linear group over ap-adic field were defined by one of the authors, and were shown to represent the cohomology of Drinfel’d’sp-adic symmetric domain. Here we define certain non-trivial natural extensions of these modules and study their properties. In particular, for a quotient of Drinfel’d’s space by a discrete cocompact group, we are able to define maps between consecutive graded pieces of its de Rham cohomology, which we show to be (essentially) isomorphisms. We believe that these maps are graded versions of the Hyodo-Kato monodromy operatorN.     

Markov Processes on the Adeles and Representations of Euler Products

In this article we construct Markov processes on the ring of adeles. Their transition probabilities are given as solutions of Kolmogorov’s differential equations, and the resulting processes have independent p-components which are p-adic-valued additive processes investigated in the last two decades.     

Existence and consistency of the maximum likelihood estimator for the extreme value index

The paper is about the asymptotic properties of the maximum likelihood estimator for the extreme value index. Under the second order condition, Drees et al. [H. Drees, A. Ferreira, L. de Haan, On maximum likelihood estimation of the extreme value index, Ann. Appl. Probab. 14 (2004) 1179-1201] proved asymptotic normality for any solution of the likelihood equations (with shape parameter γ>−1/2) that is not too far off the real value. But they did not prove that there is a solution of the equations satisfying the restrictions.In this paper, the existence is proved, even for γ>−1. The proof just uses the domain of attraction condition (first order condition), not the second order condition. It is also proved that the estimator is consistent. When the second order condition is valid, following the current proof, the existence of a solution satisfying the restrictions in the above-cited reference is a direct consequence.     

Comparing several population means: a parametric bootstrap method, and its comparison with usual ANOVA F test as well as ANOM

This paper deals with testing the equality of several homoscedastic normal population means. We introduce a newly developed computational approach test (CAT), which is essentially a parametric bootstrap method, and discuss its merits and demerits. In the process of studying the CAT’s usefulness, we compare it with the traditional one-way ANOVA’s F test as well as the analysis of means (ANOM) method. Further, the model robustness of the above three methods have been studied under the ‘t-model’. The motivation behind the proposed CAT is to provide the applied researchers a statistical tool to carry out a comparison of several population means, in a parametric setup, without worrying about the sampling distribution of the inherent test statistic. The CAT can be used to test the equality of several means when the populations are assumed to be heteroscedastic t-distributions.