Variable Selection Diagnostics Measures for High-Dimensional Regression

Markov chain Monte Carlo (MCMC) is nowadays a standard approach to numerical computation of integrals of the posterior density π of the parameter vector η. Unfortunately, Bayesian inference using MCMC is computationally intractable when the posterior density π is expensive to evaluate. In many such problems, it is possible to identify a minimal subvector β of η responsible for the expensive computation in the evaluation of π. We propose two approaches, DOSKA and INDA, that approximate π by interpolation in ways that exploit this computational structure to mitigate the curse of dimensionality. DOSKA interpolates π directly while INDA interpolates π indirectly by interpolating functions, for example, a regression function, upon which π depends. Our primary contribution is derivation of a Gaussian processes interpolant that provably improves over some of the existing approaches by reducing the effective dimension of the interpolation problem from dim(η) to dim(β). This allows a dramatic reduction of the number of expensive evaluations necessary to construct an accurate approximation of π when dim(η) is high but dim(β) is low.

We illustrate the proposed approaches in a case study for a spatio-temporal linear model for air pollution data in the greater Boston area.

Supplemental materials include proofs, details, and software implementation of the proposed procedures.     

The Block Criterion for Multiscale Inference About a Density,With Applications to Other Multiscale Problems

The use of multiscale statistics, that is, the simultaneous inference about various stretches of data via multiple localized statistics, is a natural and popular method for inference about, for example, local qualitative characteristics of a regression function, a density, or its hazard rate. We focus on the problem of providing simultaneous confidence statements for the existence of local increases and decreases of a density and address several statistical and computational issues concerning such multiscale statistics. We first review the benefits of employing scale-dependent critical values for multiscale statistics and then derive an approximation scheme that results in a fast algorithm while preserving statistical optimality properties. The main contribution is a methodology for calibrating multiscale statistics that does not require a case-by-case derivation of its specific form. We show that in the above density context the methodology possesses statistical optimality properties and allows for a fast algorithm. We illustrate the methodology with two further examples: a multiscale statistic introduced by Gijbels and Heckman for inference about a hazard rate and local rank tests introduced by Dümbgen for inference in nonparametric regression.

Code for the density application is available as the R package modehunt on CRAN. Additional code to compute critical values, reproduce the hazard rate and local rank example and the plots in the paper as well as datasets containing simulation results and an appendix with all the proofs of the theorems are available online as supplemental material.     

Convex Modeling of Interactions With Strong Heredity

We consider the task of fitting a regression model involving interactions among a potentially large set of covariates, in which we wish to enforce strong heredity. We propose FAMILY, a very general framework for this task. Our proposal is a generalization of several existing methods, such as VANISH, hierNet, the all-pairs lasso, and the lasso using only main effects. It can be formulated as the solution to a convex optimization problem, which we solve using an efficient alternating directions method of multipliers (ADMM) algorithm. This algorithm has guaranteed convergence to the global optimum, can be easily specialized to any convex penalty function of interest, and allows for a straightforward extension to the setting of generalized linear models. We derive an unbiased estimator of the degrees of freedom of FAMILY, and explore its performance in a simulation study and on an HIV sequence dataset. Supplementary materials for this article are available online.     

On Skew Power Serieswise Armendariz Rings

For a ring endomorphism α, we introduce and investigate SPA-rings which are a generalization of α-rigid rings and determine the radicals of the skew polynomial rings R[x; α], R[x, x ^?1; α] and the skew power series rings R[[x; α]], R[[x, x ^?1; α]], in terms of those of R. We prove that several properties transfer between R and the extensions, in case R is an SPA-ring. We will construct various types of nonreduced SPA-rings and show SPA is a strictly stronger condition than α-rigid.     

Ergodicity of Combocontinuous Adaptive MCMC Algorithms

This paper proves convergence to stationarity of certain adaptive MCMC algorithms, under certain assumptions including easily-verifiable upper and lower bounds on the transition densities and a continuous target density. In particular, the transition and proposal densities are not required to be continuous, thus improving on the previous ergodicity results of Craiu et al. (Ann Appl Probab 25(6):3592–3623, 2015).     

Approximation properties in tensor products

For distinct classes of locally convex spaces and tensor topologies α =? and α = π it is proved that \(E\hat \otimes _\alpha F\) has the approximation property if and only if E and F have this property.     

Comparative study and sensitivity analysis of skewed spatial processes

Asymmetric spatial processes arise naturally in finance, economics, hydrology and ecology. For such processes, two different classes of models are considered in this paper. One of them, proposed by Majumdar and Paul (J Comput Graph Stat 25(3):727–747, 2016), is the Double Zero Expectile Normal (DZEXPN) process and the other is a version of the “skewed normal process”, proposed by Minozzo and Ferracuti (Chil J Stat 3:157–170, 2012), with closed skew normal multivariate marginal distributions. Both spatial models have useful properties in the sense that they are ergodic and stationary. As a brief treatise to test the sensitivity and flexibility of the new proposed DZEXPN model (Majumdar and Paul in J Comput Graph Stat 25(3):727–747, 2016), in relation to other skewed spatial processes in the literature using a Bayesian methodology, our results show that by adding measurement error to the DZEXPN model, a reasonably flexible model is obtained, which is also computationally tractable than many others mentioned in the literature. Meanwhile, we develop a full-fledged Bayesian methodology for the estimation and prediction of the skew normal process proposed in Minozzo and Ferracuti (Chil J Stat 3:157–170, 2012). Specifically, a hierarchical model is used to describe the skew normal process and a computationally efficient MCMC scheme is employed to obtain samples from the posterior distributions. Under a Bayesian paradigm, we compare the performances of the aforementioned three different spatial processes and study their sensitivity and robustness based on simulated examples. We further apply them to a skewed data set on maximum annual temperature obtained from weather stations in Louisiana and Texas.     

The Set of Packing and Covering Densities of Convex Disks

For every convex disk $K$ (a convex compact subset of the plane, with non-void interior), the packing density $\delta (K)$ and covering density ${\vartheta (K)}$ form an ordered pair of real numbers, i.e., a point in $\mathbb{R }^2$ . The set $\varOmega $ consisting of points assigned this way to all convex disks is the subject of this article. A few known inequalities on $\delta (K)$ and ${\vartheta (K)}$ jointly outline a relatively small convex polygon $P$ that contains $\varOmega $ , while the exact shape of $\varOmega $ remains a mystery. Here we describe explicitly a leaf-shaped convex region $\Lambda $ contained in $\varOmega $ and occupying a good portion of $P$ . The sets $\varOmega _T$ and $\varOmega _L$ of translational packing and covering densities and lattice packing and covering densities are defined similarly, restricting the allowed arrangements of $K$ to translated copies or lattice arrangements, respectively. Due to affine invariance of the translative and lattice density functions, the sets $\varOmega _T$ and $\varOmega _L$ are compact. Furthermore, the sets $\varOmega , \,\varOmega _T$ and $\varOmega _L$ contain the subsets $\varOmega ^\star , \,\varOmega _T^\star $ and $\varOmega _L^\star $ respectively, corresponding to the centrally symmetric convex disks $K$ , and our leaf $\Lambda $ is contained in each of $\varOmega ^\star , \,\varOmega _T^\star $ and $\varOmega _L^\star $ .     

Model selection for density estimation with \mathbb L _2-loss

We consider here estimation of an unknown probability density $s$ belonging to $\mathbb L _2(\mu )$ where $\mu $ is a probability measure. We have at hand $n$ i.i.d. observations with density $s$ and use the squared $\mathbb L _2$ -norm as our loss function. The purpose of this paper is to provide an abstract but completely general method for estimating $s$ by model selection, allowing to handle arbitrary families of finite-dimensional (possibly non-linear) models and any $s\in \mathbb L _2(\mu )$ . We shall, in particular, consider the cases of unbounded densities and bounded densities with unknown $\mathbb L _\infty $ -norm and investigate how the $\mathbb L _\infty $ -norm of $s$ may influence the risk. We shall also provide applications to adaptive estimation and aggregation of preliminary estimators. One major technical tool of our approach is a proof of the existence of suitable tests between $\mathbb L _2$ -balls with centers belonging to $\mathbb L _\infty $ . Although of a purely theoretical nature, our method leads to results that cannot presently be reached by more concrete ones.     

Exact and approximate EM estimation of mutually exciting hawkes processes

Motivated by the availability of continuous event sequences that trace the social behavior in a population e.g. email, we believe that mutually exciting Hawkes processes provide a realistic and informative model for these sequences. For complex mutually exciting processes, the numerical optimization used for univariate self exciting processes may not provide stable estimates. Furthermore, convergence can be exceedingly slow, making estimation computationally expensive and multiple random restarts doubly so. We derive an expectation maximization algorithm for maximum likelihood estimation mutually exciting processes that is faster, more robust, and less biased than estimation based on numerical optimization. For an exponentially decaying excitement function, each EM step can be computed in a single $O(N)$ pass through the data, for $N$ observations, without requiring the entire dataset to be in memory. More generally, exact inference is $\Theta (N^{2})$ , but we identify some simple $\Theta (N)$ approximation strategies that seem to provide good estimates while reducing the computational cost.     

The Primitive Permutation Groups of Degree Less Than 4096

In this article we use the Classification of the Finite Simple Groups, the O'Nan–Scott Theorem, and Aschbacher's theorem to classify the primitive permutation groups of degree less than 4096. The results will be added to the primitive groups databases of GAP and MAGMA.     

Linear-Size Approximations to the Vietoris–Rips Filtration

The Vietoris–Rips filtration is a versatile tool in topological data analysis. It is a sequence of simplicial complexes built on a metric space to add topological structure to an otherwise disconnected set of points. It is widely used because it encodes useful information about the topology of the underlying metric space. This information is often extracted from its so-called persistence diagram. Unfortunately, this filtration is often too large to construct in full. We show how to construct an $O(n)$ O ( n ) -size filtered simplicial complex on an $n$ n -point metric space such that its persistence diagram is a good approximation to that of the Vietoris–Rips filtration. This new filtration can be constructed in $O(n\log n)$ O ( n log n ) time. The constant factors in both the size and the running time depend only on the doubling dimension of the metric space and the desired tightness of the approximation. For the first time, this makes it computationally tractable to approximate the persistence diagram of the Vietoris–Rips filtration across all scales for large data sets. We describe two different sparse filtrations. The first is a zigzag filtration that removes points as the scale increases. The second is a (non-zigzag) filtration that yields the same persistence diagram. Both methods are based on a hierarchical net-tree and yield the same guarantees.     

Characterizing moore groups by tensor products of irreducible representations

It is shown that all irreducible representations of a σ-compact Lie group G have to be finite dimensional provided that for every π in the reduced dual of G the tensor product π ? \(\bar \pi \) has a discrete support.     

Remarks on exceptional points and differentiation bases

In spite of the Lebesgue density theorem, there is a positive \({\delta}\) such that, for every measurable set \({A \subset \mathbb{R}}\) with \({\lambda (A) > 0}\) and \({\lambda (\mathbb{R} \setminus A) > 0}\), there is a point at which both the lower densities of \({A}\) and of the complement of \({A}\) are at least \({\delta}\). The problem of determining the supremum of possible values of this \({\delta}\) was studied by V. I. Kolyada, A. Szenes and others. It seems that the authors considered this quantity a feature of density. We show that it is connected rather with a choice of a differentiation basis.     

A Graphical Goodness-of-Fit Test for Dependence Models in Higher Dimensions

This article introduces a graphical goodness-of-fit test for copulas in more than two dimensions. The test is based on pairs of variables and can thus be interpreted as a first-order approximation of the underlying dependence structure. The idea is to first transform pairs of data columns with the Rosenblatt transform to bivariate standard uniform distributions under the null hypothesis. This hypothesis can be graphically tested with a matrix of bivariate scatterplots, Q-Q plots, or other transformations. Furthermore, additional information can be encoded as background color, such as measures of association or (approximate) p-values of tests of independence. The proposed goodness-of-fit test is designed as a basic graphical tool for detecting deviations from a postulated, possibly high-dimensional, dependence model. Various examples are given and the methodology is applied to a financial dataset. An implementation is provided by the R package copula. Supplementary material for this article is available online, which provides the R package copula and reproduces all the graphical results of this article.     

Automorphism Group of q-Quantum Torus Lie Algebra with q a Root of Unity*

Let α be an automorphism of a ring R. We study the skew Armendariz of Laurent series type rings (α-LA rings), as a generalization of the standard Armendariz condition from polynomials to skew Laurent series. We study on the relationship between the Baerness and p.p. property of a ring R and these of the skew Laurent series ring R[[x, x ^?1; α]], in case R is an α-LA ring. Moreover, we prove that for an α-weakly rigid ring R, R[[x, x ^?1; α]] is a left p.q.-Baer ring if and only if R is left p.q.-Baer and every countable subset of S _?(R) has a generalized countable join in R. Various types of examples of α-LA rings are provided.     

The Number of Singular Vector Tuples and Uniqueness of Best Rank-One Approximation of Tensors

In this paper we discuss the notion of singular vector tuples of a complex-valued \(d\) -mode tensor of dimension \(m_1\times \cdots \times m_d\) . We show that a generic tensor has a finite number of singular vector tuples, viewed as points in the corresponding Segre product. We give the formula for the number of singular vector tuples. We show similar results for tensors with partial symmetry. We give analogous results for the homogeneous pencil eigenvalue problem for cubic tensors, i.e., \(m_1=\cdots =m_d\) . We show the uniqueness of best approximations for almost all real tensors in the following cases: rank-one approximation; rank-one approximation for partially symmetric tensors (this approximation is also partially symmetric); rank- \((r_1,\ldots ,r_d)\) approximation for \(d\) -mode tensors.     

Existence of Isoperimetric Sets with Densities “Converging from Below” on $${\mathbb {R}}^N$$

In this paper, we consider the isoperimetric problem in the space \({\mathbb {R}}^N\) with a density. Our result states that, if the density f is lower semi-continuous and converges to a limit \(a>0\) at infinity, with \(f\le a\) far from the origin, then isoperimetric sets exist for all volumes. Several known results or counterexamples show that the present result is essentially sharp. The special case of our result for radial and increasing densities positively answers a conjecture of Morgan and Pratelli (Ann Glob Anal Geom 43(4):331–365, 2013.     

Proof of an entropy conjecture for Bloch coherent spin states and its generalizations

Wehrl used Glauber coherent states to define a map from quantum density matrices to classical phase space densities and conjectured that for Glauber coherent states the mininimum classical entropy would occur for density matrices equal to projectors onto coherent states. This was proved by Lieb in 1978 who also extended the conjecture to Bloch SU(2) spin-coherent states for every angular momentum J. This conjecture is proved here. We also recall our 1991 extension of the Wehrl map to a quantum channel from J to ${K=J+\frac{1}{2}, J+1, ... ,}$ with ${K=\infty}$ corresponding to the Wehrl map to classical densities. These channels were later recognized as the optimal quantum cloning channels. For each J and ${J < K \leqslant \infty}$ we show that the minimal output entropy for the channels occurs for a J coherent state. We also show that coherent states both Glauber and Bloch minimize any concave functional, not just entropy.