Redundant Information Neural Estimation |
| |
| Authors: | Michael Kleinman Alessandro Achille Stefano Soatto Jonathan C Kao |
| |
| Institution: | 1.Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90095, USA;2.Department of Computational and Mathematical Sciences, Caltech, Pasadena, CA 91125, USA;3.Department of Computer Science, University of California, Los Angeles, CA 90095, USA; |
| |
| Abstract: | We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the “redundant information”. We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks. |
| |
| Keywords: | redundant information usable information Partial Information Decomposition |
|
|
