The Double-Sided Information Bottleneck Function期刊界 All Journals 搜尽天下杂志传播学术成果专业期刊搜索期刊信息化学术搜索

The Double-Sided Information Bottleneck Function

Authors:	Michael Dikshtein Or Ordentlich Shlomo Shamai (Shitz)

Affiliation:	1.Department of Electrical and Computer Engineering, Technion, Haifa 3200003, Israel;2.School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem 9190401, Israel

Abstract:	A double-sided variant of the information bottleneck method is considered. Let $(X, Y)$ be a bivariate source characterized by a joint pmf $P_{X Y}$ . The problem is to find two independent channels $P_{U \| X}$ and $P_{V \| Y}$ (setting the Markovian structure $U \to X \to Y \to V$ ), that maximize $I (U; V)$ subject to constraints on the relevant mutual information expressions: $I (U; X)$ and $I (V; Y)$ . For jointly Gaussian $X$ and $Y$ , we show that Gaussian channels are optimal in the low-SNR regime but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low and are suboptimal for high correlations. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., $X = Y$ ) and provide supporting numerical evidence. Furthermore, we present a Blahut–Arimoto type alternating maximization algorithm and demonstrate its performance for a representative setting. This problem is closely related to the domain of biclustering.

Keywords:	information bottleneck lossy compression remote source coding biclustering