Estimating the errors on measured entropy and mutual information

Information entropy and the related quantity mutual information are used extensively as measures of complexity and to identify nonlinearity in dynamical systems. Expressions for the probability distribution of entropies and mutual informations calculated from finite amounts of data exist in the literature but the expressions have seldom been used in the field of nonlinear dynamics. In this paper formulae for estimating the errors on observed information entropies and mutual informations are derived using the standard error analysis familiar to physicists. Their validity is demonstrated by numerical experiment. For illustration the formulae are then used to evaluate the errors on the time-lagged mutual information of the logistic map.