Nonlinear neural networks. II. Information processing

Information processing in nonlinear neural networks with a finite numberq of stored patterns is studied. Each network is characterized completely by its synaptic kernelQ. At low temperatures, the nonlinearity typically results in 2^q–2–q metastable, pure states in addition to theq retrieval states that are associated with theq stored patterns. These spurious states start appearing at a temperature , which depends onq. We give sufficient conditions to guarantee that the retrieval states bifurcate first at a critical temperatureT _c and that /T _c 0 asq. Hence, there is a large temperature range whereonly the retrieval states and certain symmetric mixtures thereof exist. The latter are unstable, as they appear atT _c . For clipped synapses, the bifurcation and stability structure is analyzed in detail and shown to approach that of the (linear) Hopfield model asq. We also investigate memories that forget and indicate how forgetfulness can be explained in terms of the eigenvalue spectrum of the synaptic kernelQ.