Dynamics of simple electronic neural networks

We examine systems of one and two nonlinear threshold switching elements (“neurons”), of the kind used in electronic neural networks. Characteristics of these systems which deviate from standard ideal models are found to induce complex dynamics. When the neurons possess a finite frequency response or a transfer characteristic with a time delay, underdamped transients and instability leading to oscillation can occur. Inertia in the neuron connections is found to cause ringing about fixed points, convoluted basin boundaries, instability and spontaneous oscillation, and chaotic behavior when driven. Furthermore, the collective behavior of a network of multiple neurons can be underdamped even when the individual connections are overdamped. These results imply that care should be exercised in implementing networks with electronic devices or when adding inertia to enhance the performance of optimizing networks.