Chaotic Dynamics in Weight Space of Neural Networks

When a special nonlinear self-feedback term is introduced into the dynamical equation of the backpropagation training algorithm for networks, the dynamics in weight space of networks can become chaotic. Chaotic dynamics of the system can help it escape from the most commonplace local minima of the energy. Simulation on the XOR problem and the prediction of chaotic time series have shown that the proposed chaotic training algorithm can converge to the global minimum or its approximate solutions efficiently and dramatically faster than the original backpropagation training algorithm.