Nonstationary Markov chains and convergence of the annealing algorithm

We study the asymptotic behavior as timet + of certain nonstationary Markov chains, and prove the convergence of the annealing algorithm in Monte Carlo simulations. We find that in the limitt + , a nonstationary Markov chain may exhibit phase transitions. Nonstationary Markov chains in general, and the annealing algorithm in particular, lead to biased estimators for the expectation values of the process. We compute the leading terms in the bias and the variance of the sample-means estimator. We find that the annealing algorithm converges if the temperatureT(t) goes to zero no faster thanC/log(t/t₀) ast+, with a computable constantC andt₀ the initial time. The bias and the variance of the sample-means estimator in the annealing algorithm go to zero likeO(t^–1+) for some 0<1, with =0 only in very special circumstances. Our results concerning the convergence of the annealing algorithm, and the rate of convergence to zero of the bias and the variance of the sample-means estimator, provide a rigorous procedure for choosing the optimal annealing schedule. This optimal choice reflects the competition between two physical effects: (a) The adiabatic effect, whereby if the temperature is loweredtoo abruptly the system may end up not in a ground state but in a nearby metastable state, and (b) the super-cooling effect, whereby if the temperature is loweredtoo slowly the system will indeed approach the ground state(s) but may do so extremely slowly.