Semidirect parallel self-consistent field: the load balancing problem in the input/output intensive self-consistent field iterations

The full capacity of contemporary parallel computers can, in the context of iterative ab initio procedures like, for example, self-consistent field (SCF) and multiconfigurational SCF, only be utilized if the disk and input/output (I/O) capacity are fully exploited before the implementation turns to an integral direct strategy. In a recent report on parallel semidirect SCF http://www.tc.cornell.edu/er/media/1996/collabrate.html, http://www.fp.mcs.anl.gd/grand-challenges/chem/nondirect/index.html it was demonstrated that super-linear speedups are achievable for algorithms that exploit scalable parallel I/O. In the I/O-intensive SCF iterations of this implementation a static load balancing, however, was employed, dictated by the initial iteration in which integral evaluation dominates the central processing unit activity and thus determines the load balancing. In the present paper we present the first implementation in which load balancing is achieved throughout the whole SCF procedure, i.e. also in subsequent iterations. The improved scalability of our new algorithm is demonstrated in some test calculations, for example, for 63-node calculation a speedup of 104 was observed in the computation of the two-electron integral contribution to the Fock matrix.Contribution to the Björn Roos Honorary Issue Acknowledgement.We thank J. Nieplocha for valuable help and making the toolkit (including ChemIO) available to us. R.L. acknowledges the Intelligent Modeling Laboratory and the University of Tokyo for financial support during his stay in Japan.