Anti-Self-Dual Instantons with Lagrangian Boundary Conditions I: Elliptic Theory

We study nonlocal Lagrangian boundary conditions for anti-self-dual instantons on 4-manifolds with a space-time splitting of the boundary. We establish the basic regularity and compactness properties (assuming L^p-bounds on the curvature for p>2) as well as the Fredholm theory in a compact model case. The motivation for studying this boundary value problem lies in the construction of an instanton Floer homology for 3-manifolds with boundary. The present paper is part of a program proposed by Salamon for the proof of the Atiyah-Floer conjecture for homology-3-spheres.Acknowledgement I would like to thank Dietmar Salamon for his constant help and encouragement in pursuing this project. Part of this research was supported by the Swiss National Science Foundation.