A stabilized multigrid solver for hyperelastic image registration

Image registration is a central problem in a variety of areas involving imaging techniques and is known to be challenging and ill‐posed. Regularization functionals based on hyperelasticity provide a powerful mechanism for limiting the ill‐posedness. A key feature of hyperelastic image registration approaches is their ability to model large deformations while guaranteeing their invertibility, which is crucial in many applications. To ensure that numerical solutions satisfy this requirement, we discretize the variational problem using piecewise linear finite elements, and then solve the discrete optimization problem using the Gauss–Newton method. In this work, we focus on computational challenges arising in approximately solving the Hessian system. We show that the Hessian is a discretization of a strongly coupled system of partial differential equations whose coefficients can be severely inhomogeneous. Motivated by a local Fourier analysis, we stabilize the system by thresholding the coefficients. We propose a Galerkin‐multigrid scheme with a collective pointwise smoother. We demonstrate the accuracy and effectiveness of the proposed scheme, first on a two‐dimensional problem of a moderate size and then on a large‐scale real‐world application with almost 9 million degrees of freedom.