Topological states in the Hofstadter model on a honeycomb lattice

We provide a detailed analysis of a topological structure of a fermion spectrum in the Hofstadter model with different hopping integrals along the $x,y,z$-links ($t_x=t,t_y=t_z=1$), defined on a honeycomb lattice. We have shown that the chiral gapless edge modes are described in the framework of the generalized Kitaev chain formalism, which makes it possible to calculate the Hall conductance of subbands for different filling and an arbitrary magnetic flux ϕ. At half-filling the gap in the center of the fermion spectrum opens for $t>t_c=2^{\varphi }$, a quantum phase transition in the 2D-topological insulator state is realized at $t_c$. The phase state is characterized by zero energy Majorana states localized at the boundaries. Taking into account the on-site Coulomb repulsion U (where $U<<1$), the criterion for the stability of a topological insulator state is calculated at $t<<1$, $t\sim U$. Thus, in the case of $U>4\mathrm{\Delta }$, the topological insulator state, which is determined by chiral gapless edge modes in the gap Δ, is destroyed.