First principles theory for the magnetic and charge instabilities in AV$_3$Sb$_5$ systems

Vanadium-based materials AV$_3$Sb$_5$ (A=K, Rb, Cs) with layered kagome lattice structures have drawn great attention recently due to the discoveries of topologically nontrivial band structures, charge density wave states, giant anomalous Hall effect, as well as unusual superconducting phase at low temperatures. In this work, we theoretically study the magnetic and charge instabilities for this class of materials based on first principles calculations. We develop a method to calculate the generalized susceptibility tensor defined in the sublattice-orbital-spin space, with the effects of Coulomb interactions treated by generalized random phase approximation (RPA). The RPA susceptibility calculations indicate that there are three leading ferromagnetic instability modes at $\Gamma$ point, which are further verified by unrestricted self-consistent Hartree-Fock calculations including both the on-site and inter-site Coulomb interactions. The inclusion of inter-site interactions tend to suppress the spin ferromagnetism due to charge transfer from the V to Sb sites, leading to weak spin magnetic moments $\sim 0.1\mu_{\textrm{B}}$ per V atom with small intrinsic anomalous Hall conductivity. Current loops can be generated in such weak spin ferromagnetic states as a result of spin-orbit coupling effects. The electronic structures in the ferromagnetic states are significantly reconstructed which have nearly compensated electron and hole carriers from two bands. On the other hand, we do not find any diverging instability mode at $M$ point driven by electron-electron Coulomb interactions. First principles phonon calculations indicate that there are unstable phonon modes which tend to drive the system into an inverse star-of-David structure. Our results indicate that there may be separate phase transitions in the magnetic and charge channels in the system.