Structural instability and charge modulations in the kagome superconductor AV(3)Sb(5)

Recently, both charge density wave (CDW) and superconductivity have been observed in the kagome compounds AV3Sb5. However, the nature of the CDW that results in many novel charge modulations is still under hot debate. Based on the first-principles calculations, we discover two kinds of CDW states in AV3Sb5, namely, the trimerized-hexamerized 2x2 phase and the dimerized 4x1 phase. Our phonon spectrum and electronic Lindhard function calculations reveal that the most intensive structural instability in AV3Sb5 originates from a combined in-plane vibration mode of V atoms through the electron-phonon coupling, rather than the Fermi surface nesting effect. More importantly, a metastable 4x1 phase with the V-V dimer pattern and twofold symmetric bow-tie-shaped charge modulation is revealed in CsV3Sb5. Further analyses demonstrate that both the in-plane phonon instability and c direction interaction play important roles during the 4x1 phase formation, which provides key understanding for the mechanism of the 4x1 CDW and relative novel phenomena in