Frustrated magnetic interactions in FeSe

The structurally simplest high-temperature superconductor FeSe exhibits an intriguing superconducting nematic paramagnetic phase with unusual spin excitation spectra that are different from typical spin waves; thus, determining its effective magnetic exchange interactions is challenging. Here we report neutron scattering measurements of spin fluctuations of FeSe in the tetragonal paramagnetic phase. We show that the equal-time magnetic structure factor, S(Q), can be effectively modeled using the self-consistent Gaussian approximation calculation with highly frustrated nearest-neighbor (J1) and next-nearest-neighbor (J2) exchange couplings, and very weak further neighbor exchange interaction. Our results elucidate the frustrated magnetism in FeSe, which provides a natural explanation for the highly tunable superconductivity and nematicity in FeSe and related materials.