Layer-Number-Dependent Antiferromagnetic and Ferromagnetic Behavior in MnSb2Te4

MnBi2Te4, an intrinsic magnetic topological insulator, has shown layer-number-correlated magnetic and topological phases. More interestingly, in the isostructural material MnSb2Te4, the antiferromagnetic (AFM) and ferromagnetic (FM) states have been both observed in the bulk counterparts, which are also predicted to be topologically nontrivial. Revealing the layer-number-dependent magnetic properties of MnSb2Te4 down to a single septuple layer (SL) is of great significance for exploring the topological phenomena. However, this is still elusive. Here, using the polar reflective magnetic circular dichroism spectroscopy, both the A-type AFM and FM behaviors are observed and comprehensively studied in MnSb2Te4 down to a single SL limit. In A-type AFM MnSb2Te4 flakes, an obvious odd-even layer-number effect is observed. An additional surface spin-flop (SSF) transition occurs in even-SL flakes with the number of layers larger than 2. With the AFM linear-chain model, we identify that the even-SL flakes stabilize in a collinear state between the SSF transition and the spin-flop transition due to their appropriate energy ratio between the magnetic-field-scale anisotropy and interlayer interaction. In FM MnSb2Te4 flakes, we observe very different magnetic behaviors with an abrupt spin-flipping transition and very small saturation fields, indicating a weakened interlayer interaction. By revealing the rich magnetic states of few-SL MnSb2Te4 on the parameter space of the number of layers, external magnetic field, and temperature, our findings pave the way for further quantum transport studies of few-SL MnSb2Te4.