Noncollinear magnetic order, in-plane anisotropy, and magnetoelectric coupling in the pyroelectric honeycomb antiferromagnet Ni2Mo3O8

Ni2Mo3O8 is a pyroelectric honeycomb antiferromagnet exhibiting peculiar changes in its electric polarization at magnetic transitions. Ni2Mo3O8 stands out from isostructural magnetic compounds, showing an anomalously low magnetic transition temperature and unique magnetic anisotropy. We determine the magnetic structure of Ni2Mo3O8 utilizing high-resolution powder and single-crystal neutron diffraction. A noncollinear stripy antiferromagnetic order is found in the honeycomb planes. The magnetic space group is PCna21. The in-plane magnetic connection is of the stripy type for both the ab-plane and c-axis spin components. This is a simpler connection than the one proposed previously. The ferromagnetic interlayer order of the c-axis spin components in our model is also distinct. The magnetic anisotropy of Ni2Mo3O8 is characterized by orientation-dependent magnetic susceptibility measurements on a single crystal, consistent with neutron diffraction analysis. The local magnetoelectric tensor analysis using our magnetic models provides insights into its magnetoelectric coupling and polarization. Thus, our results deliver essential information for understanding both the unusual magnetoelectric properties of Ni2Mo3O8 and the prospects for observing exotic nonreciprocal, Hall, and magnonic effects characteristic of this compound family.