Transverse dynamical response of laser frequency-shifted feedback with mode mismatch

•A transverse dynamical response model of LFSF is proposed considering mode-mismatch.•By decomposing the echo wave to the eigenmodes of the laser, a quantitative calculation method of the effective reflectivity and mode-matching factor is presented.•Accurate calculations of the effective reflectivity and mode-matching factor enable precise description of LFSF response behavior and exploration of optimal coupling strategies.•A simple scheme is proposed to calibrate the effective reflectivity of scattering surfaces by employing an optimal coupling target-cat's eye retroreflector.•This model promises an effective theoretical basis to make full use of the high sensitivity of LFSF and apply it in a wide range.