Sticks and STONES may build my bones: Deep learning reconstruction of limb rotations in stick figures

•A method for estimating missing limb rotations in animations of minimal stick figures.•STONES uses an RNN to convert a few sparse 3D joint points to segment rotations.•A RollingSticks database is provided with high quality joint data from motion capture of fitness exercises.•Results better than other ML approaches with average fits ∼98% and RMSE ∼1 ∘.•Improved accuracy compared to video-based state-of-the-art DL methods using only a fraction of the data.