One-shot learning based pattern transition map for action early recognition.

•Due to the difficulty of pattern separation and definition in large quantity of action sequences, we employ one-shot learning to automatically define patterns using a few pattern samples.•Using reinforcement learning, we learn pattern transition maps which include temporal ordered pattern and their transition relationships in action sequences.•We propose a pattern transition map based soft-regression approach for action early recognition.•We evaluate the proposed approach on the MSR Action Pairs Database and the SYSU 3DHOI database. Experiments show that our approach recognizes ongoing sequences with high accuracies.