Deep Learning Application in Broadcast Tennis Video Annotation

We are in the era that sport is increasingly defined by data. Rich data is a powerful enabling foundation for novel insights on the game and on player actions, and consequently used for fan engagement and better decision making. Detailed data that has specifications of each shot is currently missing in tennis, as the game dynamics is fast and it is beyond human ability to manually record all specifications of each and every shot, even using scoring software. In this paper, we present an intelligent system to automatically recognize player actions, ball and player movements, and important game events. The system annotates the video with a suitable set of keywords for fast retrieval in broadcast production. Various techniques of computer vision, alignment, filtering, and pre-trained deep learning models are utilized by our system. The evaluation of our results on multiple broadcast videos show great accuracy and timeliness. The implications of the work presented in this paper are profound in the current workflow in broadcast coverage of a tennis match where normally multiple video operators and judges are needed to identify events and retrieve the related clips from multiple cameras.