LiveAE: Attention-based and Edge-assisted Viewport Prediction for Live 360 Video Streaming

Viewport prediction plays a crucial role in live 360 degrees video streaming as it determines which tiles should be prefetched in high quality, thereby significantly impacting the user experience. However, the current approach to viewport prediction, which integrates content-level visual features with the viewer's head movement trajectory, faces the challenge of striking a balance between prediction accuracy and computational complexity. In this paper, we propose LiveAE, a novel attention-based and edge-assisted viewport prediction framework for live 360 degrees video streaming. Specifically, we employ a pre-trained video encoder called Vision Transformer (ViT) for general visual feature extraction and a cross-attention mechanism for user-specific interest tracking. To address the computational complexity issue, we offload the aforementioned content-level operations to an edge server while retaining trajectory-related functions on the client side. Extensive experiments show that our proposed method not only outperforms state-of-the-art algorithms but also ensures the real-time requirements of live 360 degrees video streaming.