Under the hood of transformer networks for trajectory forecasting

•First in-depth experimental study of Transformer Networks and BERT for trajectory forecasting.•Comparative evaluation on the modelling of individual human trajectories.•First exhaustive evaluation of input/output representations and problem formulations.•Focus on the capability of Transformers to predict multi-modal futures.•Analysis of the impact of intention and long-term forecasting.