Table Structure Recognition and Form Parsing by End-to-End Object Detection and Relation Parsing

•A graph neural network (GNN) based unified framework named TSRNet is proposed to jointly detect and recognize the structures of various tables and forms.•GNN is used to classify and group primitive regions into page objects and classify the relationships.•The parameters of all the modules in the system is trained end-to-end to optimize the overall performance•Superior performance has been achieved on table detection, table structure recognition and template-free form parsing.