A Digital Twin of an Off Highway Vehicle based on a Low Cost Camera

The operation of heavy equipment in the construction industry requires high-skilled workers: they must safely operate the equipment in an efficient manner to avoid project delays or damages, both to equipment and material as to their colleagues and themselves. Operator productivity and safety are therefore critical in the construction industry. The advancements in technologies such as digitization and automation are more and more applied in the industry, leading to more data and knowledge on the construction equipment and environment. This can be leveraged by a digital twin to provide value for the operator.In this work, we present a digital twin architecture applied to an off highway vehicle, which provides real-time feedback to the operator while operating the vehicle. Monitoring the interactions between an off highway vehicle and workers will result in a safer and more productive environment. This work focuses on detection of the environment and monitoring the state of the vehicle. Images of a low cost camera are used to extract information of the vehicle implement as well as the environment. The motion and state of the machine can be realistically visualized in real-time in Unreal, using a 3D CAD (Computer-Aided Design) geometry model, enriched with additional relevant information and color schemes. A modular pub-sub architecture enables the communication between the sensors and camera, the data processing and the visualization of the digital twin.