Mixed reality-based active Hazard prevention system for heavy machinery operators

The use of heavy machinery in construction has significantly improved efficiency but poses a risk of serious accidents when not operated safely. While warning systems can mitigate accidents from conscious operator errors, addressing risky behaviors stemming from unconscious operators remains a challenge, with mixed reality (MR) research in the construction industry often lacking focus on operator-related aspects. This paper describes a monitoring method for heavy machinery operators is proposed. It identifies whether an operator is inadvertently engaging in behavior that could lead to danger when they are not in good shape. This method uses MR devices worn by the operator to analyze the state of their eyes, head, and hands to determine whether the operator is currently capable of safely operating the machine. Real-time judgment is achieved by classifying specific movements and abnormal behaviors in advance, further reducing the occurrence of accidents.