A framework for robotic arm pose estimation and movement prediction based on deep and extreme learning models

Human-robot collaboration has gained a notable prominence in Industry 4.0, as the use of collaborative robots increases efficiency and productivity in the automation process. However, it is necessary to consider the use of mechanisms that increase security in these environments, as the literature reports that risk situations may exist in the context of human-robot collaboration. One of the strategies that can be adopted is the visual recognition of the collaboration environment using machine learning techniques, which can automatically identify what is happening in the scene and what may happen in the future. In this work, we are proposing a new framework that is capable of detecting robotic arm keypoints commonly used in Industry 4.0. In addition to detecting, the proposed framework is able to predict the future movement of these robotic arms, thus providing relevant information that can be considered in the recognition of the human-robot collaboration scenario. The proposed framework has two main modules. The first one contains a convolutional neural network based on self-calibrated convolutions enabling better discriminative feature extraction and the support of extreme learning machine neural networks with different kernels for predicting robotic arm keypoints. The second module is composed of deep recurrent learning models, such as long short-term memory and gated recurrent unit. These models are able to predict future robotic arm keypoints. All experiments were evaluated using the mean squared error metric. Results show that the proposed framework is capable of detecting and predicting with low error, contributing to the mitigation of risks in human-robot collaboration. In addition, it was possible to verify that the use of convolutional neural networks in conjunction with extreme learning machines can offer a lower detection error in a regression task (e.g., keypoint detection), something that, as far as the authors are aware of, is not yet known, nor had been evaluated previously in the literature.