Prediction of Human Action from Hand Movement for Human-Robot Collaboration

Summary

In today's era, there has been rapid development in the fields of robotics and artificial intelligence, resulting in an increased use of robots in various aspects of human daily life. In order for robots to be used in everyday tasks, they must be able to handle unstructured, unpredictable, and constantly changing environments. Therefore, it is necessary for them to act autonomously, learn how to react to various changes in the environment, and understand the consequences of their actions on the environment. This thesis examines human-robot collaboration in a shared workspace. Specifically, to achieve better and more natural collaboration, a key requirement is for robots to predict human actions. This is supported by the observation that the initial movement of a person contains useful information capable of predicting their final action. For example, the posture of the palm changes at the beginning of a movement depending on the size of the object the person is about to grasp.

In the context of this work, appropriate data were collected and a suitable Artificial Intelligence method was sought that would allow a robot to predict how a human will act in a shared workspace based on visual information (camera) from the hand and palm movements. Initially, the collected data concerned the movements of various people who aimed to grasp and move three different-sized objects. From this data, two datasets were created. The first dataset contained three-dimensional information, and the second contained two-dimensional information for each movement. For the second dataset, in addition to the three classes, one for each object, the prediction was also tested between two classes for all possible combinations of the three classes. After processing the data to clean the noise and delimit the movement section under study, machine learning algorithms were applied to the processed data. The five machine learning algorithms used were: SVM, Decision Tree, Random Forest, Extra Tree, and Gradient Boosting. The evaluation process of the results was done through the K-Fold Cross-validation method. The results showed that the best algorithm, with a success rate of 94%, was Gradient Boosting for the two-class two-dimensional dataset, consisting of the small and large object classes.