A ToM-based human intention estimation algorithm
The rapid growth of logistics businesses has generated a need for more evolved and efficient warehouse systems. Advances in robotics could help to meet these needs by automating tasks that are currently completed by humans, significantly speeding up the workflow.
To operate safely and efficiently, however, robot warehouse assistants should be able to recognize the intentions of their human operators. Researchers at the University of Zagreb and Karlsruhe Institute of Technology have recently developed a human intention estimation algorithm based on theory of mind (ToM) principles. ToM is the human ability to recognize intent, beliefs, desires, emotions, and mental states in other humans.
“Our research was born out of the EU’s Horizon 2020 project SafeLog, which aims to allow safe entrance to humans into completely automated warehouses, where mobile robots carry racks around, without shutting down the entire fleet of robots,” David Puljiz, one of the researchers who carried out the study, told Prosyscom Tech News. “This is done through the use of a specially designed safety vest and an AR-device to guide and help the worker navigate this environment.”
To enhance security, the warehouse safety system devised by the researchers plans a robot’s path so that it does not clash with that of human workers. For it to work, it needs to know where the human operator is going, particularly if he is not heading to a previously agreed location.
Puljiz and his colleagues observed the motion of individual human workers in warehouse environments and validated them with respect to their goal locations, using an approach called generalized Voronoi diagram based planning. These observations were then fed to a hidden Markov model, which can estimate a worker’s intentions online as he navigates changing environments.
“The proposed ToM based human intention estimation algorithm for flexible robotized warehouses is based on hidden Markov model motion validation,” Tomislav Petkovic, another researcher involved in the study, told TechXplore. “We observe the worker’s motion and validate it with respect to the goal locations using generalized Voronoi diagram based path planning. These observations are then processed by the hidden Markov model framework, which estimates worker intentions online and is capable of handling changing environments. LAMOR lab at the University of Zagreb has developed the algorithm’s backend: motion validation and goal estimation.”
To evaluate the model’s effectiveness, IPR Lab at Karlsruhe Institute of Technology carried out intention estimation experiments in a small warehouse test environment, using Microsoft Hololens AR-glasses. The researchers carried out further evaluations in larger-scale virtual warehouses, using VR technology. They found that their framework could effectively estimate the intentions of warehouse workers, with remarkable precision.
“I think the most meaningful aspect of our study is that AR devices can be used for more than entertainment or marketing and can actually be useful in industrial settings,” Puljiz said. “The research field is rather new and there are many possibilities in this area.”
In the future, the real-time human intention estimation algorithm developed by Puljiz and his colleagues could increase the safety and efficiency of warehouse-based robotic systems. For instance, it could aid the development of more responsive robots that can estimate the intentions of human operators around them, adapting and planning their routes accordingly.
“In future work, we could extend the algorithm to 3D spaces and follow the gaze direction in 3D to determine, for example, if the worker intends to pick the wrong object or if he’s worried about a robot approaching,” Puljiz said. “There are plenty of other possibilities, including hand-tracking integrations.”