Human Robot Interaction and Collaboration

Future robots are expected to help us in various tasks, of which many require collaborative effort to be successfully completed. To achieve such behaviours, the robot must be able to physically interact with the human counterpart and predict the intentions of the human counterparts, despite the additional interaction with unpredictable environments. Appropriate robot control methods therefore are essential for the robot to deal with various uncertainties, dynamic aspects, and complex task requirements. A promising direction towards this goal is to adaptively regulate mechanical parameters of the robot via software, based on the task requirements and robot proprioceptive and exteroceptive sensory information. 

In this line of research, we develop novel HRC frameworks considering the ergonomic requirements for the human co-worker. Employing fit-for-workplaces (i.e., wearable and easy-to-use) sensor technologies, the humans' motion and the profile of the interaction forces they exchange with each other, and the environment is monitored. Hence, both the physical and cognitive workload during humans’ occupational activities is estimated by means of some reduced-complexity metrics that we refer to as “ergonomics indexes”, which account for multiple ergonomic risk factors. From the physical standpoint, a whole-body model of the human, inspired from the principles of humanoid robotics, is defined and both kinematic and dynamic variables are computed considering the subject-specific features thus the individual demands of the user. Such model relies on fast and online identification procedures that allow to easily obtain the human body parameters by exploiting intuitive feedback interfaces. Having comprehensive knowledge of the humans’ workload, an optimisation procedure is adopted to estimate a more ergonomic human body behaviour by minimising one or more selected ergonomic indexes according to certain constraints. Subsequently, the worker can be assisted to achieve such an optimal condition by following the guidance of a collaborative robot, thus mitigating the effect of the work-related stress and effort and improving its own productivity.

1. W. Kim, J. Lee, L. Peternel, N. Tsagarakis and A. Ajoudani, "Anticipatory Robot Assistance for the Prevention of Human Static Joint Overloading in Human-Robot Collaboration", IEEE Robotics and Automation Letters (RA-L), 2017. Link

2. L. Peternel, W. Kim, J. Babic, and A. Ajoudani, "Towards Ergonomic Control of Human-Robot Co-Manipulation and Handover", IEEE International Conference on Humanoid Robots (Humanoids), 2017. Link

3. M. Lorenzini, W. Kim, E. De Momi, and A. Ajoudani, "A Synergistic Approach to the Real-time Estimation of the Feet Ground Reaction Forces and Centres of Pressure in Humans with Application to Human-Robot Collaboration", IEEE Robotics and Automation Letters (RA-L), 2018. Link

4. W. Kim, M. Lorenzini, K. Kapicioglu, and A. Ajoudani, "ErgoTac: A Tactile Feedback Interface for Improving Human Ergonomics in Workplaces", IEEE Robotics and Automation Letters (RA-L), 2018. Link

5. A. Ciullo, M. Catalano, A. Bicchi, and A. Ajoudani, "A Supernumerary Soft Robotic Hand-Arm System for Improving Worker Ergonomics", The International Symposium on Wearable Robotics (WeRob), 2018. Link

6. M. Lorenzini, W. Kim, E. De Momi, and A. Ajoudani, "A new overloading fatigue model for ergonomic risk assessment with application to human-robot collaboration. IEEE International Conference on Robotics and Automation (ICRA), 2019. Link

7. W. Kim, M. Lorenzini, P. Balatti, and A. Ajoudani “Towards Ergonomic Control of Collaborative Effort in Multi-human Mobile-robot Teams” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019. Paper - Video

8. M. Lorenzini, F. Fusaro, P. Balatti, E. De Momi, F. Mastrogiovanni, W. Kim, and A. Ajoudani, “Toward a Synergistic Framework for Human-Robot Coexistence and Collaboration (HRC2)” Institute for Robotics and Intelligent Machines Conference (I-RIM), 2019. Paper - Video

9. W. Kim, M. Lorenzini, P. Balatti, P. DH Nguyen, U. Pattacini, V. Tikhanoff, L. Peternel, C. Fantacci, L. Natale, G. Metta, and A. Ajoudani “Adaptable workstations for human-robot collaboration: A reconfigurable framework for improving worker ergonomics and productivity” IEEE Robotics & Automation Magazine (RAM), 2019. Link

10. L. Fortini, M. Lorenzini, W. Kim, E. De Momi, and A. Ajoudani, “A Framework for Real-time and Personalisable Human Ergonomics Monitoring” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020. Link

11. L. Fortini, M. Lorenzini, W. Kim, E. De Momi, and A. Ajoudani, “A Real-time Tool for Human Ergonomics Assessment based on Joint Compressive Forces” IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2020. Link

12. W Kim, L Peternel, M Lorenzini, J Babič, A Ajoudani “A Human-Robot Collaboration Framework for Improving Ergonomics During Dexterous Operation of Power Tools” Robotics and Computer-Integrated Manufacturing (RCIM), 2021. Link

In small and medium-size companies, which are characterized by small-scale production and limited resources, a robotic team is expected to be able to perform a wide variety of tasks. In this context, collaborative robotic work cells allow flexible role interchange and fast adaptation of the cell itself to new products and configurations. However, traditional applications of pHRI involve mainly coexistence of human and robots in the same workspace. To achieve full cooperation and collaboration, agents need to share not only the place but also the goals. In this way, the human co-workers can supervise and complement robot performance with superior expertise and task understanding, assembling proper mixed human-robot teams. This scenario reveals two fundamental problems: how to systematically assign a role to each member of the team to achieve the shared goal, and how to dynamically adapt the robot plan to dynamical changes of role between the team agents and of the surroundings. To this aim, each agent, according to its capabilities, should be able to understand which actions are required to perform a complex task. Moreover, while accomplishing such tasks, robot behaviours should be adapted to the worker intentions and commands and the system should assign the tasks to the agents based on their capabilities and so that the human worker ergonomics is improved.

1. E. Lamon, A. De Franco, L. Peternel, and A. Ajoudani, "A Capability-Aware Role Allocation Approach to Industrial Assembly Tasks," in IEEE Robotics and Automation Letters (RA-L), 2019. Link

2. F Fusaro, E Lamon, E De Momi, and A. Ajoudani, "A Human-Aware Method to Plan Complex Cooperative and Autonomous Tasks using Behavior Trees," in IEEE-RAS International Conference on Humanoid Robotics (Humanoids), 2021.

3. F Fusaro, E Lamon, E De Momi, and A. Ajoudani, "An Integrated Dynamic Method for Allocating Roles and Planning Tasks for Mixed Human-Robot Teams," in IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2021.

An important aspect of robot control in human-robot interaction/collaboration is the presence of the human. The human behaviour is usually highly unpredictable and difficult to model. Indeed, recent work at HRI² laboratory aims at developing appropriate human-robot interfaces to enable the robot to predict or extract the human intention from various feedback modalities (human motor behaviour, vision, interaction force, voice commands, etc.). In addition, other key aspects such as role-allocation in human-robot cooperation and partner’s skill evaluation, which can enable the robot to adapt its behaviour to accommodate for different level of competence and capability of the human will be explored.

1. A. Ajoudani, "Transferring Human Impedance Regulation Skills to Robots", Tracts in Advances Robotics Star, Springer, 2016. Link

2. L. Peternel, N. Tsagarakis, and A. Ajoudani, "Towards Multi-Modal Intention Interfaces for Human-Robot Co-Manipulation", IEEE International Conference on Intelligent Robots and Systems (IROS), 2016. Link

Our work here focuses on the development of methods that allow a robot to adapt its physical behaviour to the human fatigue in human-robot co-manipulation tasks. This can be achieved by the robot initially imitating the human to perform the collaborative task in a leader-follower setting, using a feedback about the human motor behaviour. Simultaneously, the robot can obtain the skill in online manner. When a predetermined level of human fatigue is detected, the robot can use the learnt skill to take over the physically demanding aspect of the task and contributes to a significant reduction of the human effort. The human, on the other hand, will control and supervise the high-level interaction behaviour and will perform the aspects that require the contribution of both agents in such a dynamic co-manipulation setup.  

1. L. Peternel, C. Fang, N. Tsagarakis, and A. Ajoudani, "Online Human Muscle Force Estimation for Fatigue Management in Human-Robot Co-Manipulation", IEEE International Conference on Intelligent Robots and Systems (IROS), 2018. Link

2. L. Peternel, N. Tsagarakis, D. Caldwell and A. Ajoudani, "Robot Adaptation to Human Physical Fatigue in Human-Robot Co-Manipulation", Autonomous Robots (AURO), Special Issue on Learning for Human-Robot Collaboration, 2017. Link