IROS Workshop 2019

2nd Full-day Workshop on Progress in Ergonomic Physical Human-Robot Collaboration

8th of November, 2019 (Room LG-R14)

Work-related musculoskeletal disorder (WMSD) and injuries are the largest categories of work-related risk factors in many industries and are associated with very large loss of productivity as regarding the aspect of economic sustainability. Presently various ergonomic assessment methods are implemented to prevent such risks and to provide a better working condition for the workers. Some of these measures include replacing humans with fully autonomous robots that typically account for the worker’s safety by through physical isolation or collision avoidance. However the recent trends in industry focus on introducing a new technological concept, where robots are expected to adaptively collaborate with the worker and physically interact with them. In such highly flexible and changing scenarios, it is difficult to pre-plan for ergonomic conditions of the worker. Therefore it is crucial that the the collaborative robot is aware of the co-worker’s states in order to be able to reconfigure the working conditions in a way the human ergonomics will be accounted for.

The above-mentioned problem presents a novel challenging research topic for the community. We held a workshop at ICRA 2018 to introduce the concept and discuss the state of the art that could help forming the foundations for the new field. The conclusion of the workshop was that this is a relevant problem and research should be done to develop novel methods that can solve it.

The proposed workshop will first follow up by reviewing the progress of the research that was initiated in the previous workshop. Then, we will focus on how to incorporate known ergonomic factors into the concept of human-robot collaboration. In addition, we will also put attention on discussing potential novel ergonomic factors that can be used within this framework. We will examine novel control strategies and interfaces for adaptive behaviour of collaborative robots that can facilitate the desired ergonomic conditions. Finally, we will discuss and design appropriate validation methods that can be used for objective benchmarking within the field.

In summary, the workshop goal is to bring together researchers, industry engineers, human factors researchers and medical doctors of different backgrounds and provide an opportunity to discuss and solve challenges related to developing the concept of ergonomic human-robot collaboration.

The questions we want to address

1. ---

   How to clearly define the measures for ergonomics in human-robot collaboration?
2. How to design the appropriate optimization that the robot controller can use?
3. Are the proposed ergonomic measures sufficient to ensure ergonomic conditions and prevent work-related injuries in terms of bio-medical aspects?
4. How to improve the existing modelling of the human and effectively apply it to achieve ergonomic human-robot collaboration?
5. Learning algorithms may help the robot to overcome some of the above-mentioned issues.
6. What kind of additional sensory feedback systems are required?
7. How can robot learning methods be utilized to this end? What are the challenges in achieving the proposed goal in different applications (i.e. industrial human-robot collaboration, wearable robots, service robots, etc.)?

Interests

Physical Human-Robot Collaboration, Human Modelling, Physical Interaction Control, Adaptation and Learning, Industrial Robots, Exoskeleton Robots, Wearable Sensors, Feedback Devices, Shared Control.

Conclusions

• model complexity is almost always advantageous for ergonomic PHRI if it supports the application (e.g., real-time computation)
• hardware is normally designed to be physically comfortable but we need to make sure it will be comfortable even when it is controlled and used for prolonged period of time
• human intention recognition and various feedback sensory systems are important for robot to perform correct actions in real-time
• the goal is to minimize or eliminate sensors on the human, however at the stage it is still valuable to explore with wearable complex sensory system
• the timing of action is important to avoid awkward situations
• some human workers may feel threaten by the robot’s intelligence and adaptive behavior
• some human workers may not like to change/adapt their working conditions because they are used to a particular working habit, which may lead to emotional dissatisfaction if change is force on them by the robot
• insurance companies are rigid and international standards are still somewhat lacking in ergonomic PHRI
• skin pressure could be used as an ergonomics measure, in addition to the established ones (i.e., joint torque, muscle force, fatigue, posture, etc.) 

Contributions

We will welcome prospective participants to submit extended abstracts (up to 4 pages) to be presented as posters. The manuscripts should use the IEEE IROS two-column format. A PDF copy of manuscript should be submitted through our EasyChair platform. Each paper will receive a minimum of two reviews. The papers will be selected based on their originality, relevance to the workshop topics, contributions, technical clarity, and presentation. Accepted papers require that at least one of the authors register to the workshop.

Submission website

[Submit]

Important dates

---------------- *EXTENDED* DEADLINE! ----------------

• Submission deadline for extended abstracts: 30th of September, 2019
• Notification of acceptance: 7st of October, 2019

Selected Papers

• • Model-based Multiple Object Tracking Using Capacitive Proximity Sensors
  • Siyuan Huang, Hosam Alagi and Björn Hein 

• • Improving User Ergonomics through Adaptable Cobot Behaviour Part 1: A Generic Algorithm for the Computation of Optimal Ergonomic Postures
  • Ilias El Makrini, Greet Van De Perre, Glenn Mathijssen, Victor Van Wymeersch, Maxim Vochten, Wilm Decré and Bram Vanderborght

• • Interact with me: first insights into active pHRI
  • Yue Hu, Mehdi Benallegue, Gentiane Venture and Eiichi Yoshida 

• • Improving User Ergonomics through Adaptable Cobot Behaviour - Part 2: Towards an Ergonomic Human-Robot Handover using On-line Optimal Control and Learning by Demonstration
  • Victor van Wymeersch, Zeno Gillis, Maxim Vochten, Ilias El Makrini, Greet Van De Perre, Bram Vanderborght, Wilm Decre and Joris De Schutter 

• • An Intuitive Binary Work-Condition Map for Interactive Learning of Ergonomic Arm Postures
  • Cheng Fang, Luka Peternel, Wansoo Kim, Arash Ajoudani and Henrik Gordon Petersen 

• • A Human Perception System for Robot Collaborative Tasks
  • Claudia Latella, Yeshasvi Tirupachuri, Lorenzo Rapetti, Diego Ferigo, Silvio Traversaro, Ines Sorrentino, Francisco Javier Andrade Chavez, Francesco Nori and Daniele Pucci 

• • Partner-Aware Humanoid Robot Control: from Robot-Robot collaboration to Human-Robot Collaboration and Ergonomy Control
  • Lorenzo Rapetti, Yeshasvi Tirupachuri, Gabriele Nava, Claudia Latella, Kourosh Darvish and Daniele Pucci 

Speaker instructions

All accepted paper has to prepare the interactive poster session, it has two components: a 3-minute spotlight pitch and a 30-minute interactive presentation.

- Spotlight presentation:
Each poster would get a teaser talk in the spotlight session in order to motivate the interaction with the participants. In the spotlight session, each author would be allowed to speak for up to 3-min., accompanied by either the powerpoint slide or a video by using their own laptop.
Presentation is required to approach the podium before the start of the session and will present following the order set in the program (will announce).

- Poster session:
The poster format is A0 portrait (equivalent to 841mm wide x 1189mm). Adhesive material and/or pins will be provided for mounting the posters to the boards. Please ensure that your poster is up before the start of the poster presentation session

Schedule

The workshop will take place on Friday, 8th of November, 2019, room LG-R14.

Organizers

Wansoo Kim, Post Doc

Italian Institute of Technology, Italy

e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Wansoo Kim is a Post-Doc. at Human-Robot Interfaces and Physical Interaction Lab., Italian Institute of Technology, Italy from 2016. He received a Ph.D. degree in mechanical engineering from Hanyang University, Korea in 2015 (Intergrated MS/PhD program). He has developed several exoskeleton systems such as HEXAR-Hanyang Exoskeleton Assistive Robot, and conducted research on the control of the powered exoskeleton robot through the physical human-robot interaction (pHRI) forces. His PhD thesis is entitled “Human Synchronized Gait Control of the HEXAR-CR50 to Augment Lower Body Strength Based on the Human-Robot Interaction Force”. His research interests are in physical human-robot interaction(pHRI), human-robot collaboration, and powered exoskeleton robot.

Luka Peternel, Assistant Professor

Delft University of Technology, Netherlands

e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Luka Peternel received a Ph.D. in robotics from Faculty of Electrical Engineering, University of Ljubljana, Slovenia in 2015. He conducted Ph.D. studies at Department of Automation, Biocybernetics and Robotics, Jožef Stefan Institute in Ljubljana from 2011 to 2015, and at Department of Brain-Robot Interface, ATR Computational Neuroscience Laboratories in Kyoto, Japan in 2013 and 2014. He was with Human-Robot Interfaces and Physical Interaction Lab, Advanced Robotics, Italian Institute of Technology in Genoa, Italy from 2015 to 2018. From 2019, Luka Peternel is an Assistant Professor at Department of Cognitive Robotics, Delft University of Technology in the Netherlands.

Arash Ajoudani, Principal Investigator

Italian Institute of Technology, Italy

e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Arash Ajoudani received his PhD degree in Robotics and Automation from Centro "E Piaggio", University of Pisa, and Advanced Robotics Department (ADVR), Italian Institute of Technology (IIT), Italy (July 2014). His PhD thesis was a finalist for the Georges Giralt PhD award 2015 - best European PhD thesis award in robotics. He is currently a tenure-track scientist and the leader of the Human-Robot Interfaces and physical Interaction (HRI2) lab of the IIT. He was a winner of the Amazon Research Awards 2019, the winner of the Werob best poster award 2018, winner of the KUKA Innovation Award 2018, a finalist for the best conference paper award at Humanoids 2018, a finalist for the best interactive paper award at Humanoids 2016, a finalist for the best oral presentation award at Automatica (SIDRA) 2014, the winner of the best student paper award and a finalist for the best conference paper award at ROBIO 2013, and a finalist for the best manipulation paper award at ICRA 2012. He is the author of the book "Transferring Human Impedance Regulation Skills to Robots" in the Springer Tracts in Advanced Robotics (STAR), and several publications in journals, international conferences, and book chapters. He is currently serving as the executive manager of the IEEE-RAS Young Reviewers' Program (YRP), chair and representative of the IEEE-RAS Young Professionals Committee, and co-chair of the IEEE-RAS Member Services Committee. He has been serving as a member of scientific advisory committee and as an associate editor for several international journals and conferences such as IEEE RAL, Biorob, ICORR, etc. His main research interests are in physical human-robot interaction and cooperation, robotic manipulation, robust and adaptive control, rehabilitation robotics, and tele-robotics.

Dana Kulić, Professor

Monash University, University of Waterloo, Canada

e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Dana Kulić received the combined B.A.Sc. and M.Eng. degrees in electromechanical engineering and the Ph.D. degree in mechanical engineering from the University of British Columbia, Vancouver, Canada, in 1998 and 2005, respectively. From 1997 to 2002, she worked as a systems engineer designing fuel cell systems with Ballard Power Systems, and developing operational control software for the CanadaArm II at MacDonald Dettwiler. From 2002 to 2006, Dr. Kulić worked with Dr. Elizabeth Croft as a Ph. D. student and a post–doctoral researcher at the CARIS Lab at the University of British Columbia. The aim of this work was to develop a human–robot interaction strategy to ensure the safety of the human participant. The approach was based on quantifying the level of danger present in the interaction, and then acting to minimize that danger, both during path planning and real–time control. A second component of the work examined the feasibility of using human monitoring information (such as gaze direction, head rotation and physiological monitoring) to improve the safety of the human robot interaction. From 2006 to 2009, Dr. Kulić was a JSPS Post–doctoral Fellow and a Project Assistant Professor at the Nakamura-Yamane Laboratory at the University of Tokyo, Japan. The aim of her research was to develop algorithms for incremental learning of human motion patterns for humanoid robots. This work focused on incremental algorithms for automatically segmenting, clustering and organizing motion pattern primitives observed from human demonstration. The autonomously extracted knowledge about human movement could then be used both for human behavior analysis and prediction, as well as for motion generation for humanoid robots. Dr. Kulić is currently an Assistant Professor at the Electrical and Computer Engineering Department at the University of Waterloo. Her research interests include robot learning, humanoid robots, human–robot interaction and mechatronics.

Eiichi Yoshida, Co-Director

Intelligent Systems Research Institute (IS-AIST), AIST, Tsukuba, Japan

e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Eiichi Yoshida received M.E and Ph.D degrees on Precision Machinery Engineering from Graduate School of Engineering, the University of Tokyo in 1993 and 1996 respectively. In 1996 he joined former Mechanical Engineering Laboratory, later reorganized as National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan. He served as Co-Director of AIST/IS-CNRS/ST2I Joint French-Japanese Robotics Laboratory (JRL) at LAAS-CNRS, Toulouse, France, from 2004 to 2008. Since 2009, he is Co-Director of CNRS-AIST JRL (Joint Robotics Laboratory), UMI3218/RL, and since 2015 he serves as Deputy-Director of Intelligent Systems Research Institute (IS-AIST), AIST, Tsukuba, Japan. His research interests include robot task and motion planning, human modeling, and humanoid robots.

Invited Speakers

• Dr. Wansoo Kim

  Italian Institute of Technology, Italy

  "Towards ergonomic control of human-robot coexistence and collaboration"
  The talk will present a control approach to human-robot co-manipulation that accounts for human ergonomics and related fatigues. To achieve an adaptive and context-aware robot behaviour in physical interaction with human and environment, we will first present a human monitoring method that includes a overloading joint torque estimation technique, as well as the feedback methods to provide a guidance to human. The robot by means the mobile manipulator platform MObile Collaborative robotic Assistant (MOCA) then uses the lower level control framework in conjunction with the proposed higher level methods that can estimate and anticipate human states that contribute to ergonomics and physical fatigue. In this direction, real-time measurement systems and dynamical models are used to monitor the human online, while he/she is collaboratively performing the tasks with the robot. The robot then uses the proposed methods to control its own behaviour in a way that offloads the excessive effort of the human and ensures ergonomic working conditions, as well as the coexistence in the workplace.

• Prof. Luka Peternel

  Delft University of Technology, Netherlands

  "Ergonomics: from human motor control to human-robot collaboration"
  The first part of the talk will examine ergonomics of body movements from human motor control perspective. When optimising the movement execution the human central nervous system (CNS) accounts for two tradeoffs, i.e. speed-accuracy and cost-benefit. We have developed a computation model of CNS control that can account for these tradeoffs simultaneously. The second part of the talk will examine the ergonomics from physical human-robot collaboration perspective. The robot should account for the efficiency of the human co-worker while executing the collaborative tasks. In particular, muscle manipulability and muscle fatigue play an important role in improving human efficiency. Therefore, we have developed methods that adjust the robot control in a way that these human parameters can be estimated and optimised.

• Dr. Ko Ayusawa

  CNRS-AIST JRL (Joint Robotics Laboratory), UMI3218/RL, AIST, Tsukuba, Japan

  "Human motion reproduction for ergonomic design and evaluation of assistive devices"
  This talk will present an approach for ergonomic design and evaluation of assistive devices through reproducing human movements by using humanoid robots and digital human models. As a humanoid robot can reproduce human movements in physical world, the mechanical effects of assistive devices can be evaluated quantitatively by its sensory measurement, which is difficult with human experiments. This talk introduces such evaluation framework as well as the technology of imitating human movements by a humanoid robot. We also present another direction of ergonomic evaluation through motion reproduction by digital human models. Several related technologies will be introduced, such as subject specific human modeling, simulation and optimization of human motion with physical interaction, and real-time musculoskeletal analysis.

• Dr. Serena Ivaldi

  National Institute for Research in Computer Science and Control (Inria), France

  "Improving the ergonomics in human-robot collaboration with predictive models"
  In this talk I will present some recent results that we obtained in the European Project AnDy, where we focus on improving the ergonomics conditions of workers in collaborative robotics scenarios. I will focus on probabilistic models for prediction of human intent and online ergonomics estimation. Such models provide information for building informed robot controllers that optimize the human ergonomics.

• Prof. Heni Ben Amor

  Arizona State University, USA

  "Machine Learning and Predictive Biomechanics for Human-Robot Collaboration"
  Collaborative robots and other forms of modern assistive technology, e.g., smart prosthetic devices and exoskeletons, have the potential to change millions of lives for the better. However, for this vision to become reality, a theoretical foundation is needed that allows for the specification of safe and meaningful physical interactions between humans and robots. In this talk, I will discuss Bayesian Interaction Primitives (BIP) --  unified statistical framework for modeling dynamics among multiple agents using a compact, probabilistic, and data-driven methodology. BIPs can be used to derive algorithms for learning and adaptation which incorporate the future biomechanical state of a human user into the decision-making process. In turn, predicted biomechanical variables can be used to steer physical human-robot interaction towards biomechanically safe movement regimes. Finally, I will discuss applications of our approach to collaborative robotics and the control of intelligent prosthetics. 

• Prof. Alexander Leonessa

  Virginia Tech, USA

  "Effects of a whole-body, powered exoskeleton on the completion of industrial tasks: current metrics and future adaptations"

  Despite advances in industry automation and mechanization, the flexibility, dexterity and resourcefulness of humans are indispensable in diverse work environments. In these environments, workers are often routinely exposed to ergonomic risk factors for work-related musculoskeletal disorders (WMSDs), the single largest component of work-related illnesses in many industrialized countries. There is a growing interest in using wearable robotic devices such as “whole-body” powered exoskeletons (WB-PEXOs) to prevent WMSDs. Using an exoskeleton provides on-body assistance/augmentation to the wearer, thereby potentially reducing physical demands without limiting their mobility and resourcefulness in various work environments. Exoskeletons thus allow for effective and flexible integration of manual and automated operations by augmenting the physical capacity of workers (e.g., strength, endurance) while minimizing exposure to WMSD risks. Currently, however, there is a lack of understanding exactly how human operators learn, use, and adapt to such technologies. Previous exoskeleton research is largely limited to passive exoskeletons that require no power sources or actuators/motors, so there is practically no scientific literature available on human-subjects evaluations with WBPEXOs that can quantify the ergonomic impact on human operators. Understanding these effects is crucial to developing a robot that can closely interact with a human in a work environment.

  We conducted a study in which we assessed how using a prototype WBPEXO (early pre-alpha version of Guardian XO with 24 total degrees of freedom, 8 active, www.sarcos.com) impacts human operators in terms of body motion control, muscle activity, and task performance. Five participants were evaluated during the performance of several industry-relevant tasks, both with and without the XO. Outcome metrics derived using body kinematics and muscle activity showed the current effectiveness of XO during the different work tasks, and identified specific aspects in which it needs further development to better collaborate with the human. Our results support the potential for effective use of whole-body powered exoskeletons in the future workplace, and contribute to the evidence base regarding the use of an industrial WB-PEXO by identifying the metrics that best describe the ergonomic state of the human while operating the WB-PEXO. The findings from this study will enable the development of novel controllers for the WB-PEXO that are aware of these ergonomic metrics and are able to continuously adapt to the ergonomic state of the operator, thereby increasing the ease and safety of the industrial environment, and decreasing the risk of WMSDs.

• Prof. Bram Vanderborght

  Vrije Universiteit Brussel, Brussels, Belgium

  "How can cobots and exoskeletons improve productivity and ergonomics of workers?"
  Manufacturing matters and re-industrialisation requires major strategic initiatives. Digitalisation, AI and robotisation are often seen as a threat, but are in fact an enormous opportunity for our economy and society. By also placing health, the working conditions and the sustainable development goals central, we can ensure ourselves to reap the full benefits not only economically but also for our society. Collaborative robots are a unique technology that with proper management has the potential to improve both: the economy and the society while embracing Europe’s values. This talk will discuss the multidisciplinary approach to develop a new generation of cobots and exoskeletons and control algorithms for improving the ergonomics of workers in the manufacturing industry.

• Prof. Paolo Rocco

  Politecnico di Milano, Italy

  "Collaborative robots as assistive and intelligent fellow co-workers"
  Collaborative robots improve productivity and facilitate the worker in demanding tasks. They can assist the operator for example during manipulation of heavy and bulky objects, moving the workpiece in a way that the human is always operating close to his/her most natural and ergonomic posture. Leveraging machine learning techniques, they can also interpret the human behavior, synchronizing their actions to those of the human worker in order to maximize productivity. The talk will give an overview of some recent results obtained by the MERLIN group in this field.

• Dr. Daniele Pucci

  Italian Institute of Technology, Italy

  "A framework for the control of physical human robot collaboration with humanoid robots"

Pictures

Thank you to all the Attendees, Contributors and Speakers!

Acknowledgments

The following IEEE-RAS Technical Committees have acknowledged the full support of the proposed workshop:

1. IEEE RAS TC on Human Movement Understanding.
   Co-chairs: Prof. Emel Demircan, Prof. Dana Kulic, Prof. Denny Oetomo, and Prof. Mitsuhiro Hayashibe
2. IEEE RAS TC on Wearable Robotics.
   Co-chairs: Prof. Samer Mohammed, Prof. Yasuhisa Hasegawa, Dr. Juan C. Moreno, and Prof. Thomas Sugar
3. IEEE RAS TC on Robot Learning.
   Co-chairs: Prof. Byron Boots, Prof. Jens Kober, and Prof. Wataru Takano
4. IEEE RAS TC on Human-Robot Interaction and Coordination.
   Co-chairs: Prof. Fillippo Cavallo, Dr. Yoshio Matsumoto, and Prof. David-Feil Seifer