This interdisciplinary workshop focuses on the latest advancements in human-in-the-loop robot learning, which integrate human multi-modal input (e.g. natural language, gestures, and haptic interaction) and online feedback (e.g., rewards, corrections, and preferences) to improve robot manipulation performance, adapt to new situations and align better with human intentions. Recent breakthroughs in foundation models, such as Large Language Models (LLMs), Vision-Language Models (VLMs), and Vision-Language-Action Models (VLAs), provide unprecedented perception and reasoning capabilities. However, their effective integration into robotics remains an emerging and underexplored challenge, especially for manipulation planning and control. This workshop will explore how foundation models and human-in-the-loop learning can be synergistically integrated to effectively enhance the robot learning process through active and intuitive human participation. We will delve into three critical themes:
- How can foundation models be leveraged for adaptive and generalizable learning for robot manipulation planning and control in complex dynamic environments where robots continuously interact with the physical world?
- What are the best practices for integrating real-time human feedback to refine learning processes and improve alignment with human intentions?
- How to design adaptive learning frameworks to ensure safety and robustness in human-robot interactions and collaborations?
Topics of interest
Topics of interest include but are not limited to:
- Human-AI collaboration for robot learning
- Human-AI hybrid intelligence
- Foundation models for robot manipulation
- Transfer learning and fine-tuning of foundation models for robotic manipulation
- Knowledge representation and reasoning in robots
- Human feedback in robot learning
- Human-in-the-loop reinforcement learning
- Learning from demonstrations and corrections
- Interactive robot manipulation learning
- Multi-task robot learning
- Architectures and frameworks for human-in-the-loop learning
- Cognitive models for robot learning
- Adaptive human-robot interaction
- Safety and robustness in human-robot collaboration
Program
| Time | Description |
|---|---|
| 14.00 – 14.15 | Welcome and Introduction by the organizers |
| 14.15 – 14.40 | Talk 1: Robot Generalization with the Power of Large Vision Models by Huazhe Xu |
| 14.40 – 15.05 | Talk 2: Learning from Instrumented Human Demonstration by Shuran Song |
| 15.05 – 15.30 | Talk 3: Intelligent Physical Agents: High-Performance Human-in-the-loop Learning for Generalist Robots by Jianlan Luo |
| 15.30 – 16.00 | Poster Session, Hands-on demonstrations, and Coffee Break |
| 16.00 – 16.25 | Talk 4: Collective Shaping of Multi-Particle Systems by Robot Networks by David Navarro-Alarcon |
| 16.25 – 16.50 | Talk 5: Understanding Humans, Empowering Robots: Online Assessment of Human Psycho-Physical State and Expert Feedback for Adaptive Robot Behaviour by Marta Lagomarsino |
| 16:50 – 17.15 | Talk 6: Robot Assembly Task Learning from Human Guide by Dongheui Lee |
| 17.15 – 17.45 | Panel Discussion and Award Ceremony |
| 17.45 – 18.30 |
Networking and Happy Hour |
Invited Speakers
Dr. Huazhe Xu
Assistant Professor, Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, China; Co-founder of GalaxeaAI
Talk Title: Robot Generalization with the Power of Large Vision Models
Dr. Shuran Song
Assistant Professor, Department of Electrical Engineering, Stanford University, USA
Talk Title: Learning from Instrumented Human Demonstration
Dr. Jianlan Luo
Postdoctoral Scholar, Berkeley Artificial Intelligence Research (BAIR), University of California, Berkeley, USA
Talk Title: Intelligent Physical Agents: High-Performance Human-in-the-loop Learning for Generalist Robots
Prof. David Navarro-Alarcon
Associate Professor, Department of Mechanical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China
Talk Title: Collective Shaping of Multi-Particle Systems by Robot Networks
Dr. Marta Lagomarsino
Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
Talk Title: Understanding Humans, Empowering Robots: Online Assessment of Human Psycho-Physical State and Expert Feedback for Adaptive Robot Behaviour
Prof. Dongheui Lee
Full Professor, Faculty of Electrical Engineering and Information Technology, Technische Universität Wien, Austria
Talk Title: Robot Assembly Task Learning from Human Guide
Call for Contributions
We invite submissions of extended abstracts, position papers, and technical contributions (2-4 page manuscripts) that address the integration of foundation models with human-in-the-loop robot learning. The call will open once our proposal is accepted. Accepted papers will be invited to submit extended versions to a partnered special issue of the Robot Learning journal related to the workshop topic.
Awards
To incentivize high-quality participation, we will present:
- Two Best Workshop Paper Awards (150 USD each), sponsored by Discover Robotics, for outstanding contributions that advance human-in-the-loop learning with foundation models.
- Awardees will receive fast-tracked reviews for the journal's special issue.
- Selection criteria include technical rigor, novelty, and potential for real-world impact. A diverse committee of 5 experts will evaluate submissions.
Online Questionnaire
This questionnaire will collect topics and questions, which will be discussed in the panel discussion session. Your input will be used solely to curate panel discussion topics for this workshop. All responses are anonymized and will not be shared with third parties. Please fill it out here.
Organizers
Jianzhuang Zhao, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: jianzhuang.zhao@iit.it
website: https://www.iit.it/it/people-details/-/people/jianzhuang-zhao
Xing Liu, Professor, Research Center for Intelligent Robotics, School of Astronautics Northwestern Polytechnical University, Xi’an, P.R.China
e-mail: xingliu@nwpu.edu.cn
website: https://www.researchgate.net/profile/Xing-Liu-33
Marta Lagomarsino, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: marta.lagomarsino@iit.it
website: https://www.iit.it/it/people-details/-/people/marta-lagomarsino
Francesco Tassi, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: francesco.tassi@iit.it
website: https://www.iit.it/it/people-details/-/people/francesco-tassi
Shufei Li, Post-Doc, Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, P.R. China
e-mail: shufei.li@outlook.com
website: https://scholar.google.com/citations?user=CpCQmkwAAAAJ&hl=en&oi=ao
Gustavo Jose Giardini Lahr, Assistant Professor, Hospital Israelita Albert Einstein, San Paulo, Brazil
e-mail: gustavo.lahr@einstein.br
website: https://www.einstein.br/pesquisa/instituto-do-cerebro
Chenguang Yang, Professor, Fellow, IEEE, School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool, UK
e-mail: Chenguang.Yang@liverpool.ac.uk
website: https://www.liverpool.ac.uk/people/charlie-yang
Xuguang Lan, Professor, Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, P.R. China
e-mail: xglan@mail.xjtu.edu.cn
website: https://gr.xjtu.edu.cn/en/web/zeuslan
Panfeng Huang, Professor, Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, P.R.China
e-mail: pfhuang@nwpu.edu.cn
website: https://www.researchgate.net/profile/Panfeng-Huang
Arash Ajoudani, Senior Researcher Tenured, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: arash.ajoudani@iit.it
website: https://www.iit.it/people-details/-/people/arash-ajoudani
This workshop engages three PhD students, Elena Merlo (Italian Institute of Technology, Italy), Zihao Liu (Northwestern Polytechnical University, China), and Zifan Wang (The Hong Kong University of Science and Technology, China) who will collaboratively design promotional campaigns and coordinate hands-on demonstrations during local arrangements.
Sponsors
This workshop is supported in part by the Horizon Europe project TORNADO (grant agreement No.101189557) and in part by the National Key R&D Program of China under Grant 2022ZD0117903, and the National Natural Science Foundation of China under Grant 92370123, 62273280, and the Guangdong Major Project of Basic and Applied Basic Research under Grant 2023B0303000016.
For our workshop’s best paper award and happy hour, we are privileged and grateful to have the financial support of several confirmed sponsors:
IEEE-RAS Technical Committee on Collaborative Automation for Flexible Manufacturing
