HKU IDS Scholar Seminar Series #12: Achilles' Heel in Manipulation: Key Recipe and Missing Pieces towards Intelligent Embodied AI
Venue: IDS Seminar Room, P603, Graduate House / Zoom
Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.
Abstract
The increasing demand for versatile robotic systems to operate in diverse and dynamic environments has emphasized the importance of a generalist policy, which leverages a large cross-embodiment data corpus to facilitate broad adaptability and high-level reasoning. However, the generalist would struggle with inefficient inference and cost-expensive training. The specialist policy, instead, is curated for specific domain data and excels at task-level precision with efficiency. Yet, it lacks the generalization capacity for a wide range of applications. Inspired by these observations, we introduce RoboDual, a synergistic dual-system that supplements the merits of both generalist and specialist policy. A diffusion transformer-based specialist is devised for multi-step action rollouts, exquisitely conditioned on the high-level task understanding and discretized action output of a vision-language-action (VLA) based generalist. Compared to OpenVLA, RoboDual achieves a 12% improvement on CALVIN and 26.7% in real-world by adapting the specialist policy with 20M trainable parameters only. It maintains strong performance with merely 5% of demonstration data, and enables a 3.8 higher control frequency in real-world deployment. Code and models would be made publicly available.
Speaker
For full biography of Prof. LI, please refer to: https://datascience.hku.hk/people/hongyang-li/
Moderator
Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.
Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/
For information, please contact:
Email: datascience@hku.hk
- November 1, 2024
- Events, Gallery, Past Events
- HKU IDS Scholar Seminar Series