This post is protected. To view it, enter the password below!
Protected: IDS Research Seed Funds 2023: Four Transformative Projects Conclude in Triumph – Outcome Presentation Session Recap
HKU IDS Scholar Seminar Series #20: Towards Multimodal and Interactive Visual Generation as World Models
HKU IDS Scholar Seminar Series #20:
Towards Multimodal and Interactive Visual Generation as World Models
Abstract
As generative models achieve increasingly greater performance, the research frontier is shifting toward new challenges of multimodal and interactive world models. This talk presents recent advancements and insights across three interconnected themes. First, we introduce unified frameworks for multimodal understanding and generation, exploring methods to enhance their semantic-spatial reasoning abilities. Second, we demonstrate interactive video generation systems that incorporate action control mechanisms, enabling gaming-like experiences where users dynamically influence content evolution. Our solutions address critical challenges in memory and 3D consistency during prolonged interaction sessions. Finally, we propose autoregressive visual generation architectures that inherently support multimodal integration and interactivity. Through systematic architectural innovations, we overcome longstanding bottlenecks in output quality and computational efficiency, establishing a viable alternative to diffusion-based paradigms. Looking into the future, we aim to build multimodal and interactive visual generation models as world models.
Speaker
Prof Xihui LIU
Assistant Professor @ HKU IDS & EEE Professor Xihui Liu is an Assistant Professor at the Department of Electrical and Electronic Engineering (EEE) and the Musketeers Foundation Institute of Data Science (IDS), The University of Hong Kong. Before joining HKU, she was a Postdoctoral Researcher at UC Berkeley working with Prof. Trevor Darrell. She received her Ph.D. degree from Multimedia Lab, The Chinese University of Hong Kong in 2021 and her Bachelor’s degree from Tsinghua University in 2017. She has won several awards such as Adobe Research Fellowship 2020, MIT EECS Rising Stars 2021, CVPR 2021 Doctoral Consortium Award, WAIC Rising Star Award 2022, CVPR Outstanding Reviewers Award, and ICLR Outstanding Reviewers Award. For full biography of Prof. LIU, please refer to: https://datascience.hku.hk/people/xihui-liu/
Moderator
Prof Ping LUO
Associate Professor @ HKU IDS & CS Professor Ping Luo’s researches aim at 1) developing Differentiable/ Meta/ Reinforcement Learning algorithms that endow machines and devices to solve complex tasks with larger autonomy, 2) understanding foundations of deep learning algorithms, and 3) enabling applications in Computer Vision and Artificial Intelligence. Professor Ping Luo received his PhD degree in 2014 in Information Engineering, the Chinese University of Hong Kong (CUHK), supervised by Prof. Xiaoou Tang (founder of SenseTime Group Ltd.) and Prof. Xiaogang Wang. He was a Research Director in SenseTime Research. He has published 70+ peer-reviewed articles (including 20 first author papers) in top-tier conferences and journals such as TPAMI, IJCV, ICML, ICLR, NeurIPS and CVPR. He has won a number of competitions and awards such as the first runner up in 2014 ImageNet ILSVRC Challenge, the first place in 2017 DAVIS Challenge on Video Object Segmentation, Gold medal in 2017 Youtube‐8M Video Classification Challenge, the first place in 2018 Drivable Area Segmentation Challenge for Autonomous Driving, 2011 HK PhD Fellow Award, and 2013 Microsoft Research Fellow Award (ten PhDs in Asia). For full biography of Prof. Ping LUO, please refer to: https://datascience.hku.hk/people/ping-luo/
Moderator
Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science
Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.
Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/
HKU IDS Seminar: Learning Causal Representations
Host
Co-Host
HKU IDS Seminar:
Learning Causal Representations
Abstract
Causal representations are models of real-world data that retain causal information, so in particular, they provide information about how a system will respond when subject to experimental intervention. While there is an extensive literature on how to discover the causal relations among a given set of variables, it is much less clear how to identify and construct the causal variables in the first place. Yet, given the vast amounts of measurement and sensor data available today, identifying the causal quantities has become just as important as identifying the relations between them. This presentation will focus on one approach, Causal Feature Learning, that learns a macro level causal representation from micro level measurement data. Time permitting, we will illustrate the method with applications in climate science, economics and neuroscience.
Speaker
Prof Frederick Eberhardt
Professor of Philosophy, California Institute of Technology
Frederick Eberhardt's research primarily focuses on methods for causation and how we might learn about causal relations from data. His research projects generally fall in an area of overlap between philosophy, machine learning, statistics, and cognitive science. He has also done some historical work on the philosopher Hans Reichenbach, especially on his frequentist interpretation of probability.
Before coming to Caltech in 2013, Eberhardt was an assistant professor in the Philosophy-Neuroscience-Psychology program in the department of philosophy at Washington University in St. Louis. He spent a year as a McDonnell Postdoctoral Fellow at the Institute of Cognitive and Brain Sciences at the University of California, Berkeley. He holds a PhD in Logic, Computation and Methodology from the department of philosophy at Carnegie Mellon University (CMU), and a masters in Knowledge Discovery and Data Mining from what is now CMU's Machine Learning Department.
For full biography of Prof. Eberhardt, please refer to: https://www.hss.caltech.edu/people/frederick-eberhardt#profile-399c3458-tab
Moderator
Prof Boris BABIC
Associate Professor, HKU IDS, Dept of Philosophy & Law(by courtesy), HKU Professor Boris Babic is HKU-100 Associate Professor at the University of Hong Kong, jointly appointed in the Musketeers Foundation Institute of Data Science, the Department of Philosophy, and (by courtesy) the Faculty of Law, from Fall 2023. He also serves as an occasional visiting professor in the Decision Sciences department at INSEAD. Professor Babic’s primary research interests are in Bayesian inference and decision-making, ethics, law, and policy of artificial intelligence and machine learning, especially in medical applications. His research has been published extensively in leading journals such as Science, Nature Machine Intelligence, Nature Digital Medicine, and the Harvard Business Review. For full biography of Prof. Babic, please refer to: https://datascience.hku.hk/people/boris-babic/
Moderator
Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science
Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.
Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/
HKU IDS 3rd Year PhD Candidate REN Xubin Reaching Another New Height – Shines and Awarded with the “Ant InTech Scholarship – Future (Year 2025)” for PhD Candidates!
HKU IDS 3rd Year PhD Candidate REN Xubin Reaching Another New Height – Shines and Awarded with the “Ant InTech Scholarship – Future (Year 2025)” for PhD Candidates!
With great pleasure, we are thrilled to announce that Mr. Xubin REN, currently a Year 3 PhD candidate at HKU IDS supervised by Professor Chao Huang, has been awarded the “Ant InTech Scholarship – Future (Year 2025) (螞蟻InTech科技獎學金（未來）)” for young researchers.
The Ant InTech Award, at its second year of occurrence, aims to recognize the outstanding young Chinese researchers to continuously strive for excellence in exploring and resolving pressing issues in the scientific development of AI and machine learning. This year, the Ant Group focuses on four core areas: general artificial intelligence (AGI) technology, embodied intelligence technology, digital medicine technology, data processing, and security and privacy technology, and the award presentation ceremony for the InTech Awards & InTech Scholarships – Future took place alongside with The 2025 Inclusion · Conference on the Bund, in Shanghai, China, on September 11, 2025.
Xubin, together with 9 other junior researchers pursuing doctoral studies in various esteemed universities such as Tsinghua University, National University of Singapore, and Fudan University, received the honour. Xubin is currently supervised by Professor Chao Huang, HKU-100 IDS Scholar, whose research interests lie with Large Language Models, Autonomous Agents and Graph Machine Learning.
Let’s join us in congratulating Xubin and Professor Huang’s team on their ongoing good works! Learn more about the scholarship here: https://www.antresearch.com/cooperation/InTech
Warm Greetings to Our Future Data Science Superstars – Welcoming Our New RPg Students for 2025-26!
Warm Greetings to Our Future Data Science Superstars
HKU IDS is thrilled to kick off the 2025-26 academic year by welcoming our new batch of research postgraduate (RPg) students from around the world on September 3, 2025!
The orientation day was a vibrant start, with an exciting campus tour guided by one of our 3rd-year PhD students. Our new RPg students were warmly greeted at the HKU IDS office by Professor Yi Ma, our Institute Director and Chair of Artificial Intelligence, alongside our esteemed faculty, current RPg students, and dedicated staff. The morning was filled with useful information about our RPg programme, and a joyful lunch reception, that sparked new connections.
Join us in celebrating our newest members of the HKU IDS research family. Let’s learn and grow together!
Advancing Data Science Through Complex Networks: An Interdisciplinary Exploration at Our 2nd Interdisciplinary Workshop
Event Highlights
An In-depth Exploration of Foundational Knowledge at Our 2nd Interdisciplinary Workshop
Organizer
Collaborating partner
& 
The HKU IDS Interdisciplinary Workshop, ” Understanding Complex Networks for Advancing Fundamental Data Science,” held on August 28–29, 2025, at The University of Hong Kong, brought together researchers and scholars to explore the transformative potential of network science. Hosted by the HKU Musketeers Foundation Institute of Data Science (IDS) and chaired by Prof. Alec Kirkley, this two-day hybrid event at CPD 2.58, Centennial Campus, fostered dynamic discussions on network theory and its applications across diverse domains.
The workshop opened with remarks from Prof. Yi Ma, Director of the HKU Musketeers Foundation Institute of Data Science. Prof. Ma praised the Organizing Committee’s vision and underscored the importance of interdisciplinary approaches in unraveling complex systems. He highlighted that a deep understanding of network theory and computational methods is crucial for advancing data science, positioning HKU IDS as a pioneer in our comprehensive offering of graduate courses well-suited for this global pursuit.
We were pleased to welcome over 80 participants, including students, faculty from HKU and sister institutions, and international researchers despite the slightly unpleasant weather. The workshop featured compelling presentations bridging theoretical rigor with practical impact. Keynote speakers, including Prof. Guanrong Chen and Prof. Tiago P. Peixoto, delivered thought-provoking lectures on topics such as optimal synchronization and statistical physics on hypergraphs. The 2-day event concluded with vibrant panel discussions, sparking enthusiastic audience questions and fostering insightful exchanges among academics from HKU and beyond.
This workshop highlighted IDS’ commitment to driving interdisciplinary collaboration, paving the way for innovative breakthroughs in data science research. We look forward to future initiatives with the HKU IDS community to continue advancing network science!
Highlights
Exploring the Frontiers of Embodied AI with HKU IDS: Summer Course IDSS 2501 “Embodied AI 101”
Event Highlights
Exploring the Frontiers of Embodied AI with HKU IDS:
Organized by the HKU Musketeers Foundation Institute of Data Science and supported by the HKU Summer Institute, we joyfully concluded the 2-week summer programme IDSS 2501: Embodied AI 101, held from July 22 to August 1, 2025. This immersive and dynamic course welcomed around 100 undergraduates across the globe to explore the sea of embodied artificial intelligence, machine learning, and robotics. Guided by our distinguished HKU IDS Director & Professor, Chair of Artificial Intelligence, Professor Yi Ma (HKU IDS / SCDS), Professor Yanchao Yang (HKU IDS / Electrical and Electronic Engineering), as well as Professor Hongyang Li (HKU IDS), students were grouped by their academic backgrounds and mentored by HKU IDS PhD candidates to explore the exciting realm of embodied AI. They even got a chance to join hands-on lab visits, and complete collaborative projects.
Previous
Next
Participants to IDSS2501 revealed very positive feedback about the course structure as well as the course instructors’ keen passion on the areas of embodied AI, which aligned with the results reflected from the post-programme evaluation survey. One of the top performers at the final group presentations expressed her compliment to the programme, as “All the professors explained the complex concepts really clearly,” and she highly “recommended [this course] to [her] peers”. Another student who showed up at a post-programme interview also commented that “the course was beyond [her] expectations…and [she had] learnt a lot from the teammates.”
The course’s success reflects the exceptional teaching quality and research expertise of our HKU IDS faculty. The programme culminated in a vibrant final group presentation session, dated August 1, 2025 which was the last day of the programme, where students showcased innovative projects. Heartfelt congratulations to the Best Group Presentation winners for their diligience and excellence, the Most Favourite (Most Voted) Presenters for their captivating delivery, and the Best Teaching Assistant for their outstanding support.
Previous
Next
Beyond academics, students enjoyed a rich HKU experience, and a strong sense of camaraderie among peers. The nine dedicated teaching assistants from our HKU IDS community also earned high praise for their guidance. We hope IDSS 2501 has sparked a lasting passion for embodied AI and equipped students to navigate the future of intelligent systems. A huge round of applause to all to our programme participants, and we wish you a wonderful summer vacation ahead!
HKU IDS Interdisciplinary Workshop – Understanding Complex Networks for Advancing Fundamental Data Science
HKU IDS Scholar Seminar Series #19: Foundation Models as Embodied Agents: Towards AI That Talks to You and Acts on Your Behalf
HKU IDS Scholar Seminar Series #19:
Foundation Models as Embodied Agents: Towards AI That Talks to You and Acts on Your Behalf
Abstract
Recent advances in foundation models have enabled AI agents to operate across digital and physical environments through natural language interaction. In this talk, I will present our work on transforming large language models and vision-language models into embodied agents that execute real-world tasks on behalf of users. I will discuss three categories of agents: code agents that automatically generate code for data science and software development, computer-use agents that control your computer as humans do, and physical AI agents that take actions in the physical world. Our approach grounds natural language and perception into executable code and actions within their respective environments, enabling non-experts to access complex systems through conversational interfaces. I will cover our methods for transforming LLMs/VLMs into these agents, demonstrate how we evaluate their performance in embodied environments, and discuss the technical challenges and safety considerations of deploying AI agents that seamlessly bridge language understanding with actionable behaviors across digital and physical domains.
Speaker
Prof Tao YU
Assistant Professor @ HKU IDS & CDS Professor Tao Yu is an Assistant Professor in the Computer Science Department of the University of Hong Kong. He is also a Postdoctoral Research Fellow in the Department of Computer Science and Engineering at University of Washington and a co-director of the NLP group at the University of Hong Kong. His research interest is in Natural Language Processing and Deep Learning, with a focus on designing and building conversational natural language interfaces that can help humans explore and reason over data in any application (e.g., relational databases and mobile apps) in a robust and trusted manner. He has published and served in the program committee at ACL, EMNLP, ICLR, NAACL, etc. He co-organized the Interactive and Executable Semantic Parsing workshop at EMNLP 2020. For full biography of Prof. YU, please refer to: https://datascience.hku.hk/people/tao-yu/
Moderator
Dr Wenjie HUANG
Research Assistant Professor @ HKU IDS & DASE Dr. Wenjie Huang is Research Assistant Professor in Department of Data and Systems Engineering, The University of Hong Kong. He received Ph.D. degree from the Department of Industrial Systems Engineering and Management, National University of Singapore (NUS) in 2019 and B.S. degree in the Department of Industrial Engineering from Shanghai Jiao Tong University, China in 2014. Prior to joining HKU, he held joint postdoc positions at School of Data Science, The Chinese University of Hong Kong, Shenzhen and Group for Research in Decision Analysis (GERAD), Quebec, Canada. His research projects have been supported by NSFC research funds, NRF Singapore and NUS Young Investigator Award. For full biography of Dr. Huang, please refer to: https://datascience.hku.hk/people/wenjie-huang/
Moderator
Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science
Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.
Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/
HKU IDS Interdisciplinary Seminar Series: Exchangeability and Algorithmic Randomness: A New Proof of the Principal Principle
HKU IDS Interdisciplinary Seminar Series:
Exchangeability and Algorithmic Randomness: A New Proof of the Principal Principle
Host:
Co-host:
Abstract
We explore the role of algorithmic randomness and exchangeability in defining probabilistic laws and their implications for chance-credence principles like the Principal Principle. Building on our previous work on probabilistic constraint laws (arXiv:2303.01411), we develop a new approach to proving the Principal Principle. This proof avoids circularity by grounding it in algorithmic randomness, frequency constraints, and exchangeable priors. Our approach establishes a direct link between long-run frequencies and short-term credences, clarifying the epistemic foundations of chance, typicality, and probabilistic laws. (Joint work with Jeffrey A. Barrett.)
Speaker
Prof Eddy Chen
Associate Professor, Department of Philosophy, University of California, San Diego (UCSD) Professor Eddy Chen is an associate professor of philosophy at the University of California, San Diego (UCSD), a fellow of the John Bell Institute for the Foundations of Physics, and an affiliated faculty member of the UCSD Chinese Studies Program. His primary research interests include philosophy of physics, philosophy of science, and metaphysics. He also explores the foundations of AI, philosophy of mind, decision theory, formal epistemology, philosophy of mathematics, philosophy of religion, and Chinese philosophy. Additionally, he has a side interest in using films to popularize philosophical ideas. Currently, he is co-writing a screenplay about a time-travel romance, inspired by a fascinating article in the Stanford Encyclopedia of Philosophy (SEP). Professor Eddy Chen won the Popper Prize for his 2021 BJPS paper on quantum mechanics in a time-asymmetric universe. (Click here for a short summary.) His works on laws of nature have been published in Nature, featured as a cover story of New Scientist, discussed in Scientific American, and awarded an APA Public Philosophy Op-Ed Prize. His paper on the vagueness of physical laws (The Philosophical Review) was selected by The Philosopher's Annual as "one of the ten best articles in philosophy from 2022." He has participated in a brief interview with the American Philosophical Association (APA) Blog, a longer interview with Richard Marshall at 3:16AM, and a YouTube podcast on laws of nature with Barry Loewer and Curt Jaimungal. He serves as Co-PI of a new Templeton grant on quantum foundations and was a collaborator on a previous project on the quantum arrows of time. Professor Eddy Chen earned a Ph.D. in philosophy, an M.Sc. in mathematics, and a graduate certificate in cognitive science from Rutgers University, New Brunswick, NJ, in 2019. For full biography of Prof. Chen, please refer to: https://www.eddykemingchen.net/
Moderator
Prof Boris BABIC
Associate Professor, HKU IDS, Dept of Philosophy & Law(by courtesy), HKU Professor Boris Babic is HKU-100 Associate Professor at the University of Hong Kong, jointly appointed in the Musketeers Foundation Institute of Data Science, the Department of Philosophy, and (by courtesy) the Faculty of Law, from Fall 2023. He also serves as an occasional visiting professor in the Decision Sciences department at INSEAD. Professor Babic received a JD, cum laude, from Harvard Law School, an MS in Statistics and a PhD in Philosophy, from the University of Michigan, Ann Arbor. He also practiced law at Quinn Emanuel Urquhart & Sullivan, LLP in Los Angeles, USA. He completed his postdoctoral fellowship at the California Institute of Technology (Caltech). Professor Babic’s primary research interests are in Bayesian inference and decision-making, ethics, law, and policy of artificial intelligence and machine learning, especially in medical applications. His research has been published extensively in leading journals such as Science, Nature Machine Intelligence, Nature Digital Medicine, and the Harvard Business Review. Professor Babic will be on leave from the University of Toronto, where he is Assistant Professor in the Department of Statistics and the Department of Philosophy and a faculty fellow of the Schwartz Reisman Institute for Technology & Society. For full biography of Prof. Babic, please refer to: https://datascience.hku.hk/people/boris-babic/
Moderator
Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science
Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.
Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/
