HKU IDS Scholar Seminar Series #9: InfoBodied AI: Learning Mutual Information for Embodied AI

Title: InfoBodied AI: Learning Mutual Information for Embodied AI
Speaker: Prof. Yanchao Yang, Assistant Professor, IDS & Department of Electrical and Electronic Engineering, HKU
Moderator: Prof. Yi Ma, Director of HKU IDS; Professor, Chair of Artificial Intelligence, HKU
Date: March 26, 2024
Time: 10:00am – 11:00am

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

Embodied AI strives to create agents capable of learning and tackling complex tasks involving physical interactions, with potential applications in many areas, such as housekeeping, caregiving, and logistics. Such agents must be able to perceive their environment, construct scene representations, and carry out reasoning and actions to accomplish task-specific goals. However, existing learning approaches rely on human annotations or unrealistic simulations, leading to generalization problems in the real world. Thus, it is crucial to equip embodied agents with the ability to autonomously learn from real-world data, minimizing reliance on human supervision and enabling adaptability to new tasks. We propose that the key to autonomous learning of embodied agents is the mutual correlations in the unlabeled data. In this presentation, we will talk about how we can efficiently compute mutual correlations of data by developing novel neural estimators. We will also show how these freely available mutual correlations can help reduce human annotation effort in learning label-efficient perception, scene representation, and grounded concepts for generalizable interaction policies. We hope leveraging mutual correlations in the unlabeled data will empower embodied agents to learn in unseen environments and automatically acquire novel interaction skills.

Speaker

Prof. Yanchao Yang
Assistant Professor @ HKU IDS & Department of Electrical and Electronic Engineering

Dr Yanchao Yang is an Assistant Professor at HKU with the Department of Electrical and Electronic Engineering and the HKU Musketeers Foundation Institute of Data Science (HKU IDS). He does research in Embodied AI and is interested in self-supervised and semi-supervised techniques that allow autonomous agents to learn at low-annotation regimes. His long-term goal is to design learning algorithms that enable autonomous agents to continuously acquire object-centric representations through active perception with multimodal signals for compositional scene understanding and reasoning. His recent effort is to leverage information-theoretic tools to promote label-efficient perception, scene representation, and policy learning for embodied intelligence.

For full biography of Dr. Yang, please refer to: https://datascience.hku.hk/people/yanchao-yang/

Moderator

Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science 

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk

HKU IDS Scholar Prof Chao Huang and His Collaborators Have Been Awarded “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences”!

HKU IDS Scholar Prof Chao Huang and His Collaborators Have Been Awarded “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences”!

We are very proud to announce that HKU IDS scholar, Prof Chao Huang, Assistant Professor in our Institute and Computer Science, and his collaborators, have been selected by the International Congress of Basic Science as the recipients to the prestigious “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences” for their paper on graph neural network.

The Frontiers of Science Award was first inaugurated in 2023 under the auspices of the International Congress of Basic Sciences (ICBS), and sponsored by the City of Beijing and the Yanqi Lake Beijing Institute of Mathematical Sciences and Application (BIMSA). An FSA is awarded to a recent paper, recognized for a major breakthrough in its field. More information can be found at https://www.icbs.cn/.

Prof Chao Huang’s research interests cover data mining and machine learning. For more information about Prof Huang, please browse: https://datascience.hku.hk/people/chao-huang/

We would like to extend our warmest congratulations to Prof Huang, and the other authors of the paper, for their outstanding achievement. Please read the paper here:  https://doi.org/10.1145/3292500.3330961

 

Part-time Research Assistant I / Research Assistant II

Careers at IDS

Part-time Research Assistant I / Research Assistant II (Professor Alec Kirkley)

Position  

Applications are invited for appointment as a Part-time Research Assistant I/II at the Musketeers Foundation Institute of Data Science at the University of Hong Kong. Appointment will commence as soon as possible for six months, with the possibility of renewal subject to satisfactory performance and funding availability.

Applicants should possess a Bachelor’s or above in Computer Science or related disciplines, preferably with software engineering experience. They should also have a good command of written and spoken English. Preference will be given to those with prior experience developing open-source software packages in Python.

The appointee will assist the Principal Investigator, Professor Alec Kirkley from HKU IDS (https://datascience.hku.hk/people/alec-kirkley/) in developing an open-source software package in Python for complex network inference and machine learning methods. The appointee will be responsible for the organization, optimization, and maintenance of the open-source software as well as developing tutorials for its usage and performing any other duties as assigned. Shortlisted candidates will be invited to attend an interview.

A highly competitive salary commensurate with qualifications and experience will be offered.

Application

The University only accepts online application for the above post. Applicants should apply online and upload an up-to-date CV. Review of applications will commence immediately and continue until April 30, 2024, or until the post is filled, whichever is earlier.

Apply Now

HKU IDS Researchers Prof Ping Luo and Prof Reynold Cheng Received HKU Excellence Awards 2023!

HKU IDS Researchers Prof Ping Luo & Prof Reynold Cheng Have Been Honoured with HKU Excellence Awards 2023!

HKU IDS is very pleased to announce that researchers from the Institute have been recipients of the prestigious HKU Excellence Awards 2023. Professor Ping Luo, our Associate Director (Innovation and Outreach), has been awarded the Outstanding Young Researcher Award 2023, and Professor Reynold Cheng, our Steering Committee Member, has received the Outstanding Research Student Supervisor Award 2023!
 
The HKU Excellence Awards are a salute to the exceptional and inspirational accomplishments of academic staff at the University, and a celebration of the diverse range of disciplines and specialisations at the University.
 
For more information about the respective professors, please refer to the below:

Executive Officer / Senior Executive Assistant (at the rank of Clerk I)

Careers at IDS

Executive Officer / Senior Executive Assistant (At the rank of Clerk I) 

Position  

Applications are invited for the appointment as an Executive Officer / Senior Executive Assistant (at the rank of Clerk I) in the HKU Musketeers Foundation Institute of Data Science (HKU IDS) (Job Ref.: 525940), (to commence as soon as possible, on a two-year fixed-term basis with contract-end gratuity and University contribution to a retirement benefits scheme, totalling up to 10% of basic salary, with the possibility of renewal subject to satisfactory performance).

Qualification  

Applicants should possess a good Bachelor’s degree or equivalent with at least 3 years’ work experience within the context of tertiary education institutions, preferably in the areas of human resources, finance, and research schemes administration. They should have an excellent command of written and spoken English and Chinese (including Putonghua), good communication and problem-solving skills, and an appropriate level of IT literacy. They should be attentive to details, self-motivated with a strong sense of responsibility, capable of working under pressure, as well as a good team player who is willing to take on new challenges. Experience in Taught Postgraduate (TPg) programmes will be an advantage.

The appointee should be strong in accounting and possess relevant experience in human resources. He/she will provide a wide spectrum of administrative support to the Institute, including but not limited to the following:

  • managing financial and HR matters of the Institute; 
  • assisting in the administration of research and knowledge exchange programmes including but not limited to the execution of research grants and collaboration projects with the Institute’s partners;
  • rendering secretarial support and serving in committees and some ad-hoc projects / events and conferences; and
  • supervising a small team under Assistant Director (Administration) in supporting financial and general office administration and projects management, and performing any other duties as assigned. 

Candidates with less experience will be considered for the appointment as Executive Officer. Shortlisted candidates will be invited to attend a written test and interview.

 

 Candidates without a bachelor’s degree yet with solid experiences in tertiary education will be considered for the position of Senior Executive Assistant (at the rank of Clerk I). Shortlisted candidates will be invited to attend a written test and interview.

A highly competitive salary commensurate with qualifications and experience will be offered, in addition to annual leave and medical benefits.

Application

The University only accepts online application for the above post. Applicants should apply online and upload an up-to-date C.V. Review of applications will commence as soon as possible and continue until March 31, 2024, or until the post is filled, whichever is earlier. 

Apply Now

IDS Guest Seminar: Towards Helpful, Honest, and Harmless Information Communication with Human-Model Alignment and Multi-Cultural Understanding

IDS Guest Seminar - by Ms. Yi Ren FUNG

Title: Towards Helpful, Honest, and Harmless Information Communication with Human-Model Alignment and Multi-Cultural Understanding

Speaker: Yi Ren Fung, Ph.D. Candidate, Dept of Computer Science, University of Illinois Urbana-Champaign

Date: Feb 26, 2024
Time: 2:00am – 3:00pm
Venue: HKU IDS, P307, Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Abstract

In recent years, language and multimedia models have made significant advancements, achieving remarkable performance on a large variety of tasks, including question answering, summarization, scientific reasoning, procedural understanding, and action planning, along with demonstrating robust zero-shot/few-shot capabilities, bolstered by model scaling and innovative training techniques. Despite the exciting progress, ensuring these models align with fundamental constitutional principles remains a critical challenge. In this talk, we present a roadmap for foundation models as digital agents that are not only technologically advanced and functionally robust, but also sociocultural-aware and ethically grounded. We begin by delving into advanced mechanisms (i.e., the InfoSurgeon architecture) for identifying information inconsistencies in textual or multimedia content, along with novel strategies to counter the undesirable phenomena of generative model hallucination. Moreover, we introduce norm discovery with self-verification on-the-fly (i.e., the NormSAGE framework) as a promising solution for the explainable detection of real-world norm violation occurrences and for guiding harmless language model response generations. Particularly, we emphasize the importance of our investigative efforts in massively multicultural knowledge acquisition as a vital component to enrich model understanding of norms across diverse societal groups, ensuring more accurate and respectful human-centered interactions. By addressing these research problems and opportunities, we can help reinforce the relevance and responsibility of these foundational language and multimedia AI models in promoting healthy information communication in our increasingly interconnected world.

Speaker

Ms. Yi Ren Fung
Ph.D Candidate @ University of Illinois Urbana-Champaign 

Yi R. Fung is a final-year PhD candidate in computer science at the University of Illinois Urbana-Champaign, advised by Heng Ji. Her research specialization lies in AI/NLP and computational social science, with a particular focus on addressing the fundamental research questions of human-model alignment and ‘helpful/honest/harmless’ healthy information communication. Yi is one of the first researchers who proposed fine-grained knowledge-element level misinformation detection in multimedia news documents, along with a novel approach of event/entity manipulation for constructing targeted dataset that serves as a benchmark for this important task. In addition, she is the first to formally introduce norm discovery from conversation on-the-fly, and largely extended the scope of multicultural social norm knowledge acquisition for language model human-centered awareness and norm violation detection. 

Yi’s research not only boldly addresses crucial emerging interdisciplinary problem domains, but also pioneers advancements in core NLP reasoning techniques, including multimedia knowledge-guided reasoning and language model prompting-based knowledge elicitation with self-verification mechanisms. Moreover, she has also been a leading student driver of several multi-million dollar national-level grant projects, such as SemaFor and CCU, achieving top scores in the evaluation tasks; received various professional recognition ranging from top-conference Best Demo Paper Award to prestigious academic scholarships; organized timely and relevant tutorials at KDD’22 and AACL’22; served in TA role for three graduate-level CS courses attended in total by ~1000 students; and mentored many junior students who continue on to pursue successful graduate careers.

For information, please contact:
Email: datascience@hku.hk

IDS Guest Seminar: Compositionality in 3D Content Generation

IDS Guest Seminar - by Dr. DAI Bo

Title: Compositionality in 3D Content Generation

Speaker: DAI Bo, Scientist, Shanghai AI Lab

Date: Feb 20, 2024
Time: 2:00am – 3:00pm
Venue: HKU IDS, P307, Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Abstract

As the advance of generative AI, 3D content generation has received growing attention. In this talk, we will introduce our recent exploration on exploting and embedding the natural compositionality of 3D content to overcome challenges in scalability, fidelity, controllability, and accessibility. In particular, we will cover AnimateDiff for video generation, LandMark for city-scale rendering and beyond, and COMODO for 3d human motion generation.

Speaker

Dr. Bo Dai
Scientist @ Shanghai AI Lab
DDr. Bo Dai is a scientist with Shanghai AI Laboratory, leading a research group on Content Generation and Digitization [https://idc-sh.github.io/]. Prior to joining Shanghai AI Lab, he was a Research Assistant Professor with S-Lab, Nanyang Technological University, Singapore. He received his Ph.D. (2014-2018) from Multimedia Laboratory (MMLab) at CUHK, advised by Prof.Dahua Lin. He obtained his B.Eng. (2010-2014) from ACM honored class of SJTU. His main research focus is generative AI and its applications, with recent interests in 3D and dynamic generative models and their applications. He has published over 50 papers in top-tier AI conferences and journals, with a citation of 5083 according to Google Scholar.

For information, please contact:
Email: datascience@hku.hk

IDS Guest Seminar: Towards General and Intelligent Autonomous Agent and System

IDS Guest Seminar - by Dr. Hongyang Li

Title: Towards General and Intelligent Autonomous Agent and System

Speaker: Hongyang Li, Research Scientist and PI at OpenDriveLab, Shanghai AI Lab

Date: Feb 19, 2024
Time: 2:30am – 3:30pm
Venue: HKU IDS, P307, Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Abstract

End-to-end autonomous driving (E2E AD) has become a popular hit recently. Compared to conventional modular based design, the main advantages of E2E AD descend from the global optimization across sub-modules in the system, and the unanimous objective towards planning and control task. Some leading corporations from industry have also adopted such an end-to-end philosophy and rolled out to customers on the product end. In this talk, we will first walkthrough the key milestones and roadmap of E2E AD, including the first prototypical work UniAD, proposed by OpenDriveLab and recognized as IEEE CVPR 2023 Best Paper. Then the pivotal challenges (generalization, world model, etc.) that the community currently are confronted with are discussed. With the prevalence of foundation models, how to utilize LLM/VLM techniques and distill common knowledge from experienced human experts are covered as well.The talk would be concluded with some future perspectives on building towards a general and intelligent system in a wide span of downstream applications for the next couple years.

Speaker

Dr. Hongyang Li
Research Scientist and PI @ OpenDriveLab, Shanghai AI Lab

Dr. Hongyang Li is a full-time Research Scientist and PI at OpenDriveLab, Shanghai AI Lab. His research focus is on autonomous driving and embodied AI. He obtained the Ph.D. degree from The Chinese University of Hong Kong in 2019 and worked a few years at industry on L2 assistant autonomous driving. In 2021, he built and led the OpenDriveLab team at Shanghai AI Lab. He proposed the bird’s-eye-view perception work, BEVFormer, that won Top 100 AI Papers in 2022 and was recognized by Jensen Huang, CEO of NVIDIA in a Keynote at CompteX 2023. He lead the end-to-end autonomous driving project, UniAD and won the IEEE CVPR 2023 Best Paper Award. UniAD has a large impact both in academia and industry, including the recent rollout to customers by Tesla in FSD V12 version. He is the Notable Area Chair at NeurIPS 2023, the Senior Member of IEEE. For more details, please visit https://opendrivelab.com is a fellow of the IEEE and a distinguished member of the
ACM.

For information, please contact:
Email: datascience@hku.hk

1st Conference on Parsimony and Learning (CPAL) was successfully held at HKU on January 3 – 6, 2024!

1st Conference on Parsimony and Learning (CPAL) was successfully held at HKU on January 3 - 6, 2024!

On January 3-6, 2024, HKU Musketeers Foundation Institute of Data Science (HKU IDS) proudly hosted and sponsored the inaugural Conference on Parsimony and Learning (CPAL) at The University of Hong Kong. This annual research meeting, set foot in the newly established data science research institute for its first gathering, is dedicated to exploring the parsimonious, low dimensional structures prevalent in machine learning, signal processing, optimization, and more.

 

The Conference has invited internationally renowned data scientists from prestigious universities all over the world. Their vision is to create a scientific forum where faculty, students, industrial partners, and researchers, from various fields can share insights and work towards a unified theoretical and computational framework for understanding intelligence and science from the perspective of parsimonious learning. Over 200 participants attended the conference the event, during which insightful lectures by our 9 Keynote Speakers coming from prestigious institutions such as UCLA, University of Pennsylvania, New York University, KAIST, and more, 30 diversified presentations by researchers with accepted papers as well as 16 Rising Stars Awardees who are early-career scholars, 2 Half-day Tutorials by area experts in machine learning and artificial intelligence, and countless fruitful discussions, were featured.

Day 1 of the event, which included a glamorous Opening Ceremony in the early morning of January 3 ,2024, was graced by Guests of Honours including Prof Xiang Zhang, President and Vice-Chancellor of The University of Hong Kong, Mr Ka-chai Leong, benefactor of the HKU IDS from The Musketeers Education and Culture Charitable Foundation Ltd, Prof Yi Ma, one of the CPAL2024’s General Chair & Director as well Professor, Chair of Artificial Intelligence, HKU IDS & Department of Computer Science, and Prof Harry Shum, another CPAL2024’s General Chair & IAS Professor-at-Large Emeritus, The Hong Kong University of Science and Technology. CPAL is meant to be a new chapter in conferences which emphasizes on true academic values as both the senior and rising researchers are rendered a chance to exchange latest research ideas and interdisciplinary knowledge in data science.

 

Apart from the main programme of the conference at daytime, HKU IDS also shared joy with conference attendees in organizing social and networking events including a dinner banquet at a traditional Chinese-style restaurant on January 4, 2024 evening, and a unique Ding-ding tour on 3 specially designed party trams on January 5, 2024 evening. The attendees, after being immersed in an engaging, yet fairly tight, schedule at the conference on parsimonious learning, got a chance to taste the cultural distinctiveness of the city of Hong Kong through cuisine and entertainment, and also to interact further with other collaborators in a light-hearted manner. Both events were received with positive feedback.

 

In general, CPAL2024 was met with enthusiastic responses. The team at HKU IDS echoes Professor Yi Ma’s conference vision of trying to “make a different conference” which could render better service to the research communities, and looks forward to carrying out other knowledge exchange initiatives at the Institute in an international scale.