IDS Seminar: Principles of Reasoning: Designing Compositional and Collaborative Generative AIs

Host: HKU Musketeers Foundation Institute of Data Science
Co-host: Department of Computer Science, HKU

IDS Seminar - by Prof. William Wang

Title: Principles of Reasoning: Designing Compositional and Collaborative Generative AIs

Speaker: William Wang, Mellichamp Professor of Artificial Intelligence; Director of the UCSB Center for Responsible Machine Learning

Date: Apr 18, 2024
Time: 11:00am – 12:00pm
Venue: HKU IDS Seminar Room (P603), Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Abstract

A majority of existing research in large language models and generative AI systems focuses on scaling and engineering. In this talk, I argue that we need a principled understanding of the science of generative AI, in particular, to understand the reasoning ability of large language models. First, I present a Bayesian latent variable approach to enhancing in-context learning in large language models (LLMs) through optimal demonstration selection, demonstrating substantial improvements across various text classification tasks. Second, I argue that modern generative AI systems must be modular and collaborative to solve complex reasoning problems. I will introduce Logic-LM, a locally grounded neuro-symbolic framework that synergizes LLMs with symbolic solvers, significantly boosting logical problem-solving abilities. We will also briefly elaborate on how to build neuro-symbolic solutions to improve the compositionality in text-to-image systems. Our observations indicate that the future of generative AI is modular and collaborative, as opposed to a single-model system.

Speaker

Prof. William Wang 
Mellichamp Professor of Artificial Intelligence; Director of the Center for Responsible Machine Learning @ UC Santa Barbara

William Wang is Mellichamp Professor of Artificial Intelligence and Director of the UCSB Center for Responsible Machine Learning, UCSB Mind and Machine Intelligence Initiative, and the UCSB Natural Language Processing Group. His Ph.D. was from Carnegie Mellon University. His interests include the science of large language models and generative AIs, vision and language, neuro-symbolic reasoning, and responsible AI. He was recognized with several awards, including the Pierre-Simon Laplace Award by IEEE SPS (2024), the CRA Undergraduate Research Faculty Mentoring Award (2023), the British Computer Society – Karen Spärck Jones Award (2022), and the NSF CAREER Award in 2021. Dr. Wang was also listed among IEEE AI’s 10 to Watch in 2020 and has received accolades for his research, including the CVPR Best Student Paper Award in 2019 and the DARPA Young Faculty Award in 2018.

Moderator

Professor Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis.

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk

IDS Distinguished Speaker Series #5: The Future of AI & Entrepreneurship – Direct from Silicon Valley

IDS Distinguished Speaker Series #5 - Captain Hoff from Founders Space

Host: HKU Musketeers Foundation Institute of Data Science
Co-Host: HKU Techno-Entrepreneurship Core

Title: The Future of AI & Entrepreneurship – Direct from Silicon Valley
Speaker: Mr. Steven Hoffman (Captain Hoff), Chairman & CEO, Founders Space
Moderator: Prof. Yi Ma, Director of HKU IDS; Professor, Chair of Artificial Intelligence, HKU
Date: April 12, 2024
Time: 5:00pm – 6:15pm HKT
Venue: Rayson Huang Theatre / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

See the latest breakthroughs in AI from Silicon Valley and how entrepreneurs are leveraging the power of generative AI to transform the world. Steve Hoffman (Captain Hoff) will show you how AI is set to upend every aspect of our lives and society, as he takes us on a journey into the future.

Speaker

Mr Steven Hoffman (Captain Hoff)
Chairman & CEO @ Founders Space
 

Steven Hoffman (Captain Hoff) is an expert on AI business strategy and author of three award-winning books: “The Five Forces” (AI) published by BenBella and distributed by Penguin Random House; “Make Elephants Fly” (innovation) published by Hachette; “Surviving a Startup” (entrepreneurship) published by HarperCollins.

 

Hoffman is also a venture investor and Chairman & CEO of Founders Space, a global startup accelerator and innovation hub, with over 50 partners in 22 countries. Founders Space was ranked #1 accelerator for international startups by Forbes and Entrepreneur Magazines.

Hoffman has helped hundreds of entrepreneurs launch their companies in Silicon Valley and around the globe. He has also advised corporations on innovation strategy, including Qualcomm, Huawei, Bosch, Intel, Disney, Warner Brothers, Ad Age, MetLife, NBC, A&E, Siemens, Viacom, Turner, Gulf Oil, etc.

Prior to this, Hoffman worked as a TV development executive at Fries Entertainment, which produced over a hundred TV shows, movies, and mini-series that were acquired by MGM. He went on to pioneer interactive television with his venture-funded startup, Spiderdance, which produced interactive TV shows with NBC, MTV, Turner, Warner Brothers, History Channel, Game Show Network, and others.

While in Hollywood, Hoffman served on the Board of Governors of the Producers Guild of America’s New Media Council. He was also founder and Chairman of the Producers Guild Silicon Valley Chapter and a founding member of the Academy of Television’s Interactive Media Group.

As a serial entrepreneur, Hoffman combined new technologies with entertainment, producing groundbreaking games and interactive shows, like WebRIOT, No Boundaries, Weakest Link, Yumby, and RocketOn. He was also Head of Infospace’s Mobile Games Studio, whose play-for-prizes games included Tetris, Wheel of Fortune, Tomb Raider, Thief, Hitman, Skee-Ball, X-Files, and more.

Hoffman earned a bachelor’s degree in computer engineering from the University of California and a master’s degree in film and television from the University of Southern California. He currently resides in California but spends most of his time helping entrepreneurs and innovators around the world bring their dreams to life.

Moderator

Professor Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science 

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk

HKU IDS Scholar Seminar Series #9: InfoBodied AI: Learning Mutual Information for Embodied AI

Title: InfoBodied AI: Learning Mutual Information for Embodied AI
Speaker: Prof. Yanchao Yang, Assistant Professor, IDS & Department of Electrical and Electronic Engineering, HKU
Moderator: Prof. Yi Ma, Director of HKU IDS; Professor, Chair of Artificial Intelligence, HKU
Date: March 26, 2024
Time: 10:00am – 11:00am

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

Embodied AI strives to create agents capable of learning and tackling complex tasks involving physical interactions, with potential applications in many areas, such as housekeeping, caregiving, and logistics. Such agents must be able to perceive their environment, construct scene representations, and carry out reasoning and actions to accomplish task-specific goals. However, existing learning approaches rely on human annotations or unrealistic simulations, leading to generalization problems in the real world. Thus, it is crucial to equip embodied agents with the ability to autonomously learn from real-world data, minimizing reliance on human supervision and enabling adaptability to new tasks. We propose that the key to autonomous learning of embodied agents is the mutual correlations in the unlabeled data. In this presentation, we will talk about how we can efficiently compute mutual correlations of data by developing novel neural estimators. We will also show how these freely available mutual correlations can help reduce human annotation effort in learning label-efficient perception, scene representation, and grounded concepts for generalizable interaction policies. We hope leveraging mutual correlations in the unlabeled data will empower embodied agents to learn in unseen environments and automatically acquire novel interaction skills.

Speaker

Prof. Yanchao Yang
Assistant Professor @ HKU IDS & Department of Electrical and Electronic Engineering

Dr Yanchao Yang is an Assistant Professor at HKU with the Department of Electrical and Electronic Engineering and the HKU Musketeers Foundation Institute of Data Science (HKU IDS). He does research in Embodied AI and is interested in self-supervised and semi-supervised techniques that allow autonomous agents to learn at low-annotation regimes. His long-term goal is to design learning algorithms that enable autonomous agents to continuously acquire object-centric representations through active perception with multimodal signals for compositional scene understanding and reasoning. His recent effort is to leverage information-theoretic tools to promote label-efficient perception, scene representation, and policy learning for embodied intelligence.

For full biography of Dr. Yang, please refer to: https://datascience.hku.hk/people/yanchao-yang/

Moderator

Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science 

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk

HKU IDS Scholar Prof Chao Huang and His Collaborators Have Been Awarded “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences”!

HKU IDS Scholar Prof Chao Huang and His Collaborators Have Been Awarded “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences”!

We are very proud to announce that HKU IDS scholar, Prof Chao Huang, Assistant Professor in our Institute and Computer Science, and his collaborators, have been selected by the International Congress of Basic Science as the recipients to the prestigious “2024 Frontiers of Science Award in Theoretical Computer and Information Sciences” for their paper on graph neural network.

The Frontiers of Science Award was first inaugurated in 2023 under the auspices of the International Congress of Basic Sciences (ICBS), and sponsored by the City of Beijing and the Yanqi Lake Beijing Institute of Mathematical Sciences and Application (BIMSA). An FSA is awarded to a recent paper, recognized for a major breakthrough in its field. More information can be found at https://www.icbs.cn/.

Prof Chao Huang’s research interests cover data mining and machine learning. For more information about Prof Huang, please browse: https://datascience.hku.hk/people/chao-huang/

We would like to extend our warmest congratulations to Prof Huang, and the other authors of the paper, for their outstanding achievement. Please read the paper here:  https://doi.org/10.1145/3292500.3330961

 

Part-time Research Assistant I / Research Assistant II

Careers at IDS

Part-time Research Assistant I / Research Assistant II (Professor Alec Kirkley)

Position  

Applications are invited for appointment as a Part-time Research Assistant I/II at the Musketeers Foundation Institute of Data Science at the University of Hong Kong. Appointment will commence as soon as possible for six months, with the possibility of renewal subject to satisfactory performance and funding availability.

Applicants should possess a Bachelor’s or above in Computer Science or related disciplines, preferably with software engineering experience. They should also have a good command of written and spoken English. Preference will be given to those with prior experience developing open-source software packages in Python.

The appointee will assist the Principal Investigator, Professor Alec Kirkley from HKU IDS (https://datascience.hku.hk/people/alec-kirkley/) in developing an open-source software package in Python for complex network inference and machine learning methods. The appointee will be responsible for the organization, optimization, and maintenance of the open-source software as well as developing tutorials for its usage and performing any other duties as assigned. Shortlisted candidates will be invited to attend an interview.

A highly competitive salary commensurate with qualifications and experience will be offered.

Application

The University only accepts online application for the above post. Applicants should apply online and upload an up-to-date CV. Review of applications will commence immediately and continue until April 30, 2024, or until the post is filled, whichever is earlier.

Apply Now

HKU IDS Researchers Prof Ping Luo and Prof Reynold Cheng Received HKU Excellence Awards 2023!

HKU IDS Researchers Prof Ping Luo & Prof Reynold Cheng Have Been Honoured with HKU Excellence Awards 2023!

HKU IDS is very pleased to announce that researchers from the Institute have been recipients of the prestigious HKU Excellence Awards 2023. Professor Ping Luo, our Associate Director (Innovation and Outreach), has been awarded the Outstanding Young Researcher Award 2023, and Professor Reynold Cheng, our Steering Committee Member, has received the Outstanding Research Student Supervisor Award 2023!
 
The HKU Excellence Awards are a salute to the exceptional and inspirational accomplishments of academic staff at the University, and a celebration of the diverse range of disciplines and specialisations at the University.
 
For more information about the respective professors, please refer to the below:

Executive Officer / Senior Executive Assistant (at the rank of Clerk I)

Careers at IDS

Executive Officer / Senior Executive Assistant (At the rank of Clerk I) 

Position  

Applications are invited for the appointment as an Executive Officer / Senior Executive Assistant (at the rank of Clerk I) in the HKU Musketeers Foundation Institute of Data Science (HKU IDS) (Job Ref.: 525940), (to commence as soon as possible, on a two-year fixed-term basis with contract-end gratuity and University contribution to a retirement benefits scheme, totalling up to 10% of basic salary, with the possibility of renewal subject to satisfactory performance).

Qualification  

Applicants should possess a good Bachelor’s degree or equivalent with at least 3 years’ work experience within the context of tertiary education institutions, preferably in the areas of human resources, finance, and research schemes administration. They should have an excellent command of written and spoken English and Chinese (including Putonghua), good communication and problem-solving skills, and an appropriate level of IT literacy. They should be attentive to details, self-motivated with a strong sense of responsibility, capable of working under pressure, as well as a good team player who is willing to take on new challenges. Experience in Taught Postgraduate (TPg) programmes will be an advantage.

The appointee should be strong in accounting and possess relevant experience in human resources. He/she will provide a wide spectrum of administrative support to the Institute, including but not limited to the following:

  • managing financial and HR matters of the Institute; 
  • assisting in the administration of research and knowledge exchange programmes including but not limited to the execution of research grants and collaboration projects with the Institute’s partners;
  • rendering secretarial support and serving in committees and some ad-hoc projects / events and conferences; and
  • supervising a small team under Assistant Director (Administration) in supporting financial and general office administration and projects management, and performing any other duties as assigned. 

Candidates with less experience will be considered for the appointment as Executive Officer. Shortlisted candidates will be invited to attend a written test and interview.

 

 Candidates without a bachelor’s degree yet with solid experiences in tertiary education will be considered for the position of Senior Executive Assistant (at the rank of Clerk I). Shortlisted candidates will be invited to attend a written test and interview.

A highly competitive salary commensurate with qualifications and experience will be offered, in addition to annual leave and medical benefits.

Application

The University only accepts online application for the above post. Applicants should apply online and upload an up-to-date C.V. Review of applications will commence as soon as possible and continue until March 31, 2024, or until the post is filled, whichever is earlier. 

Apply Now

IDS Guest Seminar: Towards Helpful, Honest, and Harmless Information Communication with Human-Model Alignment and Multi-Cultural Understanding

IDS Guest Seminar - by Ms. Yi Ren FUNG

Title: Towards Helpful, Honest, and Harmless Information Communication with Human-Model Alignment and Multi-Cultural Understanding

Speaker: Yi Ren Fung, Ph.D. Candidate, Dept of Computer Science, University of Illinois Urbana-Champaign

Date: Feb 26, 2024
Time: 2:00am – 3:00pm
Venue: HKU IDS, P307, Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Register Now

Abstract

In recent years, language and multimedia models have made significant advancements, achieving remarkable performance on a large variety of tasks, including question answering, summarization, scientific reasoning, procedural understanding, and action planning, along with demonstrating robust zero-shot/few-shot capabilities, bolstered by model scaling and innovative training techniques. Despite the exciting progress, ensuring these models align with fundamental constitutional principles remains a critical challenge. In this talk, we present a roadmap for foundation models as digital agents that are not only technologically advanced and functionally robust, but also sociocultural-aware and ethically grounded. We begin by delving into advanced mechanisms (i.e., the InfoSurgeon architecture) for identifying information inconsistencies in textual or multimedia content, along with novel strategies to counter the undesirable phenomena of generative model hallucination. Moreover, we introduce norm discovery with self-verification on-the-fly (i.e., the NormSAGE framework) as a promising solution for the explainable detection of real-world norm violation occurrences and for guiding harmless language model response generations. Particularly, we emphasize the importance of our investigative efforts in massively multicultural knowledge acquisition as a vital component to enrich model understanding of norms across diverse societal groups, ensuring more accurate and respectful human-centered interactions. By addressing these research problems and opportunities, we can help reinforce the relevance and responsibility of these foundational language and multimedia AI models in promoting healthy information communication in our increasingly interconnected world.

Speaker

Ms. Yi Ren Fung
Ph.D Candidate @ University of Illinois Urbana-Champaign 

Yi R. Fung is a final-year PhD candidate in computer science at the University of Illinois Urbana-Champaign, advised by Heng Ji. Her research specialization lies in AI/NLP and computational social science, with a particular focus on addressing the fundamental research questions of human-model alignment and ‘helpful/honest/harmless’ healthy information communication. Yi is one of the first researchers who proposed fine-grained knowledge-element level misinformation detection in multimedia news documents, along with a novel approach of event/entity manipulation for constructing targeted dataset that serves as a benchmark for this important task. In addition, she is the first to formally introduce norm discovery from conversation on-the-fly, and largely extended the scope of multicultural social norm knowledge acquisition for language model human-centered awareness and norm violation detection. 

Yi’s research not only boldly addresses crucial emerging interdisciplinary problem domains, but also pioneers advancements in core NLP reasoning techniques, including multimedia knowledge-guided reasoning and language model prompting-based knowledge elicitation with self-verification mechanisms. Moreover, she has also been a leading student driver of several multi-million dollar national-level grant projects, such as SemaFor and CCU, achieving top scores in the evaluation tasks; received various professional recognition ranging from top-conference Best Demo Paper Award to prestigious academic scholarships; organized timely and relevant tutorials at KDD’22 and AACL’22; served in TA role for three graduate-level CS courses attended in total by ~1000 students; and mentored many junior students who continue on to pursue successful graduate careers.

For information, please contact:
Email: datascience@hku.hk

IDS Guest Seminar: Compositionality in 3D Content Generation

IDS Guest Seminar - by Dr. DAI Bo

Title: Compositionality in 3D Content Generation

Speaker: DAI Bo, Scientist, Shanghai AI Lab

Date: Feb 20, 2024
Time: 2:00am – 3:00pm
Venue: HKU IDS, P307, Graduate House / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered. 

Register Now

Abstract

As the advance of generative AI, 3D content generation has received growing attention. In this talk, we will introduce our recent exploration on exploting and embedding the natural compositionality of 3D content to overcome challenges in scalability, fidelity, controllability, and accessibility. In particular, we will cover AnimateDiff for video generation, LandMark for city-scale rendering and beyond, and COMODO for 3d human motion generation.

Speaker

Dr. Bo Dai
Scientist @ Shanghai AI Lab
DDr. Bo Dai is a scientist with Shanghai AI Laboratory, leading a research group on Content Generation and Digitization [https://idc-sh.github.io/]. Prior to joining Shanghai AI Lab, he was a Research Assistant Professor with S-Lab, Nanyang Technological University, Singapore. He received his Ph.D. (2014-2018) from Multimedia Laboratory (MMLab) at CUHK, advised by Prof.Dahua Lin. He obtained his B.Eng. (2010-2014) from ACM honored class of SJTU. His main research focus is generative AI and its applications, with recent interests in 3D and dynamic generative models and their applications. He has published over 50 papers in top-tier AI conferences and journals, with a citation of 5083 according to Google Scholar.

For information, please contact:
Email: datascience@hku.hk