IDS Seminar: Human-AI Interaction in the Age of Large Language Models

Host: HKU Musketeers Foundation Institute of Data Science
Co-host: Department of Computer Science, HKU

IDS Seminar - by Dr. Diyi Yang from Stanford University

Title: Human-AI Interaction in the Age of Large Language Models

Speaker: Dr. Diyi Yang, Assistant Professor, Department of Computer Science, Stanford University
Moderator: Dr. Tao Yu, Assistant Professor, HKU IDS / Department of Computer Science

Date: Dec 11, 2023
Time: 2:30pm – 3:30pm

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

Large language models have revolutionized the way humans interact with AI systems, transforming a wide range of applications and disciplines. In this talk, we discuss several approaches to enhancing human-AI and AI-AI interactions using LLMs. The first one explores how large language models transform computational social science, and how human-AI collaboration can reduce costs and improve the efficiency of social science research. We then introduce efficient machine unlearning techniques to enable LLMs to forget sensitive user data if needed, towards secure and responsible interaction. The last part looks at AI-AI interaction via a dynamic LLM agent network for multi-agent collaboration on complicated reasoning and generation tasks. We conclude by discussing how LLMs enable collaborative intelligence by redefining the interactions between humans and AI systems.

Speaker

Dr. Diyi Yang
Assistant Professor @ Department of Computer Science, Stanford University

Dr. Diyi Yang is an assistant professor in the Computer Science Department at Stanford University, also affiliated with the Stanford NLP Group, Stanford HCI Group, and Stanford Human-Centered Artificial Intelligence (HAI). Diyi received her PhD from Carnegie Mellon University, and her bachelor’s degree from Shanghai Jiao Tong University. Her research focuses on natural language processing, machine learning, and computational social science. Her work has received multiple best paper nominations or awards at top NLP and HCI conferences (e.g., ACL, EMNLP, SIGCHI, ICWSM, and CSCW). She is a recipient of IEEE “AI 10 to Watch” (2020), Intel Rising Star Faculty Award (2021), Samsung AI Researcher of the Year (2021), Microsoft Research Faculty Fellowship (2021), NSF CAREER Award (2022), and an ONR Young Investigator Award (2023).

Moderator

Dr. Tao Yu
Assistant Professor @ HKU IDS & Department of Computer Science

Dr. Tao Yu is an Assistant Professor in the HKU IDS and the Computer Science Department of the University of Hong Kong. He is also a Postdoctoral Research Fellow in the Department of Computer Science and Engineering at University of Washington and a co-director of the NLP group at the University of Hong Kong. His research interest is in Natural Language Processing and Deep Learning, with a focus on designing and building conversational natural language interfaces that can help humans explore and reason over data in any application (e.g., relational databases and mobile apps) in a robust and trusted manner. He has published and served in the program committee at ACL, EMNLP, ICLR, NAACL, etc. He co-organized the Interactive and Executable Semantic Parsing workshop at EMNLP 2020.

For information, please contact:
Email: datascience@hku.hk

1st Conference on Parsimony and Learning (CPAL) will take place in HKU on January 3 – 6, 2024!

1st Conference on Parsimony and Learning (CPAL) will take place in HKU on January 3 - 6, 2024!

Exciting news! The 1st Conference on Parsimony and Learning (CPAL), which is an annual research conference focused on addressing the parsimonious, low dimensional structures that prevail in machine learning, signal processing, optimization, and beyond, will take place between January 3 and 6, 2024, at The University of Hong Kong.

If you are interested in theories, algorithms, applications, hardware and systems, as well as scientific foundations for learning with parsimony, please stay tuned for the prestigious keynote speeches with details here: 

Key Dates & Deadlines

  • August 28, 2023: Submission Deadline for Proceedings Track
  • October 10, 2023: Submission Deadline for Recent Spotlight Track
  • October 14, 2023: 2-Week Rebuttal Stage Starts (Proceedings Track)
  • October 27, 2023: Rebuttal Stage Ends, Authors-Reviewers Discussion Stage Starts (Proceedings Track)
  • November 5, 2023: Authors-Reviewers Discussion Stage Ends (Proceedings Track)
  • November 20, 2023: Final Decisions Released (Both Tracks)
  • December 5, 2023: Camera-Ready Deadline (Both Tracks)
  • December 15, 2023: Registration & Payment Deadline
  • January 3-6, 2024: Main Conference (In-Person at HKU)

Contact Us

Seminar: Biology-inspired network medicine approach to drug discovery

Seminar - by Dr. Qingpeng Zhang

Host: Centre of Cancer Medicine, School of Clinical Medicine, Li Ka Shing Faculty of Medicine, HKU
Co-host: HKU Musketeers Foundation Institute of Data Science

Title: Biology-inspired network medicine approach to drug discovery

Speaker: Dr. Qingpeng Zhang, Associate Professor, HKU IDS / Department of Pharmacology and Pharmacy
Date: November 28, 2023 (Tuesday)
Time:
1:00pm – 2:00pm
Venue: 3SR-SR3, Room 402, 4/F Academic Building, 3 Sassoon Road, Pokfulam (Capacity: 50 – No Registration Required)

Abstract

Drug discovery is a challenging and costly process that requires a deep understanding of the mechanism of drug action (MODA), which is how a drug affects the biological system at the molecular level. In this talk, I will present our recent studies on using a network-based machine learning approach to characterize MODA by analyzing a comprehensive biological network that captures the complex high-dimensional molecular interactions between genes, proteins and chemicals. I will show that our methods outperform state-of-the-art machine learning baselines in predicting MODA. I will also demonstrate that our methods can identify explicit critical paths that are consistent with clinical evidence, and explain how these paths reveal the underlying biological mechanisms of drug action. Our research provides a novel interpretable artificial intelligence perspective on drug discovery, and has the potential to facilitate the development of new and effective drugs.

Speaker

Dr. Qingpeng Zhang
Associate Professor @ HKU IDS / Department of Pharmacology and Pharmacy

Dr. Qingpeng Zhang is an Associate Professor at The University of Hong Kong (HKU), affiliated with the Musketeers Foundation Institute of Data Science and the Department of Pharmacology and Pharmacy. He joined HKU in August 2023, after serving as an Associate Professor at the School of Data Science of The City University of Hong Kong (CityU). He obtained his Ph.D. degree in Systems and Industrial Engineering from the University of Arizona and conducted his postdoctoral research in the Tetherless World Constellation, Department of Computer Science at Rensselaer Polytechnic Institute. He is a senior member of IEEE, and an associate editor for BMJ Mental Health, IEEE TITS, and IEEE TCSS.

His research focuses on medical informatics, AI in drug discovery, healthcare data analytics and network science. He has published in top journals such as Nature Human Behaviour, Nature Communications, PNAS, JAMIA and MIS Quarterly, and his work has been featured in media outlets such as The Washington Post, The New York Times, New York Public Radio, The Guardian and Ming Pao. He has received several awards for his research excellence, including The President’s Award (2022) and the Outstanding Research Award (2021) from CityU and the Andrew P. Sage Best Transactions Paper Award (2021) from IEEE Systems, Man, and Cybernetics Society.

For information, please contact:
Email: datascience@hku.hk

IDS Distinguished Speaker Series #4: Learned Imaging Systems

IDS Distinguished Speaker Series #4 - Professor Wolfgang Heidrich

Host: HKU Musketeers Foundation Institute of Data Science
Co-Host: Department of Computer Science & Department of Electrical and Electronic Engineering, HKU

Title: Learned Imaging Systems
Speaker: Professor Wolfgang Heidrich, Professor of Computer Science and Electrical & Computer Engineering, King Abdullah University of Science and Technology (KAUST) Visual Computing Center
Moderator: Dr Evan Peng, Assistant Professor, Department of Electrical and Electronic Engineering, HKU
Date: Nov 9, 2023
Time: 10:30am – 11:30am
Venue: CPD-LG.18, Centennial Campus / Zoom

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

Computational imaging systems are based on the joint design of optics and associated image reconstruction algorithms. Of particular interest in recent years has been the development of end-to-end learned “Deep Optics” systems that use differentiable optical simulation in combination with backpropagation to simultaneously learn optical design and deep network post-processing for applications such as hyperspectral imaging, HDR, or extended depth of field. In this talk I will in particular focus on new developments that expand the design space of such systems from simple DOE optics to compound refractive optics and mixtures of different types of optical components.

Speaker

Professor Wolfgang Heidrich
Professor of Computer Science and Elecrical & Computer Engineering @ King Abdullah University of Science and Technology (KAUST) Visual Computing Center
Prof. Wolfgang Heidrich is a Professor of Computer Science and Electrical and Computer Engineering in the KAUST Visual Computing Center, for which he also served as director from 2014 to 2021. Prof. Heidrich joined King Abdullah University of Science and Technology (KAUST) in 2014, after 13 years as a faculty member at the University of British Columbia. He received his PhD in from the University of Erlangen in 1999, and then worked as a Research Associate in the Computer Graphics Group of the Max-Planck-Institute for Computer Science in Saarbrucken, Germany, before joining UBC in 2000. Prof. Heidrich’s research interests lie at the intersection of imaging, optics, computer vision, computer graphics, and inverse problems. His more recent interest is in computational imaging, focusing on hardware-software co-design of the next generation of imaging systems, with applications such as High-Dynamic Range imaging, compact computational cameras, hyperspectral cameras, to name just a few. Prof. Heidrich’s work on High Dynamic Range Displays served as the basis for the technology behind Brightside Technologies, which was acquired by Dolby in 2007.
Prof. Heidrich is a Fellow of the IEEE, AAIA, and Eurographics, and the recipient of a Humboldt Research Award as well as the ACM SIGGRAPH Computer Graphics Achievement Award.

Moderator

Dr Evan Peng
Assistant Professor @ Assistant Professor, Department of Electrical and Electronic Engineering, HKU

Dr Evan Peng is an Assistant Professor at The University of Hong Kong. Before this, he was a Postdoctoral Research Scholar in the Computational Imaging Laboratory, Stanford University, and received a PhD in Computer Science from Imager Lab, the University of British Columbia. During the PhD, he was a Visiting Student Researcher at Visual Computing Center, King Abdullah University of Science and Technology. He received both his MSc and BS in Optical Science and Engineering from Zhejiang University.

His research interest lies in the interdisciplinary field of Optics, Graphics, Vision, and Artificial Intelligence, particularly with the focus of : Computational Optics, Photography, Sensing, and Display; Holographic Imaging/Display & VR/AR/MR; Low-level Computer Vision; Inverse Rendering; Human-centered Visual & Sensory Systems.

For information, please contact:
Email: datascience@hku.hk

IDS Distinguished Speaker Series #3: Charting the AI Convergence to Translation Journey

IDS Distinguished Speaker Series #3 - Professor Nitesh Chawla

Title: Charting the AI Convergence to Translation Journey
Speaker: Professor Nitesh Chawla, Frank M. Freimann Professor of Computer Science and Engineering, University of Notre Dame; Director, Lucy Family Institute for Data and Society; ACM & IEEE Fellow
Moderator: Dr Chao Huang, Assistant Professor, HKU IDS / Department of Computer Science
Date: Oct 26, 2023
Time: 5:00pm – 6:00pm

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

AI and data science are at the forefront of several fundamental advances, innovations and potential for societal impact. While there is a pertinent movement in advancing societal applications of data science / AI, there are also challenges in truly realizing that potential. In this presentation, I’ll discuss our experience in navigating research convergence for innovation to translation journey, and also opportunities and challenges to truly realize the potential of data science / AI for society.  

Speaker

Professor Nitesh Chawla
Frank M. Freimann Professor of Computer Science and Engineering, University of Notre Dame; Director, Lucy Family Institute for Data and Society; ACM & IEEE Fellow
Nitesh Chawla is the Frank M. Freimann Professor of Computer Science and Engineering and the Founding Director of the Lucy Family Institute for Data and Society at University of Notre Dame. His research is focused on artificial intelligence, data science, and network science, and is motivated by the question of how technology can advance the common good through interdisciplinary research. He has published more than 300 papers, accumulating over 58,000 citations and an h-index of 78. He is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) and a Fellow of the Association of Computing Machinery (ACM).
He is the recipient of multiple awards including National Academy of Engineers New Faculty Fellowship, IEEE CIS Outstanding Early Career Award, Rodney F. Ganey Community Impact Award, IBM Big Data & Analytics Faculty Award, IBM Watson Faculty Award, and the 1st Source Bank Technology Commercialization Award. He is co-founder of Aunalytics, a data science software and cloud computing company.

Moderator

Dr Chao Huang
Assistant Professor @ HKU IDS & Department of Computer Science 

Dr Chao Huang is a tenure-track assistant professor at the University of Hong Kong. He is a faculty member of the Institute of Data Science and Department of Computer Science. Before that, he was a research scientist at JD Research America in Silicon Valley. He obtained the Ph.D degree from the Computing Science and Engineering Department at University of Notre Dame in United States.

For full biography of Dr Huang, please refer to: https://datascience.hku.hk/people/chao-huang/

For information, please contact:
Email: idatasci@hku.hk

Full-time Office Attendant

Careers at IDS

Full-time Office Attendant 全職辦公室服務員

Position  

Applications are invited for the appointment as a Full-time Office Attendant in the HKU Musketeers Foundation Institute of Data Science in the HKU Musketeers Foundation Institute of Data Science (HKU IDS) (Job Ref.: 523315), (to commence as soon as possible for one year with the possibility of renewal, subject to satisfactory performance).

Qualification & Job Duties

Applicants should have completed Form 3 or above. They should be able to speak and write simple English and Chinese, self-motivated, with the ability to work independently and proactively. The appointee is expected to perform duties, including but not limited to general and students research premises cleaning, preparation of meeting logistics and refreshments, receiving and dispatching mails and documents within and outside campus, and rendering office assistance in events and other Institute’s organized activities. Those with experience in tertiary institutions would be an advantage.

Application

The University only accepts online application for the above post. Applicants should apply online and upload an up-to-date C.V. Review of applications will commence as soon as possible, or until the post is filled, whichever is earlier. 

Apply Now

職位
香港大學同心基金數據科學研究院聘請:全職辦公室服務員 (檔案編號:523315) (以一年臨時合約形式聘用;若工作表現滿意,可獲續聘)

學歷要求及工作範疇
應徵者須具備中三或以上程度,能操流利粵語,略懂英語及普通話,並能閱讀及書寫簡單中、英文更佳。受聘者須積極主動、守時盡責及具獨立處理事務能力。受聘者須負責執行一般雜務,包括清潔打掃辦公室及學生研究中心、準備會議及茶水、外勤、文件派遞、協助處理辦公室事務及為研究院的活動提供支援等。具大專院校工作經驗者將獲優先考慮。

申請方法
大學只接受透過網上系統遞交的申請。應徵者請到大學人才招聘網站遞交網上申請及上載最新的個人履歷。大學會盡快展開遴選工作,截止申請日期:2023年12月31日。 

IDS Seminar: Demystifying Attention Mechanism in Transformer and its application to Better Inference of Large Language Models (LLMs)

IDS Seminar - by Dr. Yuandong Tian from Meta

Title: Demystifying Attention Mechanism in Transformer and its application to Better Inference of Large Language Models (LLMs)
Speaker: Dr. Yuandong Tian, Research Scientist & Senior Manager in Meta AI Research (FAIR)
Moderator: Prof. Yi Ma, Director of HKU IDS; Professor, Chair of Artificial Intelligence, HKU
Date: Sep 26, 2023
Time: 3:00pm – 4:00pm

Mode: Hybrid. Seats for on-site participants are limited. A confirmation email will be sent to participants who have successfully registered.

Abstract

Large Language Models (LLMs) have demonstrated remarkable efficacy across diverse applications, with the multi-layer Transformer architecture and self-attention playing a pivotal role. In this talk, we analyze the training dynamics of self-attention in 1-layer and multi-layer Transformer in a mathematically rigorous manner. This analysis characterizes the training dynamics of self-attention and how tokens are composed to form high-level latent patterns. Our theoretical insights are corroborated by extensive experimental evidence. Notably, one property called “contextual sparsity” enables us to develop novel approaches such as Deja Vu and H2O that substantially accelerate LLM inference. Finally, further study of the attention behavior yields positional interpolation (PI) that extends context window beyond pre-trained models with very few fine-tuning steps.  

Speaker

Dr. Yuandong Tian
Research Scientist & Senior Manager @ Meta AI Research (FAIR)

Dr. Yuandong Tian is a Research Scientist and Senior Manager in Meta AI Research (FAIR), working on reinforcement learning, optimization and understanding of neural networks. He has been the project lead for story generation (2023) and OpenGo project (2018). He is the first-author recipient of 2021 ICML Outstanding Paper Honorable Mentions and 2013 ICCV Marr Prize Honorable Mentions, and also received the 2022 CGO Distinguished Paper Award. Prior to that, he worked in Google Self-driving Car team in 2013-2014 and received a Ph.D in Robotics Institute, Carnegie Mellon University in 2013. He has been appointed as area chairs for NeurIPS, ICML, AAAI and AIStats.

Moderator

Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk

IDS Seminar: Emergence of Segmentation with Minimalistic White-Box Transformers

Title: Emergence of Segmentation with Minimalistic White-Box Transformers
Speaker: Yaodong Yu, PhD Student, EECS Department, University of California, Berkeley
Date: Sep 7, 2023

Time: 4pm
Venue: HKU IDS Office, P307, Graduate House (Registration required)

Abstract

Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown that segmentation properties emerge in vision transformers (ViTs) trained using self-supervised methods such as DINO, but not in those trained on supervised classification tasks. In this study, we probe whether segmentation emerges in transformer-based models solely as a result of intricate self-supervised learning mechanisms, or if the same emergence can be achieved under much broader conditions through proper design of the model architecture. Through extensive experimental results, we demonstrate that when employing a white-box transformer-like architecture known as CRATE, whose design explicitly models and pursues low-dimensional structures in the data distribution, segmentation properties, at both the whole and parts levels, already emerge with a minimalistic supervised training recipe. Layer-wise finer-grained analysis reveals that the emergent properties strongly corroborate the designed mathematical functions of the white-box network. Our results suggest a path to design white-box foundation models that are simultaneously highly performant and mathematically fully interpretable. Code is at https://github.com/Ma-Lab-Berkeley/CRATE

This is a joint work with Tianzhe Chu, Shengbang Tong, Ziyang Wu, Druv Pai, Sam Buchanan, and Yi Ma. 

Speaker

Yaodong Yu
PhD Student @ EECS Department, University of California, Berkeley

Yaodong Yu is a PhD student in the EECS department at UC Berkeley advised by Michael I. Jordan and Yi Ma. He obtained his B.S. from the Department of Mathematics at Nanjing University, and his M.S. from the Department of Computer Science, University of Virginia.

His research interests include topics in machine learning and optimization. His goal is to make machine learning systems more robust.

For information, please contact:
Email: datascience@hku.hk

Welcoming Our First Batch of HKU IDS RPg Students at Orientation Programme!

Warm Welcome to Our First Batch of HKU IDS Research Postgraduate Students (Year 2023-24)

We have proudly received our first batch of research postgraduate students from across the globe at this new academic year 2023-24!

September 6, 2023 was a day to remember as HKU IDS welcomed our RPg students with a range of orientation activities. Exploring the different spots on HKU campus aside, our students were warmly greeted at the HKU IDS office by Professor Yi Ma, the Institute’s Director & Professor, Chair of Artificial Intelligence, HKU IDS Scholars, and our office administrators with an induction programme. Our students had a good chat with their counterparts and advisors during the welcome lunch session.
 

Wishing them all a fruitful and successful academic journey at HKU by joining the HKU IDS research family!

IDS Guest Seminar: Reconstruct a World, Generate a New World

Title: Reconstruct a World, Generate a New World
Speaker: Professor Shenghua Gao, School of Information Science and Technology, ShanghaiTech University 
Date: September 12, 2023
Time: 1:00pm – 2:00pm

Venue: P307, HKU IDS Office, Graduate House

Abstract

We live in a 3D world. When we interact with the environment, objects, and humans—such as walking on a road, grasping a cup, or shaking hands—we are actually aware of the geometric shape of the scenes, objects, and humans. Furthermore, we are actively shaping our environment through architectural design, interior decoration, and the creation of novel objects. These capabilities correspond to the task of 3D reconstruction and generation in the field of computer vision.

Conventional point-matching-based 3D reconstruction methods often stumble when dealing with textureless or repetitively textured surfaces, yielding point clouds that are too sparse to effectively serve the downstream applications. Additionally, single-image-based 3D reconstruction is an ill-posed problem in conventional 3D reconstruction framework. In response to these challenges, we propose the integration of geometric priors concerning scenes and objects into the 3D reconstruction, such as representing a scene as piecewise planar surfaces. We then attempt to merge the geometric priors with a signed distance function-based implicit neural representation for 3D reconstruction. Furthermore, we delve into the realm of 3D image generation with image/text conditions, which offers a potential solution for single-image-based 3D reconstruction. Recognizing that an image is a projection of the 3D world, we contend that the 3D shape priors play an important role for ensuring multi-view consistency and geometric accuracy in 2D image generation. As an example, we will showcase our efforts in tackling human motion imitation, novel view synthesis, and appearance transfer (virtual try-on) within a unified framework by leveraging 3D human representation.

Speaker

Professor Shenghua Gao
Professor @ School of Information Science and Technology, ShanghaiTech University 
Shenghua Gao is a professor at the ShanghaiTech University, China. He received his B.E. degree from University of Science and Technology of China in 2008 and his Ph.D. degree from Nanyang Technological University in 2012. Between June 2012 and August 2014, he worked as a research scientist at UIUC Advanced Digital Sciences Center in Singapore. He joined in ShanghaiTech University in 2014. His research interests include 3D reconstruction, image generation, and video understanding. He has been awarded the Microsoft Research Fellowship, ACM Shanghai Young Research Scientist, Shanghai Excellent Academic Leader, Shanghai Teaching Achievements award and National Young Talents award. He has published over 120 peer-reviewed papers with a total citation count of 14,800+ (source: Google scholar) and an H-index 51. He has served as an area chair for many top-tier AI conferences, including NeurIPS, CVPR, ICCV, and AAAI. He is a publicity chair of CVPR 2024. He is/was an associate editor for IEEE TPAMI(2023-), IEEE TCSVT (2018-2022) and Neurocomputing(2018-).

For information, please contact:
Email: datascience@hku.hk