Skip to content

Host

Co-Host

      

HKU IDS Seminar:

Revisiting Vision Transformers and RL-Based Image Editing

Speaker

Prof Yizhou Yu

Associate Director, HKU IDS

Chair Professor, HKU SCDS

Date

19 Nov, 2026 (Thu)

Time

2:00pm – 3:00pm

Venue

Tam Wing Fan Innovation Wing Two, HKU

Mode

On-site. Seats for on-site participants are limited. 

Abstract

Pretrained ViTs provide general-purpose representations for diverse downstream tasks. However, artifacts in ViTs are widely observed across different supervision paradigms. In the first part of the seminar, I present a through analysis of artifacts in ViTs, and conclude that these artifacts originate from a lazy aggregation behavior: ViT uses semantically irrelevant background patches as shortcuts to represent global semantics, driven by global attention and coarse-grained semanticsupervision. Our solution selectively integrates patch features into the CLS token, and substantially improves performance across 12 benchmarks.

Existing image editing reward models usually give overall scores withoutdetailed checks, causing biased rewards. To address this, we move from a simple scorer to a reasoning verifier. In the second part of the seminar, I introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) and then leverages it for image editing. To build such an RRM, we introduce Group Contrastive Preference Optimization (GCPO), a reinforcement learning algorithm that leverages human pairwise preference data to reinforce our pointwise RRM. Extensive experiments demonstrate that our Edit-RRM surpasses powerful VLMs as an editing-specific reward model with a scaling trend from 3B to 7B parameters.

Speaker

Prof Yizhou Yu

Associate Director, HKU Musketeers Foundation Institute of Data Science
Chair Professor, HKU School of Computing and Data Science

Prof Yizhou Yu is an Associate Director of the HKU Musketeers Foundation Institute of Data Science and a Chair Professor at the HKU School of Computing and Data Science. He was also a faculty member at University of Illinois, Urbana-Champaign for more than ten years. He received his PhD degree in computer science from University of California at Berkeley. He is an ACM fellow and IEEE fellow.

For full biography of Prof Yu, please refer to: https://datascience.hku.hk/people/yizhou-yu/

Moderator

To be announced

Prof Boris BABIC

Associate Professor, HKU IDS, Dept of Philosophy & Law(by courtesy), HKU

Professor Boris Babic is HKU-100 Associate Professor at the University of Hong Kong, jointly appointed in the Musketeers Foundation Institute of Data Science, the Department of Philosophy, and (by courtesy) the Faculty of Law, from Fall 2023. He also serves as an occasional visiting professor in the Decision Sciences department at INSEAD.

Professor Babic’s primary research interests are in Bayesian inference and decision-making, ethics, law, and policy of artificial intelligence and machine learning, especially in medical applications. His research has been published extensively in leading journals such as Science, Nature Machine Intelligence, Nature Digital Medicine, and the Harvard Business Review.

For full biography of Prof. Babic, please refer to: https://datascience.hku.hk/people/boris-babic/

Moderator

Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science 

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk