Skip to content

Host

Co-Host

      

HKU IDS Seminar:

Revisiting Vision Transformers and RL-Based Image Editing

Speaker

Prof Yizhou Yu

Associate Director, HKU IDS

Chair Professor, HKU SCDS

Date

19 Nov, 2026 (Thu)

Time

2:00pm – 3:00pm

Venue

Tam Wing Fan Innovation Wing Two, HKU

Mode

On-site. Seats for on-site participants are limited. 

Abstract

Pretrained ViTs provide general-purpose representations for diverse downstream tasks. However, artifacts in ViTs are widely observed across different supervision paradigms. In the first part of the seminar, I present a through analysis of artifacts in ViTs, and conclude that these artifacts originate from a lazy aggregation behavior: ViT uses semantically irrelevant background patches as shortcuts to represent global semantics, driven by global attention and coarse-grained semanticsupervision. Our solution selectively integrates patch features into the CLS token, and substantially improves performance across 12 benchmarks.

Existing image editing reward models usually give overall scores withoutdetailed checks, causing biased rewards. To address this, we move from a simple scorer to a reasoning verifier. In the second part of the seminar, I introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) and then leverages it for image editing. To build such an RRM, we introduce Group Contrastive Preference Optimization (GCPO), a reinforcement learning algorithm that leverages human pairwise preference data to reinforce our pointwise RRM. Extensive experiments demonstrate that our Edit-RRM surpasses powerful VLMs as an editing-specific reward model with a scaling trend from 3B to 7B parameters.

Speaker

Prof Yizhou Yu

Associate Director, HKU Musketeers Foundation Institute of Data Science
Chair Professor, HKU School of Computing and Data Science

Prof Yizhou Yu is an Associate Director of the HKU Musketeers Foundation Institute of Data Science and a Chair Professor at the HKU School of Computing and Data Science. He was also a faculty member at University of Illinois, Urbana-Champaign for more than ten years. He received his PhD degree in computer science from University of California at Berkeley. He is an ACM fellow and IEEE fellow.

For full biography of Prof Yu, please refer to: https://datascience.hku.hk/people/yizhou-yu/

Moderator

Prof Jinhong Du

Member, HKU Musketeers Foundation Institute of Data Science
Assistant Professor, Department of Statistics and Actuarial Science, HKU School of Computing and Data Science

Prof Jinhong Du is a Member of the HKU Musketeers Foundation Institute of Data Science and an Assistant Professor at the Department of Statistics and Actuarial Science, HKU School of Computing and Data Science.

His research bridges statistical theory and high-impact applications, focusing on causal inference, interpretable machine learning, high-dimensional statistics, and statistical genomics. Dr. Du’s work has been published in leading venues across multiple disciplines, including premier statistics journals like the Journal of the American Statistical Association and the Journal of the Royal Statistical Society, Series B, top machine learning conferences like NeurIPS and ICML, and prominent scientific journals such as the Proceedings of the National Academy of Sciences.

He earned his Ph.D. in Statistics and Machine Learning from Carnegie Mellon University, an M.S. in Statistics from the University of Chicago, and a B.S. in Statistics from Sun Yat-sen University.

For full biography of Prof Du, please refer to: https://datascience.hku.hk/people/jinhong-du/

Moderator

Prof. Yi Ma
Director; Professor, Chair of Artificial Intelligence @ HKU IDS & Department of Computer Science 

Professor Yi Ma is a Chair Professor in the Musketeers Foundation Institute of Data Science (HKU IDS) and Department of Computer Science at the University of Hong Kong. He took up the Directorship of HKU IDS on January 12, 2023. He is also a Professor at the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He has published about 60 journal papers, 120 conference papers, and three textbooks in computer vision, generalized principal component analysis, and high-dimensional data analysis. 

Professor Ma’s research interests cover computer vision, high-dimensional data analysis, and intelligent systems. For full biography of Professor Ma, please refer to: https://datascience.hku.hk/people/yi-ma/

For information, please contact:
Email: datascience@hku.hk