Skip to content
Title: Reconstruct a World, Generate a New World
Speaker: Professor Shenghua Gao, School of Information Science and Technology, ShanghaiTech University 
Date: September 12, 2023
Time: 1:00pm – 2:00pm

Venue: P307, HKU IDS Office, Graduate House


We live in a 3D world. When we interact with the environment, objects, and humans—such as walking on a road, grasping a cup, or shaking hands—we are actually aware of the geometric shape of the scenes, objects, and humans. Furthermore, we are actively shaping our environment through architectural design, interior decoration, and the creation of novel objects. These capabilities correspond to the task of 3D reconstruction and generation in the field of computer vision.

Conventional point-matching-based 3D reconstruction methods often stumble when dealing with textureless or repetitively textured surfaces, yielding point clouds that are too sparse to effectively serve the downstream applications. Additionally, single-image-based 3D reconstruction is an ill-posed problem in conventional 3D reconstruction framework. In response to these challenges, we propose the integration of geometric priors concerning scenes and objects into the 3D reconstruction, such as representing a scene as piecewise planar surfaces. We then attempt to merge the geometric priors with a signed distance function-based implicit neural representation for 3D reconstruction. Furthermore, we delve into the realm of 3D image generation with image/text conditions, which offers a potential solution for single-image-based 3D reconstruction. Recognizing that an image is a projection of the 3D world, we contend that the 3D shape priors play an important role for ensuring multi-view consistency and geometric accuracy in 2D image generation. As an example, we will showcase our efforts in tackling human motion imitation, novel view synthesis, and appearance transfer (virtual try-on) within a unified framework by leveraging 3D human representation.


Professor Shenghua Gao
Professor @ School of Information Science and Technology, ShanghaiTech University 
Shenghua Gao is a professor at the ShanghaiTech University, China. He received his B.E. degree from University of Science and Technology of China in 2008 and his Ph.D. degree from Nanyang Technological University in 2012. Between June 2012 and August 2014, he worked as a research scientist at UIUC Advanced Digital Sciences Center in Singapore. He joined in ShanghaiTech University in 2014. His research interests include 3D reconstruction, image generation, and video understanding. He has been awarded the Microsoft Research Fellowship, ACM Shanghai Young Research Scientist, Shanghai Excellent Academic Leader, Shanghai Teaching Achievements award and National Young Talents award. He has published over 120 peer-reviewed papers with a total citation count of 14,800+ (source: Google scholar) and an H-index 51. He has served as an area chair for many top-tier AI conferences, including NeurIPS, CVPR, ICCV, and AAAI. He is a publicity chair of CVPR 2024. He is/was an associate editor for IEEE TPAMI(2023-), IEEE TCSVT (2018-2022) and Neurocomputing(2018-).

For information, please contact: