Currently, Dr. Huang’s project focuses on high-quality 3D asset synthesis with generative artificial intelligence learned from large amounts of 2D image and video data.

Humans are situated in a 3D world. To simulate this experience for production or entertainment, we require a significant number of 3D assets to populate virtual environments like robotics simulations and games. Generating such 3D assets is both expensive and time-consuming, necessitating skilled artists with extensive aesthetic and 3D modeling knowledge. Recent advances in generative artificial intelligence show promise for automatic 3D content creation, but still suffer from low visual quality, high computational cost, and poor controllability. In general, existing native 3D generative methods are limited by the scarcity of 3D data, while popular 2D lifting methods struggle with the ill-posed problem of inferring 3D entities from inconsistent image observations. This project aims to explore effective generative models and algorithms for 3D asset synthesis, particularly the use of physical and semantic priors.