HKU IDS Guest Seminar Series:
Pointwise Generalization in Deep Neural Networks
Abstract
We address the fundamental question of why deep neural networks generalize by establishing a pointwise generalization theory for fully connected networks. This framework resolves long-standing barriers to characterizing the rich nonlinear feature-learning regime and builds a new statistical foundation for representation learning. For each trained model, we characterize the hypothesis via a pointwise Riemannian Dimension, derived from the eigenvalues of the learned feature representations across layers. This establishes a principled framework for deriving hypothesis-dependent, spectrum-aware generalization bounds. These bounds offer a systematic upgrade over approaches based on model size, products of norms, and infinite-width linearizations, yielding guarantees that are orders of magnitude tighter in both theory and experiment. Analytically, we identify the structural properties and mathematical principles that explain the tractability of deep networks. Empirically, the pointwise Riemannian Dimension exhibits substantial feature compression, decreases with increased over-parameterization, and captures the implicit bias of optimizers. Taken together, our results indicate that deep networks are mathematically tractable in practical regimes and that their generalization is sharply explained by pointwise, spectrum-aware complexity.
Speaker
Prof Yunbei XU
Assistant Professor, National University of Singapore Professor Yunbei XU is a Presidential Young Assistant Professor at the National University of Singapore. He received a B.S. in Pure Mathematics from Peking University, a Ph.D. in Decision Sciences from Columbia Business School, and completed postdoctoral training in computing at MIT. He is a recipient of the ICML Outstanding Paper Award and First Place in the INFORMS Student Paper Competition. His research develops mathematical foundations for AI and real‑world systems, focusing on structural principles of neural networks and dynamical systems in language, decision-making, and the physical world. For full biography of Prof. Xu, please refer to: https://cde.nus.edu.sg/isem/staff/xuyunbei/
Moderator
Prof Qingpeng ZHANG
Associate Professor @ HKU IDS & HKUMed Prof Qingpeng ZHANG is an Associate Professor at the Musketeers Foundation Institute of Data Science and the Department of Pharmacology and Pharmacy, LKS Faculty of Medicine, University of Hong Kong. His research lies at the intersection of data science and medicine, with a focus on developing interpretable machine learning and evolutionary modeling to understand complex biological systems. His work has appeared in journals such as Nature Human Behaviour, Nature Communications, and PNAS, and has been highlighted in media outlets such as The Washington Post, The New York Times, The Times, CNN, and BBC. For full biography of Prof. Zhang, please refer to: https://datascience.hku.hk/people/qingpeng-zhang/
