Seminars

Home Centers & Programs AI and Natural Sciences Seminars

ABSTRACT: The curse of dimensionality (CoD) refers to the exponential growth of computational complexity or data requirements as the dimension of the input or computation space increases. While CoD has been extensively investigated in neural network approximation and generalization theory, its emergence in neural network optimization—particularly from the perspective of training time—has received limited attention. In this talk, we demonstrate the presence of CoD in neural network optimization and explore its dependence on the smoothness of the target function. This stands in contrast to the predominant focus of current research, which seeks positive results in highly overparameterized regimes, often under strong assumptions on the training data or activation function smoothness. To the best of our knowledge, this work is the first to rigorously analyze how the smoothness of the target function influences the curse of dimensionality in the context of neural network optimization theory. This is joint work with Haizhao Yang (University of Maryland, College Park).