Singular perturbations and hierarchical learning in two-layer neural networks

We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension.

We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension. The two layers are optimized jointly, with a perturbative parameter tuning the relative training speed between the first and second…