How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks

Grasping how generalization behaves under joint scaling of width and data remains a central challenge in machine learning since prior scaling laws analyses are restricted to fixed-feature or infinite-width regimes.

Understanding how performance scales jointly with model size and data is a central problem in modern machine learning. Existing theoretical works on scaling laws typically describe generalization as a function of data or compute, often in fixed-feature or infinite-width regimes…