Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data

Compression is fundamental to intelligence.

Compression is fundamental to intelligence. A model that can represent its training data as a short code has discovered regularities that enable generalization. Large neural networks may learn functions far simpler than their parameter counts suggest, but it is challenging to…