Machine Learning

The Stochastic Parrot in the Coal Mine: Model Collapse is a Threat to Low-Resource Communities
Model collapse, the degradation in performance that arises when generative models are trained on the outputs of prior models, has …
Devon Jarvis, Richard Klein, Benjamin Rosman, Steven James, Stefano Sarao Mannelli
PDF Cite
If Dropout Limits Trainable Depth, Does Critical Initialisation Still Matter? A Large-scale Statistical Analysis on ReLU Networks
Recent work in signal propagation theory has shown that dropout limits the depth to which information can propagate through a neural …
Arnu Pretorius, Elan Van Biljon, Benjamin Van Niekerk, Ryan Eloff, Matthew Reynard, Steven James, Benjamin Rosman, Herman Kamper, Steve Kroon
PDF Cite