Workshop II — Theory and Practice of Deep Learning

Neural networks have confounded traditional ML beliefs about the dangers of overfitting and the need for regularization. They have also given rise to many new empirical findings around transfer learning, adversarial examples, compressibility, scaling laws (relating the size of datasets, models, and compute), grokking, and so on. What is needed to explain and predict all this is a rich new theory of learning capable of addressing the delicate interplay between model, data, and optimizers at large scale. This workshop will bring together top researchers driving the frontiers of this work with experts in both theory and experiment for natural intelligence. The result will be a scholarly discussion on how to frame questions about learning and how to distill the similarities and differences between learning with biological and artificial systems.

Topics: Part of the Long Program Mathematics of Intelligences