This course will explore the mathematical foundations of a rapidly evolving new field: large-scale machine learning. We will focus on recent texts in machine learning, statistics, and optimization, with the goal to understand the tradeoffs that are driving algorithmic design in this new discipline. These tradeoffs will revolve around statistical accuracy, scalability, algorithmic complexity, and implementation.
Sample topics include:
- Optimization and Learning
- Memorization, Generalization, and Algorithmic Stability
- Stochastic Methods for Convex and Nonconvex Settings
- Expressive Power of Neural Nets, Hardness, and Recent Results
- From Supervised Learning to Language Modeling
- Large Scale Learning and Systems
- Modern Architectures: Transformers and State Space Models
- System Tradeoffs, Platforms, and Modern Architectures
- Centralized and Decentralized Distributed Optimization
- Delays, Communication Bottlenecks, and Adversarial Attacks
- Memorization, Generalization, and Algorithmic Stability
- Stochastic Methods for Convex and Nonconvex Settings
- Expressive Power of Neural Nets, Hardness, and Recent Results
- From Supervised Learning to Language Modeling
- Modern Architectures: Transformers and State Space Models
- System Tradeoffs, Platforms, and Modern Architectures
- Centralized and Decentralized Distributed Optimization
- Delays, Communication Bottlenecks, and Adversarial Attacks