Course Information
Instructors
Arpit Agrawal
Sujoy Bhore
Teaching Assistants
To be announced
Lectures
Venue: To be announced
Timings: To be announced
Office Hours
To be announced
Course Description
This course provides a rigorous introduction to the fundamental concepts and theoretical underpinnings of machine learning. Key topics include the PAC (Probably Approximately Correct) learning model, VC dimension, Rademacher complexity, generalization bounds, online learning, and other advanced theoretical topics. The goal is to provide students with a deep understanding of the principles that govern machine learning.
Prerequisites
A strong foundation in probability and linear algebra is expected. Prior exposure to machine learning is also expected.
Reference Book
The primary reference for this course will be:
Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David.
Lecture Schedule
| Week | Dates | Topics to be Covered | Lecturer | Lecture Notes | |
|---|---|---|---|---|---|
| 1 | July 28 - August 1 | Introduction and Logistics; Overview of Machine Learning Algorithms; Consistency Model and Intro to PAC Learning | Arpit | ||
| 2 | August 4 - August 8 | PAC Continued; Uniform Convergence; Bias-Variance Tradeoff | Arpit | ||
| 3 | August 11 - August 15 | VC Dimension; ε-net; Radamacher Complexity | Sujoy | ||
| 4 | August 18 - August 22 | Gradient Descent Based Generalization Proofs, Sample complexity; Stochastic Gradient Descent | Arpit | ||
| 5 | August 25 - August 29 | Modern results on Neural Network Generalization; Double Descent | Arpit | ||
| 6 | September 1 - September 5 | Online Learning; Online Perceptron; Experts and Multiplicative Weights | Arpit | ||
| 7 | September 8 - September 12 | Online Convex Optimization; Follow The (Regularized) Leader (FTL and FTRL) algorithms | Arpit | ||
| Mid-Semester Break (September 15 - September 19) | |||||
| 8 | September 22 - September 26 | Bandit algorithms and internal regret; UCB regret bound, Thompson sampling; EXP3 Algorithm | Arpit | ||
| 9 | September 29 - October 3 | Ranking and Complex Prediction | Arpit | ||
| 10 | October 6 - October 10 | Dimensionality Reduction | Sujoy | ||
| 11 | October 13 - October 17 | Clustering | Sujoy | ||
| Diwali Week (October 20 - October 24) | |||||
| 12 | October 27 - October 31 | Invited Talks on Advanced Topics | Guest | ||
| End Semester Exams (November 3 - November 7) | |||||
Grading Policy
The grading policy will be based on a combination of assignments, scribing notes, a mid-term exam, and a final exam. The exact weights will be announced in the first lecture.