Date Lecture Readings Logistics
Fri 01/08 Lecture #1 :
Overview of Statistical Machine Learning
[ slides notes ]
  • Vapnik, V. (1991). Principles of risk minimization for learning theory. Advances in neural information processing systems, 4. PDF
  • Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science, 16(3):199-231. PDF
  • Section 12.2: The Support Vector Classifier (The Elements of Statistical Learning)
  • On a class of of Perceptions (translated version) PDF

Fri 01/16 Lecture #2 :
Approximation error and estimation error
[ slides notes ]
  • chapter 4 of Cucker, F., & Zhou, D. X. (2007). Learning theory: an approximation theory viewpoint (Vol. 24). Cambridge University Press.

Fri 01/23 Lecture #3 :
Uniform concentration inequality
[ slides notes ]
  • Lugosi, G. Concentration-of-measure inequalities. PDF

Fri 01/30 Lecture #4 :
Rademacher complexity I
[ slides notes ]
  • Bartlett, P. L., & Mendelson, S. (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):331-372. PDF
  • Chapter 3 of Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018). Foundations of Machine Learning. MIT press.

HW1 out
(due Feb 13)

Fri 02/06 Lecture #5 :
Rademacher complexity II
[ slides notes ]
  • Chapter 27: Covering Numbers (Understanding Machine Learning: From Theory to Algorithms by Shalev-Shwartz and Ben-David)
  • Wainwright, M. J. (2019). High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge University Press. (Chapter 5: Metric Entropy and Its Uses)
  • Section 4.2: Covering and Packing (Probability in High Dimensions by Roman Vershynin)

Fri 02/13 Lecture #6 :
Rademacher complexity III
[ slides notes ]
  • Pollard, D. Chaining (course handout, Yale STAT 607, Spring 2005). PDF
  • Talagrand, M. (2005). The Generic Chaining. Springer. Link