What are some resources on computational learning theory?

Question

Pretty soon I will be finishing up Understanding Machine Learning: From Theory to Algorithms by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in JMLR's COLT.

1. Is there a book similar to "Understanding Machine Learning: From Theory to Algorithms" that would progress my knowledge further and would go well after reading UML?

2. Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?

(Also, taking courses in this is not really an option, so this will be for self-study).

(Note that I have also asked this question here on TCS SE, but it was recommended I also ask here.)

Answer 1

Although I have only partially read or not read at all some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.

I would also like to note that if you understand the contents of the book you are currently reading, you are probably already prepared for reading some (if not most of) the research papers you wish to read. Initially, you may find them a little bit too succinct and sometimes unclear or complex, but you need to get used to this format, so there's nothing stopping you from trying to read them and learn even more by doing this exercise.

Books

• An Introduction to Computational Learning Theory (1994) by Kearns and Vazirani (no free PDF is available, afaik)
• The Nature of Statistical Learning Theory (1995, 2000) by Vapnik
• Machine Learning (1997) by Mitchell
• Statistical Learning Theory (1998) by Vapnik
• Prediction, Learning, and Games (2006) by Cesa-Bianchi and Lugosi
• Foundations of Machine Learning (2012) by Mohri et al.
• Boosting: Foundations and Algorithms (2012) by Schapire and Freund
• Bandit Algorithms (2020) by Lattimore and Szepesvári (with associated solutions to the exercises)

Papers

• Language identification in the limit (1967) by E. Mark Gold (this paper introduces algorithmic learning theory, i.e. the non-statistical approach to CLT)

• A theory of the learnable (1984) by Valiant (this paper introduces PAC learning)

• An overview of statistical learning theory (1999) by Vapnik

• Introduction to Statistical Learning Theory (2014) by Bousquet et al.

Courses (videos)

• Learning from Data (Caltech) (2012) by Yaser Abu-Mostafa
• Machine Learning Theory (2015) by Shai Ben-David

Lecture notes

• Computational Learning Theory - Michaelmas Term 2018 by Varun Kanade

• Introduction to Machine Learning (2009) by Shai Shalev-Shwartz

• Concentration-of-measure inequalities (2009) by Gábor Lugosi

• Stat 928: Statistical Learning Theory (2011) by Sham Kakades

• COMS E6253, Advanced Topics in Computational Learning Theory (2012) Rocco Servedio

• 9.520: Statistical Learning Theory and Applications, Spring 2012 by Tomaso Poggio et al.

• STAT928: Statistical Learning Theory and Sequential Prediction (2014) by Alexander Rakhlin and Karthik Sridharan

• ECE 543, Statistical Learning Theory (2019) by Bruce Hajek and Maxim Raginsky

Other

See also this list of resources https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc compiled by Kiran Vodrahalli.