I am looking for a book about machine learning that would suit my physics background. I am more or less familiar with classical and complex analysis, theory of probability, сcalculus of variations, matrix algebra, etc. However I have not studied topology, measure theory, group theory and other more advanced topics. I try to find a book that is written neither for beginners, nor for mathematicians.

Recently I have read great book "Statistical inference" written by Casella and Berger. They write in Introduction that "The purpose of this book is to build theoretical statistics (as different from mathematical statistics) from the first principles of probability theory". So I am looking for some "theoretical book" about machine learning.

There are many online courses and brilliant books out there that focus on the practical side of applying machine learning models and using the appropriate libraries. It seems to me that there are no problems with them, but I would like to find a book on theory.

By now I have skimmed through following books:

Pattern Recognition And Machine Learning

It looks very nice the only point of concern is that the book was published in 2006. So I am not sure about the relevance of the chapters considering Neural nets since this field is developing rather fast.

The elements of statistical learning

This book also seems very good. It covers most of topics as well as the first book. However I am feeling that its style is different and I do not which book will suit me better.

Artificial Intelligence. A Modern Approach

This one covers more recent topics such as natural language processing. As far as I understand it represents the view of computer scientist on the machine learning.

Machine Learning A Probabilistic Perspective

Maybe it has a slight bias towards probability theory which is stated in the title. However book looks fascinating as well.

I think that the first or the second book should suit me but I do not know what decision to make.

I am sure that I have overlooked some books. Are there some other ML books that focus on theory?

  • $\begingroup$ Hi and welcome to our community! This is not a bad post, but it's not fully clear what you mean by "Machine learning book for physicists" and what your question really is. Moreover, note that asking for recommendations or opinions are off-topic. Please, explicitly ask a specific question that can be answered objectively. I think it would be nice to have a post that lists all the ML books that focus on theory, but you need to ask that explicitly. Is the "physicist" part really important? I don't think so. $\endgroup$ – nbro Sep 10 at 21:20
  • $\begingroup$ If you want a theory book, foundations of machine learning is it. But you're going to be spending time on something which most people don't care about, even within the ML field. Things like VC dimension aren't really applicable to real life like images. $\endgroup$ – FourierFlux Sep 11 at 7:29
  • $\begingroup$ @nbro Thank you for your suggestion! I have tried to make question more focused. $\endgroup$ – Ilya Sep 11 at 8:54
  • $\begingroup$ @FourierFlux Thank you for your answer! It is a valuable suggestion! $\endgroup$ – Ilya Sep 11 at 9:00

Some of the books that you mention are often used as reference books in introductory courses to machine learning or artificial intelligence.

For example, if I remember correctly, in my introductory course to machine learning, the professor suggested the book Pattern Recognition And Machine Learning (2006) by Bishop, although we never used it during the lessons. This is a good book, but, in my opinion, it covers many topics, such as variational inference or sampling methods, that are not suited for an introductory course.

The book Artificial Intelligence. A Modern Approach, by Norvig and Russell, definitely does not focus on machine learning, but it covers many other aspects of artificial intelligence, such as search, planning, knowledge representation, machine learning, robotics, natural language processing or computer vision. This is probably the book that you should read and use if you want to have an extensive overview of the AI field. Although I never fully read it, I often used it as a reference, as I use the other mentioned book. For instance, during my bachelor's and, more specifically, an introductory course to artificial intelligence, we had used this book as the reference book, but note that there are other books that do not just focus on machine learning.

The other two books are not as famous as these two, but they are probably also good books, although their focus may be different.

There are at least three other books that I think you should also be aware of, given that they also cover the actual theory of learning, aka (computational) learning theory, before diving into more specific topics, such as kernel methods.

| improve this answer | |

Pattern Recognition And Machine Learning is a great theoretical book. I don't know anything better on standard ML. I read several pages from it myself and all my colleagues researchers suggest to look there if you are not sure about some concepts. The 2 problems of it is that it's huge and it doesn't cover almost all deep learning models known for today. So in addition I'd suggest you to look the "Deep Learning" by Ian Goodfellow, Yoshua Bengio, Aaron Courville (https://blog.floydhub.com/best-deep-learning-books-updated-for-2019/). Your concerns about not studying topology, measure theory and group theory are groundless. These sections of math aren't prerequisites in any way, they aren't even discussed anywhere I know. Actually ML theory is more like probability theory and statistics. Especially statistical learning theory (which is nothing more than probability theory and statistics). I haven't read any books on SLT so have a look at this thread, maybe it will help you https://www.quora.com/Which-is-a-more-useful-read-for-someone-interested-in-ML-research-Statistical-Learning-Theory-by-Vapnik-or-Elements-of-Statistical-Learning-by-Friedman-Tibshirani-Hastie

| improve this answer | |
  • $\begingroup$ Thank you very much for your help! $\endgroup$ – Ilya Sep 11 at 8:55
  • $\begingroup$ You are welcome $\endgroup$ – Michael Solotky Sep 28 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.