Which courses in computer science and logic are relevant to Machine Learning?

Question

Although I have a decent background in math, I'm trying to understand which courses from CS and logic to look into. My aim is to get into a Machine Learning PhD program.

Answer 1

I worked as a professor for a time, and often advised students on this. For a PhD in machine learning, I think the ideal background is:

1. Core CS Courses
   • Programming (typically 3-4 courses). Language choice is not highly important, but Python, C++, Java, and perhaps JavaScript, are reasonable picks, if only because of their prevalence.
   • Core topics: data structures, algorithms, operating systems, databases
   • numerical linear algebra or numerical methods
   • advanced algorithm design and analysis

Together these will allow you to read and write the code that even highly optimized versions of ML algorithms are written in, and to understand what might be going wrong within them.

2. AI & ML courses, usually offered through a CS department

• a broad survey course in AI (like AI:AMA by Russel & Norvig), usually offered to senior undergraduates.
• A course applied machine learning or data mining.

You may also take other AI courses, but they are not as common to see offered to undergraduates, so many students wait until graduate school:

• reinforcement learning
• soft computing
• computational learning theory
• Bayesian Methods
• Deep Learning
• Multiagent Systems
• Information Retrieval
• Natural Language Processing
• Computer Vision
• Robotics

Together, these will give you the broadest possible background in AI & ML. These can allow you to find new applications of ML, or to pull AI techniques from one area into another as you need.

3. Statistics courses

• a 1 or 2 term course in probability theory, ideally a version that requires and uses calculus.
• at minimum a course in statistical hypothesis testing.

Much stronger would be to also take courses in:

• regression
• generalized linear models
• experiment design
• causal inference
• Bayesian methods

These courses allow you to reason formally and comfortably about uncertainty. They also give you the correct framework for answering questions about whether your ML algorithm is working, and what patterns an ML algorithm uncovers mean.

4. Mathematics courses

• 3 semesters of calculus, going at least as far as multi-variate/vector calculus.
• optionally, a more advanced course that builds on calculus, like real analysis, but only to reinforce calculus concepts.
• at least 1, and preferably 2, courses in linear algebra
• at least 1, and preferably 2, courses in discrete mathematics.
• ideally, something like Knuth et al.'s Concrete Mathematics
• ideally a course in advanced optimization techniques
• optionally, courses in logic, but be aware that this is almost a fringe area in AI now, and essentially irrelevant to a PhD in machine learning. The parts you need are usually covered in a broad survey AI course.

These courses give you the basic mathematical fluency to understand most machine learning algorithms well.

Answer 2

From the top of my mind roughly in order of priority excl. math:

Practical/applied CS: machine learning, artificial intelligence (incl. symbolic AI), data mining, algorithms, data structures

Theoretical CS: complexity theory

Programming: Python

Logic is highly relevant for symbolic AI but not so much for sub-symbolic approaches like ML.

For all topics mentioned in the first two categories you can find free online lectures from different universities.

Answer 3

In several projects, I found data analysis and data structures to be critical. Machine Learning requires huge amounts of data and, most likely, the data will come from multiple sources. Prior to use, data requires analysis, cleaning, interpretation, feature engineering (subject matter expertise), and structure.