5
$\begingroup$

Although I have a decent background in math, I'm trying to understand which courses from CS and logic to look into. My aim is to get into a Machine Learning PhD program.

$\endgroup$
  • 1
    $\begingroup$ Think about probability, statistics, and causal inference as well. $\endgroup$ – rigo Oct 18 '19 at 6:53
4
$\begingroup$

I worked as a professor for a time, and often advised students on this. For a PhD in machine learning, I think the ideal background is:

  1. Core CS Courses
    • Programming (typically 3-4 courses). Language choice is not highly important, but Python, C++, Java, and perhaps JavaScript, are reasonable picks, if only because of their prevalence.
    • Core topics: data structures, algorithms, operating systems, databases
    • numerical linear algebra or numerical methods
    • advanced algorithm design and analysis

Together these will allow you to read and write the code that even highly optimized versions of ML algorithms are written in, and to understand what might be going wrong within them.

2. AI & ML courses, usually offered through a CS department

  • a broad survey course in AI (like AI:AMA by Russel & Norvig), usually offered to senior undergraduates.
  • A course applied machine learning or data mining.

You may also take other AI courses, but they are not as common to see offered to undergraduates, so many students wait until graduate school:

  • reinforcement learning
  • soft computing
  • computational learning theory
  • Bayesian Methods
  • Deep Learning
  • Multiagent Systems
  • Information Retrieval
  • Natural Language Processing
  • Computer Vision
  • Robotics

Together, these will give you the broadest possible background in AI & ML. These can allow you to find new applications of ML, or to pull AI techniques from one area into another as you need.

3. Statistics courses

  • a 1 or 2 term course in probability theory, ideally a version that requires and uses calculus.
  • at minimum a course in statistical hypothesis testing.

Much stronger would be to also take courses in:

  • regression
  • generalized linear models
  • experiment design
  • causal inference
  • Bayesian methods

These courses allow you to reason formally and comfortably about uncertainty. They also give you the correct framework for answering questions about whether your ML algorithm is working, and what patterns an ML algorithm uncovers mean.

4. Mathematics courses

  • 3 semesters of calculus, going at least as far as multi-variate/vector calculus.
  • optionally, a more advanced course that builds on calculus, like real analysis, but only to reinforce calculus concepts.
  • at least 1, and preferably 2, courses in linear algebra
  • at least 1, and preferably 2, courses in discrete mathematics.
  • ideally, something like Knuth et al.'s Concrete Mathematics
  • ideally a course in advanced optimization techniques
  • optionally, courses in logic, but be aware that this is almost a fringe area in AI now, and essentially irrelevant to a PhD in machine learning. The parts you need are usually covered in a broad survey AI course.

These courses give you the basic mathematical fluency to understand most machine learning algorithms well.

$\endgroup$
  • $\begingroup$ I have found studying optimization techniques to be invaluable when trying to decide what technique to use in solving ml/ai problems $\endgroup$ – nickw Oct 25 '19 at 19:38
  • 1
    $\begingroup$ @nickw I agree. A survey course in AI, as well as a typical course in advance algorithm design will cover these topics pretty well, and probably well enough for someone to start on an ML PhD. That said, if they can do a mathematics course that is specifically about optimization, they'll be way ahead of the pack. $\endgroup$ – John Doucette Oct 25 '19 at 19:46
  • $\begingroup$ It's interesting to see, that machine learning is located within the computer science and the core courses are programming and algebra. Because in the corpus of published literature such a bias is not there. Or let me explain it differently. A machine learning student who has attended a university will learn something different from an autodidact who has read academic papers by himself. $\endgroup$ – Manuel Rodriguez Oct 26 '19 at 8:40
2
$\begingroup$

From the top of my mind roughly in order of priority excl. math:

Practical/applied CS: machine learning, artificial intelligence (incl. symbolic AI), data mining, algorithms, data structures

Theoretical CS: complexity theory

Programming: Python

Logic is highly relevant for symbolic AI but not so much for sub-symbolic approaches like ML.

For all topics mentioned in the first two categories you can find free online lectures from different universities.

$\endgroup$
1
$\begingroup$

Because it's a generalized overview question the best format to answer the request is a mindmap. This allows to explore the basic keywords from a domain, and it's possible to extend the picture with additional subjects.

enter image description here

$\endgroup$
  • $\begingroup$ Thanks. I upvoted but it doesn't show because of low rep (below 15). $\endgroup$ – stranger Oct 17 '19 at 12:50
  • 2
    $\begingroup$ I think this diagram might benefit if you perhaps highlighted the bubbles that are relevant to machine learning $\endgroup$ – Recessive Oct 18 '19 at 6:53
1
$\begingroup$

In several projects, I found data analysis and data structures to be critical. Machine Learning requires huge amounts of data and, most likely, the data will come from multiple sources. Prior to use, data requires analysis, cleaning, interpretation, feature engineering (subject matter expertise), and structure.

$\endgroup$
  • $\begingroup$ Data analysis is not necessarily part of the core of computer science. It is more of a statistical sub-area. $\endgroup$ – nbro Oct 17 '19 at 21:55
  • $\begingroup$ Thanks. I upvoted but it doesn't show because of low rep (below 15). $\endgroup$ – stranger Oct 18 '19 at 1:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.