13
$\begingroup$

What is the mathematical background required to start learning AI? What else should I also learn?

$\endgroup$

13 Answers 13

10
$\begingroup$

To excel in in AI you need a mathematical intuition or point of view. In order to become a full stack AI engineer, it is important that you have a firm understanding of the mathematical foundations of machine learning.

My advice to anyone preparing to jump into the field is that learning mathematics is about doing. Remember the 20/80 rule. You need to study theory 20% of the time and practice what you have learnt 80% of the time in order to be proficient.

The first step before you dive into undergraduate math is for you to refresh on foundation level math. This includes revisiting and mastering high school mathematics especially Algebra. This is necessary for you to understand higher courses.

Linear Algebra

Matrix Operations, Projections, Eigenvalues & Eigenvectors, Principal Component Analysis (PCA), Singular Value Decomposition (SVD), Eigen-decomposition of a matrix, LU Decomposition, QR Decomposition/Factorization, Symmetric Matrices, Orthogonalization & Orthonormalization, Vector Spaces and Norms are needed for understanding the optimization methods used for machine learning.

A nice thing about Linear Algebra is that there are wonderful online resources such as Khan Academy's course on linear algebra https://www.khanacademy.org/math/linear-algebra.

Probability Theory and Statistics

Probability Rules & Axioms, Conditional and Joint Distributions, Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian), Moment Generating Functions, Bayes' Theorem, Random Variables, Variance and Expectation, Maximum Likelihood Estimation (MLE), Prior and Posterior, Maximum a Posteriori Estimation (MAP) and Sampling Methods.

I refer you to this online Statistics and Probability MOOC at Khan Academy https://www.khanacademy.org/math/probability

Algorithms and Optimization

Knowledge of data structures (Binary Trees, Hashing, Heap, Stack etc), Dynamic Programming, Randomized & Sublinear Algorithm, Graphs, Gradient/Stochastic Descents and Primal-Dual methods are needed.

For in depth tuition, I recommend that you sign up for Andrew Ng's Coursera course available here https://www.coursera.org/learn/machine-learning.

Multivariate Calculus

Topics such as Differential and Integral Calculus, Directional Gradient, Hessian, Jacobian, Laplacian and Lagragian Distribution, Partial Derivatives, Vector-Values Functions. Here is a lint to Khan Academy's Calculus course https://www.khanacademy.org/math/calculus-home.

Others

This consists of math topics that are not covered in the above four major areas. They include Information Theory (Entropy, Information Gain), Function Spaces and Manifolds, Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits, Cauchy Kernel, Fourier Transforms).

More resources

Below is an Excellent resource of free Machine Learning Mathematics ebooks http://blog.paralleldots.com/data-science/list-of-free-must-read-books-for-machine-learning/

Finally to keep up with recent development and the latest papers, I recommend that you follow this blog, which aggregates AI and ML papers, http://www.arxiv-sanity.com/.

| improve this answer | |
$\endgroup$
10
$\begingroup$

Start with Andrew Ng's introduction to Machine Learning course on Coursera. There are not many prerequisites for that course, but you will learn how to make some useful things. And, more importantly, it will clearly show you which subjects you need to learn next.

| improve this answer | |
$\endgroup$
5
$\begingroup$

AI is quite large in scope and it sits at the intersection of several areas. However, there are a few essential fields or topics that you need to know

  1. Set theory
  2. Logic
  3. Linear algebra
  4. Calculus
  5. Probability and statistics

I would recommend you to first explore the AI algorithms that you might be interested in. I advise you to start with machine learning and deep learning.

Do not forget one very important prerequisite, passion, without it you are probably wasting your time!

| improve this answer | |
$\endgroup$
4
$\begingroup$

I would suggest you to

  1. start with Andrew Ng's Machine Learning course on Coursera. He provides the brief introduction to mathematics necessary for machine learning. Though not complete, it will be enough to cruise through the course.
  2. Next carefully learn logistic regression in the course. The sigmoid function will be widely used in neural networks.
  3. In the course, he will introduce you to neural networks and error minimization using back propagation. The back propagation will use optimization technique called Gradient Descent. It is a very important topic.
  4. After completing above steps try Geoff Hinton's neural networks course on Coursera.

If you want to go deep in math. Try these:

  • Linear algebra - Gilbert Strang
  • probability - khan academy

I would also like to suggest one of the best books for deep learning: Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. http://www.deeplearningbook.org/

| improve this answer | |
$\endgroup$
4
$\begingroup$

The current, cutting edge AI methods all heavily rely on statistical modeling. You might want to browse the Data Science and Cross Validated stacks to see what people are doing, and the types of maths they are using. (This is not really my field, so I'll leave it to the Neural Network and Deep Learning crowd to provide more detail here.)

I'd also strongly recommend looking at Combinatorics, Combinatorial Game Theory, and Computational Complexity Theory, since one of the the core problems of optimal decision making is intractability.

Game Theory is also important, because the other core issues of optimal decisio making involve imperfect information and incomplete information. (The former is seen more in combinatorial games, but both lead to the need for probability analysis.)

| improve this answer | |
$\endgroup$
4
$\begingroup$

The most important skill you need is self-discipline.

Regarding the mathematical prerequisites, you will need to study statistics, probability theory, calculus, and linear algebra, given that e.g. most machine learning algorithms are highly based on concepts from these areas.

Regarding the programming prerequisites, Python and R are usually a good choice, given the relevant libraries that are available.

You might also need to learn frameworks like Hadoop, in case you want to work with big data.

| improve this answer | |
$\endgroup$
2
$\begingroup$

Artificial Intelligence is a very broad field and therefore things will change accordingly.

Some Prerequisites: (Being a student of CS you should have fulfilled them)

  • Sound knowledge of algorithms and Data Structures. This skill will come in handy while solving problems that require use of alpha-beta pruning, minimax algorithm, etc.
  • Basic knowledge of programming languages like Java, Python. Python will help as it focuses more on the development part. For more info read this. Knowledge of LISP will be very helpful. Go through this answer.

The book, Artificial Intelligence: A Modern Approach (by Stuart J. Russell and Peter Norvig) is considered the Bible of AI. I strongly recommend you to read the complete book and solve the exercises. You can find the pdf of the book here. For solution manual visit this link. It will be better if you can buy a hardcopy of the book.

Knowledge of Computational Theory will greatly help you. Especially when you are working in the field of Natural Language Processing. Other sub-fields of AI that might interest you will be Machine Learning, Evolutionary Computing, Genetic Algorithms, Reinforcement Learning, Deep Learning etc. The list goes on.
Better your knowledge in Statistics, better it will be for Artificial Intelligence. Stay tuned to recent goings in the field via forums, websites, etc. Open AI website is also a very good source.

| improve this answer | |
$\endgroup$
1
$\begingroup$

In addition to Maheshwar's answer, once you feel you want to try more practical Machine Learning, I'd start with Weka. The software is free and effective, they have a good manual and relevant exercises and there are plenty of free videos available on Youtube!

| improve this answer | |
$\endgroup$
1
$\begingroup$

To complement the other answers:

I recommend you to take the Artificial Intelligence course from the AI micromaster given by Columbia on edx.

The course cover a wide range of AI problems and the most important is that give you a general framework to think with a mix of applications on python. Based on the book of Artificial Intelligence: A Modern Approach by Peter Norvig and Stuart Russell

From the perspective of machine learning as well said gokul, the Machine Learning course of Andrew Ng. on coursera is a good introductory course and very oriented to a potential practitioner.

I found useful to combine the study of some machine learning algorithms with the statistical programming language R to experiment with many algorithms to catch the concepts. Useful the following books: Elements of Statistical Learning and Introduction to Statistical Learning, both are available free on the authors websites.

| improve this answer | |
$\endgroup$
1
$\begingroup$

The OP asked about AI, not Machine Learning. Machine Learning is a sub-discipline of AI. Most AI used in games and motion planning does not involve machine learning.

The background you will need to "jump into the field of AI":

  • Math through calculus
  • Algorithms
  • Basic combinatorics and probability theory

If you already have these requirements, I suggest looking at different types of AI to see what sounds most interesting to you. Mostly, getting into AI just means learning how the algorithms work. The algorithms for facial recognition (a machine learning problem) for example are much different than coordinating a team of robots (a combinatorial optimization problem).

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for contributing and welcome to AI! We've definitely had questions asking about terminology, and I strongly agree ML is a subfield of the umbrella of AI. I have noted that a very large proportion of question in the past couple of years have been focused on ML sub-fields, and it seems to be the area with the most "heat" at present. In regards to your answer, would you consider pathfinding important enough to warrant a specific mention? $\endgroup$ – DukeZhou Jan 22 '18 at 22:28
  • 1
    $\begingroup$ I would say combinatoric search deserves mention -- pathfinding is a type of combinatoric search and falls under that umbrella. $\endgroup$ – thayne Jan 23 '18 at 15:52
1
$\begingroup$

"artificial intelligence" is now a mature field and there are many subfields in it. there are entire theories erected around each subfield of artificial intelligence. a simple analogy would like "what are the skills needed for succeeding in mathematics/physics?". the answer really depends on what branch you want to delve into.

if you are planning to go into more application specific side of artificial intelligence (machine learning/ deep learning) then you must be thorough with linear algebra . since most of machine learning algorithms can be boiled down to simple tricks in linear algebra. learning about optimization will help a lot because you will encounter many algorithms involving convex/nonconvex optimization methods.

Also statistics and probability are must for any field in artificial algorithms.

That said, at core artificial intelligence deals with machines that think. So it would be better if you are through with formal logic , automata and complexity theory .

| improve this answer | |
$\endgroup$
1
$\begingroup$

You can start with they videos on Youtube. It is simple, with a basic knowledge in Python, you can do much things with ML.

Machine Learning - Recipes #1

I found too, a new video of Cloud AI Adventures explain "The 7 Steps of Machine Learning", you can watch HERE.

About math, I asked the same question when I started learning ML / AI. I'm not good with math, just the basics (also developer). But today, with TensorFlow and other libraries, you can focus on the theory and definition. So, do not worry, over time, AI models will be construct with a few buttons and labels ... maybe it's happening.

| improve this answer | |
$\endgroup$
0
$\begingroup$

As they suggested good resources and there are many resources but I will recommend you to start with What Is The Best Book On Artificial Intelligence (AI)? , Introduction to Machine Learning and Artificial Intelligence - Machine Learning

And this video summary which knowledge of math that you need Mathematics of Machine Learning And this link explain Mathematics for AI: All the essential math topics you need

After that, you can see these links, these my favorite

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.