7
$\begingroup$

My background is in electrical engineering. I have a good grasp of CS foundations (e.g. data structures, algorithms, operating systems, discrete math and software engineering).

I have option of enrolling in a MS program in applied math at a good school. However, I would like to study AI.

Which areas of applied math are relevant to AI (not specifically ML)?

$\endgroup$
6
$\begingroup$

EE Math Refresh

AI is an interdisciplinary field. You can begin by ensuring you are fresh in the mathematics you've already taken. You may already have all the books from your BS and MS.

  • Infinite series
  • Logical proofs
  • Linear algebra and matrices
  • Analytic geometry, especially the distinction between local and global extremes (minima and maxima), saddle points, and points of inflection
  • Set theory
  • Probability, especially Bayes' Theorem and the common probability distributions
  • Statistics, especially variance and correlation coefficient
  • Time series
  • Infinite series
  • Convergence — Central to AI batch processing
  • Partial differentials
  • Jacobian and Hessian matrices
  • Multivariate math — Spaces beyond ℝ3 are common in AI.
  • Boundary regions
  • Discrete math
  • Criteria for system stability — Central to real time intelligent control
  • Bernhard Riemann's manifolds is important for more advanced AI theory

An EE would already understand circuit paths (Gustav Kirchhoff's work, signal transmission, feedback theory, and control theory. To see the connection between AI and EE, Norbert Wiener's seminal work, Cybernetics, 1948, MIT Press, will be a helpful read and give you a clearer picture that both fields have common roots.

Finite Math

Since finite math curricula differ between schools of engineering, this list can be used as a checklist for AI.

  • Predicate logic and rules engines
  • Graphs (as in vertices connected by edges) and algorithms for them — pioneered by Leonhard Euler
  • Game theory — John von Neumann and Oskar Morgenstern's Game Theory (published in 1944 but still fresh and pertinent)
  • Decision trees
  • Markov chains and the Markov property — Andrey Markov's work
  • Information theory — Claude Shannon's extensions of the thermodynamic concepts of entropy, especially uniqueness, redundancy, relative entropy, and cross entropy — A necessity to more fully understand feature abstraction and auto-encoding
  • Chaos theory, especially auto-correlation, for analysis of chaotic system behavior in phase space — Chaos Theory Tamed, Garnett P. Williams, 1997, provides an excellent overview
  • Difference between random and pseudo random number generation
  • Curve fitting and gradient descent, especially the Levenberg–Marquardt algorithm
  • Stateless versus stateful algorithms, AI system components, and services
  • Algorithmic theory, especially tail recursion and algorithms for generalization, abstraction, and object recognition
  • Gödel's uncertainty theorems
  • Turing completeness and how it does not overturn yet overcomes Gödel's uncertainty
  • Turing's Imitation Game as a test for conversational intelligence
  • Other test criteria for intelligence outside the conversation context (i.e. piloting, driving, business intelligence, et cetera)
  • Topology — pioneered by Henri Poincaré
  • Computational linguistics — pioneered by Richard Hook Richens — the foundation of much NLP (natural language processing)

Machine Learning

Since machine learning is trending and will probably remain strong even as other AI technologies catch up, there are a few

  • MLPs (multilayer perceptrons)
  • Convolution and the use of convolution layers and pooling layers in deep convolution networks
  • LSTM (long-short term memory) networks, their predecessor RNNs (recurrent neural networks), and some of the newer competitors in the stateful network space
  • Attention based networks and other gating strategies (largely found online using academic searches for papers)

Soft Sciences

Because most of what is occurring, has occurred, and will occur in AI for the foreseeable future is the replication of human brain capabilities, the EE can point her or his analytical skills at biology (especially neurology), genetics as it may apply to the emergence and control of human intelligence (bioinformatics), psychology (especially cognitive science), and sociology (such as.

Because of the emergence of GANs and other devices that work on the basis of balance and equilibrium, it is useful to recall chemical equilibria and biological stasis. These concepts apply directly to AI and are likely to play a more major role as the gap between brain network research and computing narrows further.

Automated Vehicles

The obvious emerging AI technology that is nicely aligned with the EE background is the movement of road and air vehicles from human driving and piloting through the various phases of automation. The goal that most corporations see as the ideal is to progress forward until the driver's seats and cockpits are no longer installed in cars and planes because human interaction other than to provide the transportation destination poses a public safety issue.

Obviously, computer vision, object recognition, sound recognition, and collision avoidance technique are essentials and highly dependent upon rigorous mathematical treatment.

$\endgroup$
  • 1
    $\begingroup$ Did I understand you right, that a degree in mathematics contains of subjects like discrete mathematics, set theory and time series and that this amount of knowledge can be used in artificial Intelligence too? I'm asking because AI is often described as something not invented yet. $\endgroup$ – Manuel Rodriguez Oct 15 '18 at 22:39
0
$\begingroup$

To get a solid foundation of most of the ideas behind machine learning techniques, you would need

  • Linear Algebra
  • Vector Calculus
  • Statistics
  • Basic programming skills (e.g. NumPy)

Of course, the more you know the better.

To study other aspects of AI (e.g. the theory of agents with unbounded computation power), you need to study

  • Decision theory
  • Game theory
  • Set theory
$\endgroup$
  • 1
    $\begingroup$ No I specifically meant AI. ML I know about the methods. $\endgroup$ – doubleE Oct 16 '18 at 4:00
  • 2
    $\begingroup$ you might want to take a look at the research the MIRI intelligence.org to get a flavour of modern AI research $\endgroup$ – k.c. sayz 'k.c sayz' Oct 16 '18 at 4:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.