Hot answers tagged

15

Good Mathematics Foundation Begin by ensuring full competency with intermediate algebra and some other foundations of calculus and discrete math, including the terminology and basic concepts within these topics. Infinite series Logical proofs Linear algebra and matrices Analytic geometry, especially the distinction between local and global extremes (minima ...


9

I work as a professor, and recently designed the mathematics requirements for a new AI major, in consultation with many of my colleagues at other institutions. The other answers, particularly this one do a good job of cataloging all the specific topics that might be useful somewhere in AI, but not all of them are equally useful for understanding core topics. ...


7

Although I have only partially read (or not read at all) some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you. I would also like to note that if you understand the contents of the ...


4

Neural Network Design (2nd edition) by Hagan et al. is one resource you could look at. It's a huge tome, weighing in at over 1000 pages in pdf form, but it is freely available (you can also buy a dead-tree version if you really want one).


4

There is actually a book called Artificial General Intelligence by Ben Goertzel and Cassio Pennachin. It's a bit out of date (from 2008), and published as a Springer-Verlag monograph (which tends to have fairly low editorial standards). This one is also an anthology, with each chapter written by a different author. It's probably not suitable as an ...


4

The following articles Ising models for networks of real neurons (2006) by Gasper Tkacik et al. Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models (2018) by Kyle Mills et al. Inverse Ising inference by combining Ornstein-Zernike theory with deep learning (2017) by Soma Turi, Alpha A. Lee et al. ...


4

Statistical efficiency in this context essentially means that a CNN would require fewer training examples than a fully connected network to learn. Intuitively this seems reasonable: more parameters to learn should mean more samples needed. Of course it is always desirable to minimise the number of training samples needed, so that's a definite advantage of ...


3

In essence, your question is about convergence of infinite series. The mathematical discipline that studies such series is hundreds (if not thousands) years old an has nothing to do with "hardware architecture". A basic example of an infinite series is the geometric series: $$ S = 1 + \gamma + \gamma^2 + \gamma^3 + \dots$$ Note that the series is ...


3

A quick review of resolving expectations: If you know that a discrete random variable $X$, drawn from set $\mathcal{X}$ has probability distribution $p(x) = \mathbf{Pr}\{X=x \}$, then $$\mathbb{E}[X] = \sum_{x \in \mathcal{X}} xp(x)$$ This equation is the core of what is going on when resolving the expectation in your quoted equation. Resolving the ...


3

In addition to the ones you mentioned, I would add Algorithms of Reinforcement Learning by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example this one). It generally follows the same outline as Sutton & Barto's book (except the part on bandits, it is included in the Chapter on Control). ...


3

Some of the books that you mention are often used as reference books in introductory courses to machine learning or artificial intelligence. For example, if I remember correctly, in my introductory course to machine learning, the professor suggested the book Pattern Recognition And Machine Learning (2006) by Bishop, although we never used it during the ...


3

If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone ...


3

The following 2 books helped me understand the basics and guided me through my first AI / CI implementations. Computational Intelligence: An Introduction by Andries P. Engelbrecht It includes the most relevant developments in computational intelligence with good discussions on intelligence and artificial intelligence (AI). Computational Intelligence: ...


3

As far as simple algorithms like Gradient Descent are concerned, you need to have a good grasp of partial derivatives. Especially if you want to implement neural networks. Also most algorithms are vectorised to improve computing speed and so you need to be comfortable with matrix math. This involves being really quick and comfy with dimensions of matrices, ...


3

Your intuition is good. Because "nearby" is only defined with "connected", there could only be 1 station between them. However, it says that the stations are "not_too_far" if at most one station is between them. What about if no stations are between them? If 2 stations are "connected" they should be "not_too_far" as well. So it should be: not_too_far(X,Y) ...


2

Why is AIMA dense? Artificial intelligence is a broad field: that's why Artificial Intelligence: A Modern Approach (AIMA) may look a bit dense to newcomers, given that it covers many different aspects of AI, such as search, machine learning, and natural language processing. AI is not just ML! The first book in this answer is a good book, but it focuses on ...


2

This is a commonly used notation in theoretical computer science. $[m]$ is not the variable $m$, but is instead the set of integers from $1$ to $m$ inclusive. The empirical error equation thus reads in English: The cardinality of a set consisting of the elements $i$ of the set of integers $[m]$ such that the hypothesis given input $x_i$ disagrees with ...


2

The section of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) that shows the limitations of the perceptron should be 11.8 The Nonseparable Case (p. 181), where the authors write There are many reasons for studying the operation of the perceptron learning program when there is no $\mathbf{A}^*$ with ...


2

It's much easier to deal with logarithms, as the relevant numbers are usually very small or very large. If you have a long exponential expression, it's hard to see the difference, but if you're looking at 4.3 vs 5.6, you can immediately see what's happening. And logarithms are a well-known (and well-understood) way of achieving this compression. You can ...


2

It is very confusingly worded, and I would think it's incorrect according to linguistic terminology. A lemma is the canonical form of a word, commonly the infinitive of a verb, the nominative singular of a noun, and the positive of an adjective. The inflected forms belonging to a word would the the forms used for other tenses and persons etc for verbs, case ...


2

I check again with the subchapters of Artificial Intelligence: A Modern Approach, 4th Global ed / US ed from this website the pdf subchapters reference of Global Edition and US Edition. I can confirm you the difference between Global US edition is this subchapter: 20 Knowledge in Learning 739 20.1 A Logical Formulation of Learning 739 20.2 Knowledge ...


1

I would like to add details to Oliver's answer. From the book "Pattern Recognition and Machine Learning" by Bishop (Section 1.2.5): In practice, it is more convenient to maximize the log of the likelihood function. Because the logarithm is monotonically increasing function of its argument, maximization of the log of a function is equivalent to ...


1

In its most raw form, convolution is defined as: $(f*g)(t) = \int_{-\infty}^\infty f(\tau) \cdot g(t-\tau) d\tau$. Here, t doesn't represent the time domain. Infact, it represents the real valued argument the book is talking about. In this notion, at moment t, convolution can be thought of as a weighted average of the function $f(\tau)$ weighted by $g(–\tau)$...


1

A bag-of-words-model (BOW) is usually used to represent a text: you throw all the words together (as if in a bag), without keeping track of their sequence. This is a gross simplification over a text, as word sequencing plays an important role in creating the meaning of a text. But on the positive side it's easier to handle, eg in information retrieval tasks, ...


1

I can't really make much sense of Eisenstein's distinction between distributional and distributed. And I think in your question you actually mix up the two terms as well, as distributed semantics involve symbolic structures, whereas distributional semantics are numerical vectors according to his definition. EDIT: actually, he seems to mix it up himself there?...


1

Having a sound understanding on language processing will help you understand all its concepts. This summarise must reads for NLP.


1

I propose you try this. It's about modern Natural Language Processing, Computational Linguistics and Speech Recognition, including Embeddings methods.


1

The most popular theoretical framework in use currently, in the neuromorphic (brain-inspired) computing community is the Neural Engineering Framework (NEF). Neural Engineering by Chris Eliasmith and Charles Anderson explains the framework comprehensively. As a follow up to that, How to Build a Brain by Chris Eliasmith describes the more recent and more high-...


1

Pattern Recognition And Machine Learning is a great theoretical book. I don't know anything better on standard ML. I read several pages from it myself and all my colleagues researchers suggest to look there if you are not sure about some concepts. The 2 problems with it are that it's huge and it doesn't cover almost all deep learning models known for today. ...


1

Foundations of Deep Reinforcement Learning: Theory and Practice in Python (Addison-Wesley Data & Analytics Series) 1st Edition This book does not give a detailed background information on Markov Decision Processes, different Bellman equations and relationships between the value function and action-value function, etc. It focuses on Deep Reinforcement ...


Only top voted, non community-wiki answers of a minimum length are eligible