8

To complete the first answer that is rather graph oriented, I will write a little about deep learning on manifolds, which is quite general in terms of GDL thanks to the nature of manifolds. Note that the description of GDL through the explanation of what are DL on graphs and manifolds, in opposition to DL on euclidean domains, comes from the 2017 paper ...


8

In reinforcement learning (RL), an agent interacts with an environment in time steps. On each time step, the agent takes an action in a certain state and the environment emits a percept or perception, which is composed of a reward and an observation, which, in the case of fully-observable MDPs, is the next state (of the environment and the agent). The goal ...


6

The dynamic programming algorithms (like policy iteration and value iteration) are often presented in the context of reinforcement learning (in particular, in the book Reinforcement Learning: An Introduction by Barto and Sutton) because they are very related to reinforcement learning algorithms, like $Q$-learning. They are all based on the assumption that ...


6

A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the following equality holds. $$ p(s_{t+1} \mid s_{t}, s_{t-1:1}) = p(s_{t+1} \mid s_{t}), \forall t $$ The hidden Markov model (HMM) is an example of a model ...


5

In general the different reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ are not equivalent mathematically, so you will not find any formal proof. It is possible for the functions to resolve to the same value in a specific MDP, if for instance you use $R(s, a, s')$ and the value returned only depends on $s$, then $R(s, a, s') = R(s)$. This is not true ...


4

No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: github.com/keon/deep-q-learning/blob/master/dqn.py DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is ...


4

Let $R(s)$ denote a probability distribution over rewards that our agent may get in some MDP as a reward for entering a state $s$. The easiest case is to demonstrate that we can also choose to write this as $R(s, a)$ or $R(s, a, s')$: simply take $\forall a: R(s, a) = R(s)$, or $\forall a \forall s': R(s, a, s') = R(s)$, as also described in Neil's answer. ...


4

A non-starving policy is a (behavior) policy that is theoretically guaranteed to visit each state and take all possible actions from each state an infinite number of times, so that to always update $Q(s, a)$, $\forall s, \forall a$, an infinite number of times. In the context of off-policy prediction, this criterion implies that any trajectory will have no ...


4

Here is a paper with the mathematical definition of each term: Let Nt,n,σ,L be all target functions that can be implemented using a neural network of depth t, size n, activation function σ, and when we restrict the input weights of each neuron to be |w|1 + |b| ≤ L.


4

A surrogate model is a simplified model. It is a mapping $y_S=f_S(x)$ that approximates the original model $y=f(x)$, in a given domain, reasonably well. Source: Engineering Design via Surrogate Modelling: A Practical Guide In the context of Bayesian optimization, one wants to optimize a function $y=f(x)$ which is expensive (very time consuming) to evaluate, ...


4

As you can see from the picture above (In courtesy to this guy post). Deep learning is a field inside machine learning which involves using deep neural networks to solve machine learning problems. Tensorflow.js and Brain.js are software architectures built in order to assemble and develop ML models inside the browser. If you would like to delve into this ...


4

There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may perform badly in general (but those functions may still work in specific cases) If you are using gradient descent as a training method, then differentiability ...


3

In his book Machine Learning: A Probabilistic Perspective (2012), Kevin P. Murphy defines machine learning as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!) He divides ...


3

That is a wonderfully fundamental question. Learning is the use of a system to change another system so that, instead of doing what it did before, which may have been nothing, it does something else. In the human brain, the system is the way that genetic expression caused the directed mutability of that brain so that human intentions and responses to ...


3

The acclaimed book Artificial Intelligence: A Modern Approach (by Stuart Russell and Peter Norvig) gives a definition of an agent An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. This definition is illustrated by the following figure This definition (and ...


3

tl;dr It helps to think that the channels dimension of a convolutional layer works like a fully connected layer (i.e. the layer computes the weighted sum over all channels). For a single pixel... Let's consider a single pixel (e.g. the top left pixel). This pixel has $C$ different values, where $C$ are the number of channels. In order to produce the ...


3

There is a problem with confining Artificial Intelligence to a single definition, because it has become an umbrella term encompassing many fields of science. It has come a long way from the "thinking machines" of the 50s. Actually, the term was coined in a summer workshop in 1956, whose proposal was: The study is to proceed on the basis of the conjecture ...


3

What is the difference between the definition of a stationary policy in reinforcement learning and contextual bandit? There is no difference. A policy decides which action to take in each state. This is usually split into deterministic policies of the form $\pi(s) : \mathcal{S} \rightarrow \mathcal{A}$ and stochastic policies of the form $\pi(s|a) : \...


3

As far as I know, no true artificial general intelligent system (AGI) has been implemented or is practically useful. Yes, there is Sophia and similar robots that may look like an AGI, but they aren't really AGI systems, as they lack several capabilities that we humans have and they can't really adapt to new circumstances. AlphaGo and AlphaStar are narrow AI ...


2

In the paper Universal Intelligence: A Definition of Machine Intelligence (2007), Legg and Hutter provide a definition of intelligence, which should capture the intuitive notion of intelligence (that people often refer to). Intelligence measures an agent's ability to achieve goals in a wide range of environments. This definition "favors" general ...


2

The article Geometric deep learning: going beyond Euclidean data (by Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, Pierre Vandergheynst) provides an overview of this relatively new sub-field of deep learning. It answers all the questions asked above (and more). If you are familiar with deep learning, graphs, linear algebra and calculus, you ...


2

Think of a computer as a Turing Machine--this idea is a model of computation, and all of modern computing is based on the Turing-Church thesis. Machine and program can be interchangeable—at the end of the day, it's all algorithms, whether hard coded in the form of microchip, or in the form of software. (Any microchip can be emulated as software.) Pre-...


2

for example, the "greedy policy" always chooses the action with the highest expected return, no matter which state we are in The "no matter which state we are in" there is generally not true; in general, the expected return depends on the state we are in and the action we choose, not just the action. In general, I wouldn't say that a policy is a mapping ...


2

There is also the AI effect, that is, the tendency to not consider something an AI once it is well understood. For example, neural networks are not yet fully understood, so people still tend to call them AI. Once we know exactly all the details about neural networks and their inner workings, we might start to consider them just computation. This is an old ...


2

You can find a lot of content regarding the mathematical definitions. Let's take it simpler: It is a universal function approximator. Think about this: You can approximate this locus with a line, two parameters: slope and offset. That's the simplest you can do, with a straight line. Now, Think about this: You approximated it with planes and curvatures, ...


2

In general, the expression "temporal feature" might refer to any feature that is associated with or changes over time. However, in the context of signal processing, a temporal feature might refer to any feature of the data before being transformed to the Fourier, frequency or spectral domain, using the Fourier transform. In this context, the domain of the ...


2

In Reinforcement Learning: An Introduction the authors suggest that the topic of reinforcement learning covers analysis and solutions to problems that can be framed in this way: Reinforcement learning, like many topics whose names end with “ing,” such as machine learning and mountaineering, is simultaneously a problem, a class of solution methods that ...


2

You're probably looking for regression, either linear or non-linear, which usually refers to a set of methods that can be used to predict a continuous (or numerical) value (the value of the so-called dependent variable), given one or more possibly numerical values (the values of the independent variables). (The other common task is called classification, ...


2

Answers: Generally its the former. The next layer would learn at each filter how to merge the channels of the previous layer, that is why in a 2D convolution the kernel is a 3-dimensional tensor. But the number of parameters is $nmc_ic_{i+1}$ at the $i^{th}$ layer (this is ignoring bias). lets assume all channels are $O(c)$ then the spatial complexity ...


2

Both are incorrect. using your notation You do not take a sliding frobenius inner product of a singular channel of $I$ with $F$, but with all the channels at once. This may be easier to understand if you do not assume the number of channels of the input and output are the same (ie different number of input channels then filters). So lets say your input ...


Only top voted, non community-wiki answers of a minimum length are eligible