Neil Slater
  • Member for 5 years, 5 months
  • Last seen this week
  • Durham, United Kingdom
What is the relation between Q-learning and policy gradients methods?
Accepted answer
39 votes

However, both approaches appear identical to me i.e. predicting the maximum reward for an action (Q-learning) is equivalent to predicting the probability of taking the action directly (PG). Both ...

View answer
What's the difference between model-free and model-based reinforcement learning?
37 votes

What's the difference between model-free and model-based reinforcement learning? In Reinforcement Learning, the terms "model-based" and "model-free" do not refer to the use of a neural network or ...

View answer
What are "bottlenecks" in neural networks?
Accepted answer
27 votes

The bottleneck in a neural network is just a layer with fewer neurons than the layer below or above it. Having such a layer encourages the network to compress feature representations (of salient ...

View answer
How to define states in reinforcement learning?
Accepted answer
22 votes

The problem of state representation in Reinforcement Learning (RL) is similar to problems of feature representation, feature selection and feature engineering in supervised or unsupervised learning. ...

View answer
Why does the policy network in AlphaZero work?
Accepted answer
18 votes

The output of the policy network is as described in the original paper: A move in chess may be described in two parts: selecting the piece to move, and then selecting among the legal moves for ...

View answer
How does LSTM in deep reinforcement learning differ from experience replay?
Accepted answer
17 votes

How does this method differ from the experience replay, as they both use past information in the training? What's the typical application of both techniques? Using a recurrent neural network is one ...

View answer
What is the difference between an observation and a state in reinforcement learning?
15 votes

Sometimes observation and state overlap completely, which is convenient. However, there is no reason to expect it in all cases, and that's where interesting problems occur. Reinforcement learning ...

View answer
In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?
15 votes

In a convolutional neural network, is there a unique filter for each input channel or are the same new filters used across all input channels? The former. In fact there is a separate kernel defined ...

View answer
Why should the number of neurons in a hidden layer be a power of 2?
Accepted answer
14 votes

I have read somewhere on the web (I lost the reference) that the number of units (or neurons) in a hidden layer should be a power of 2 because it helps the learning algorithm to converge faster. I ...

View answer
What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?
14 votes

The concepts of on-policy vs off-policy and online vs offline are separate, but do interact to make certain combinations more feasible. When looking at this, it is worth also considering the ...

View answer
Why does DQN require two different networks?
Accepted answer
12 votes

My best guess that it's been done to reduce the computation time, otherwise we would have to find out the q value for each action and then select the best one. It has no real impact on computation ...

View answer
How can an AI freely make decisions?
Accepted answer
12 votes

Neural networks, deep learning and other supervised learning algorithms do not "take actions" by themselves, they lack agency. However, it is relatively easy to give a machine agency, as far as ...

View answer
How is it that AI can become biased, and what are the proposals to mitigate this?
Accepted answer
12 votes

Lately with my Google searches, the AI model keeps auto filling the ending of my searches with: “...in Vietnamese” I can see how this would be annoying. I don't think Google's auto-complete algorithm ...

View answer
Is neural networks training done one-by-one?
Accepted answer
11 votes

Should I be changing the weights/biases on every single sample before moving on to the next sample, You can do this, it is called stochastic gradient descent (SGD) and typically you will shuffle the ...

View answer
Is LSTM a subcategory of RNN?
Accepted answer
10 votes

The Wikipedia article is more technically correct, in that the term RNN is formally taken to mean "a neural network with recurrent connections", and that includes many architectures that ...

View answer
Why exactly do neural networks require i.i.d. data?
10 votes

There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - ...

View answer
What is a "trajectory" in reinforcement learning?
Accepted answer
10 votes

In answer that you linked, I may have used an informal definition of "trajectory", but essentially the same thing as the quote. A "trajectory" is the sequence of what has happened (in terms of state, ...

View answer
Why does the discount rate in the REINFORCE algorithm appear twice?
Accepted answer
9 votes

The discount factor does appear twice, and this is correct. This is because the function you are trying to maximise in REINFORCE for an episodic problem (by taking the gradient) is the expected return ...

View answer
Why are neural networks considered to be artificial intelligence?
Accepted answer
8 votes

Why are we now considering neural networks to be artificial intelligence? "We" aren't. It is generally due to reporting by media sources that simplify science and technology news. The ...

View answer
What is the formula for the momentum and Adam optimisers?
Accepted answer
8 votes

I'm going to use slightly different notation, $\leftarrow$ for an assignment, $\alpha$ for learning rate, $\nabla_w J$ in place of $g$* and implied multiplication as these are slightly more common. ...

View answer
Is the optimal policy always stochastic if the environment is also stochastic?
8 votes

Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic? No. An optimal policy is generally ...

View answer
What does "stationary" mean in the context of reinforcement learning?
Accepted answer
8 votes

A stationary policy is a policy that does not change. Although strictly that is a time-dependent issue, that is not what the distinction refers to in reinforcement learning. It generally means that ...

View answer
Can the mean squared error be negative?
8 votes

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be ...

View answer
Is Experience Replay like dreaming?
Accepted answer
8 votes

The speaker argued that a dream is a random addition of memories, just as experience replay. The speaker is taking some liberties due to a general lack of scientific understanding of what dreams are. ...

View answer
Do we know what the units of neural networks will do before we train them?
Accepted answer
8 votes

In reverse order to how you asked: all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal This actually happens if you ...

View answer
How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?
7 votes

In general the different reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ are not equivalent mathematically, so you will not find any formal proof. It is possible for the functions to resolve to ...

View answer
Is this idea to calculate the required number of hidden neurons for a single hidden layer neural network correct?
Accepted answer
7 votes

I have an idea to find the optimal number of hidden neurons required in a neural network but I'm not sure how accurate it is. It's a complete non-starter, and there is a no such calculation possible ...

View answer
Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?
7 votes

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write ...

View answer
Can Q-learning be used for continuous (state or action) spaces?
7 votes

Q-learning for continuous state spaces Yes, this is possible, provided you use some mechanism of approximation. One approach is to discretise the state space, and that doesn't have to reduce the ...

View answer
What is the difference between expected return and value function?
Accepted answer
7 votes

There is a strong relationship between a value function and a return. Namely that a value function calculates the expected return from being in a certain state, or taking a specific action in a ...

View answer
1
2 3 4 5
20