Neil Slater
• Member for 5 years, 4 months
• Last seen this week
• Durham, United Kingdom

Is this a valid implementation of second-order regression? No, but it is not far off. To perform a full second-order regression, you will need all terms for $x_{i,j}x_{i,k}$ where the first index is ...

Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information? The stride referred to by the quote "only plays with the size and ...

Intuitively, I feel like if there are 30 foods, each with 2 states, then that is 60 states, no $2^{30}$. Let's try it with 3 pellets. If you are right there would be $2 \times 3 = 6$ states, if the ...

In the question, you are not describing the environment changing. Instead, there is a fixed 20% chance of a bad weather event each year. Such events can me modelled as a static environment with ...

It is important to note that the graph shows reward received during training. This includes rewards due to exploratory moves, which sometimes involve the agent falling off the cliff, even if it has ...

there should be absolutely no problem with training an agent on any available episode roll-out data. That is because a MDP implies for an any state S, the optimal action to take is entirely dependent ...

As you made this experiment available on Colab, I was able to test my thoughts on it, which was handy. First, the simple "fix" is to run many epochs. Eventually even your relatively small ...

As I understand it, the Bellman equation assumes the setting to be deterministic, meaning that, if you're in state $s_t$ as you take action $a_1$, you should always reach the same $s_{t+1}$. This is ...

Why are we choosing more than 1 action in SARSA? There is never a state where more than one action is chosen. The appearance of two Choose statements is an artifact of the loop design and variable ...

Do we only select one action at the very beginning and then we always choose the same action for each step? No. The pseudocode is clear on this, by using the word Choose and referencing a policy. If ...

For learning, it doesn't matter much how you choose the first action before starting the main loop. That is because the agent doesn't need to learn about transitions to the first state of an episode. ...

What explains the apparent 'mirroring' of the graphs on the RHS, The model starts untrained and no better than random guessing (the baseline). As the training progresses, the model does better than ...

Your setting (of randomly dropping out reward signals) impacts expected future reward by multiply everything by a common factor $(1-\epsilon)$. As reinforcement learning (RL) control is based on ...

The tanh functions within the cell represent cell output or cell state. These are the values that either get passed on to other layers, or within the layer to the next time step. In theory, other ...

Reinforcement learning already has the objective of maximising the sum of future expected reward. By making each reward the sum of all previous rewards, you will make the the difference between good ...

The core differences between using $V(s)$ or $Q(s,a)$ are: $V(s)$ cannot be used stand-alone to decide a policy. You either need a separate policy function $\pi(a|s)$ that it is the value function for,...

You cannot code an $\epsilon$-soft policy directly, because it is not specific enough. A policy is $\epsilon$-soft provided that there is at least a probability of $\frac{\epsilon}{|\mathcal{A}|}$ for ...

Probably flipping a video left/right will be OK and useful for your case. When considering data augmentation approaches, you should think about two things that may prevent it working: Could the ...

This would mean we decrease the value of this state. Yes. This update that reduces the estimate is correct because it adjusts for the inevitable over-estimate of value when the exploration policy ...

Can't it be that the optimal policy thinks a state isn't that good and gives him a low value but perform best in comparison with other policies which have higher values for this state? No, this is ...

Is it okay to call anything that needs to be learned outside the training algorithm a hyperparameter? I think so, yes. Personally, I would reserve the term to discuss values that I could choose ...

how should the reward scheme be for a game like this? i.e., whether one action is good or not depends on other actions taken before it? The reward scheme should always be a "natural" ...

My understanding of your environment is: The batch number $b$ is the same thing as a time step $t$. Each batch is associated with a single static representation of the environment, the agent makes ...

You don't have a full reinforcement learning problem, but appear to have a context-free k-armed bandit problem: The start state at time $t$ is essentially irrelevant to the problem. It does not ...

Unfortunately, even with large amounts of training data, hyperparameter choices can strongly influence the performance of a trained model. What you can usually drop when you have large amounts of ...

For a Google term you could use "computational creativity". It covers a wide range of ideas, and the artist here is not using one single tool or approach. There are clearly a range of ...

This one is a bit crazy: pool1 = nn.AvgPool3d(kernel_size = (361, 1, 1), stride= 1) because it averages large numbers of the features at once. Very little information about individual features will ...

The dimensionality used to discuss convolutional layers in CNNs is based on the dimensionality of the input without considering channels. 1D CNNs might process raw audio sources (mono or stereo), ...