You can definitely define an environment that accepts more types of action, including actions that take multiple steps in a direction. The first thing you would need to do is implement support for ...

So is the baseline expected to be used in the next iteration when our policy has changed? Yes. To compute the advantage we subtract the state value $V(s_{t})$ from the action value $Q(s_{t},a_{t})$...

From what I can find, reinforcement algorithms work on a grid or 2-dimensional environment. A lot of teaching materials use a "grid world" presentation to demonstrate basic reinforcement learning (RL)...

This depends on the complexity of the environment being learned, and the purpose for learning it. There is no general answer. For the simple environments used to teach reinforcement learning (RL), ...

should I let both 'players' use, and update, the same Q Table? Yes this works well for zero-sum games, when player 1 wants to maximise a result (often just +1 for "player 1 wins") and player 2 wants ...

You should look into contextual bandits, and specifically gradient bandit solvers (see section 13). Your derivation of the gradient seems correct to me. Instead of a sampled/bootstrapped value ...

Is there a way to estimate a behavior policy from this dataset so that it can be used in an off-policy learning algorithm? If you have enough examples of $(s,a)$ pairs for each instance of $s$ then ...

It looks generally valid to me. There are a couple of things missing/implied that I'd like to give feedback on though: I am not using replay here Then it won't work, except for the most simple and ...

This kind of problem does not really have a name other than "toy problem" since no-one needs to teach an AI to add up, multiply or divide* - there are already far more reliable and far faster ways to ...

The most likely outcome of this approach is wasted time and very little effect on accuracy. There will be changes to the model. Some will be beneficial and improve the model, but some will backfire ...

What is the technical name for this kind of training? The name for the problem is Sequential Decision Making or Optimal Control. There are a few different approaches you can take when solving this ...

Can an RL algorithm trained in one environment be succesfull on a different one? Strictly the answer here is "no". You train an agent to solve a single environment. If a second environment is ...

This approach: train it as a classification problem which output confidence, and hope that the confidence will reflect the actual probability. Sometimes the network would output the correct ...

A neural network essentially is a function: $$\mathbf{y} = f(\mathbf{x}, \mathbf{\theta})$$ Where $\mathbf{x}$ is a vector input, $\mathbf{\theta}$ are changeable or learnable parameters, and \$\...

I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages? It is usually just for memory use ...

Where can I find sources showing that policy gradients initialize with random policies, whereas Q-Learning uses epsilon-greedy policies? You can find example algorithms for Q learning and policy ...

It is possible in principle, but you will end up emulating floating point arithmetic using integers in multiple places, so it is unlikely to be efficient for general use. Training is likely to be an ...