# Tag Info

## Hot answers tagged reinforcement-learning

Accepted

### Unclear paragraph in Sutton-Barto on exploration/exploitation relating to bandit like decision tasks

The quoted text is from the end of a paragraph that is explaining some aspects of bandit algorithms, and looking forward to how knowledge of them is applicable in a reinforcement learning context. ...
• 31.7k

### Would AlphaZero perform better if made with transformers?

Yes! LeelaChessZero, an open-source re-implementation and continuation of AlphaZero, has been experimenting with this for a while now. Their strongest networks are currently transformers, not ...
• 448
Accepted

### Unclear point in derivation of action-value function

As $q_{\pi}(s,a)$ is defined in S&B RL book as follows: we define the value of taking action a in state s under a policy $\pi$, denoted $q_{\pi}(s,a)$, as the expected return starting from $s$, ...
• 1,323
Accepted

### REINFORCE with Baseline update rule

The value $\delta$ is already representing a derivative equivalent to derivative of MSE loss for the difference between observed and predicted return. Multiplying it by the gradient of $\hat{v}$ to ...
• 31.7k
Accepted

### Should I use RL for this allocation problem and, if yes, which RL approach?

I was just reading Sutton and Barto last night and a passage stuck with me which I will paraphrase. RL is marked by the evaluation of an action, not the instruction of an action. So I would say if ...
• 550
Accepted

### How can reward at time step $t$ can be a function of a state at time step $t+1$?

There is a free choice when deciding the indexing of rewards in reinforcement learning. You can decide to associate the immediate reward with the time step of state and action that generated it, or ...
• 31.7k

• 1,323
1 vote

### What is the logic in including/not including subscript $\pi$ in in "E" for value functions?

The subscript just indicates over which variable the expectation is taken. For consistency it can be included everywhere. Sometimes, when the variable over which the expectation is taken is obvious, ...
• 474
1 vote
Accepted

### Is it possible to describe wordle as a multi armed bandit problem?

Is it possible to describe wordle as a multi armed bandit problem? Yes, although the data management to present the current choice would involve constructing a state much like a reinforcement ...
• 31.7k
1 vote

### Batch wise Inference to speed up Muzero's MCTS

Batching: A Good Idea You're right, batching is a great way to speed up AlphaZero or MuZero self-play! Your proposed solution of running multiple games in parallel is the easiest way to achieve some ...
• 448
1 vote

### Where can I find good sources (textbook, lecture notes, etc) on multi-armed bandits?

There is the bandit book by Lattimore and Szepesvari, which is, I think, the main reference in the field: you can find pretty everything about Bandits, even more recent and research oriented topics. ...
• 2,748
1 vote

### How to properly model the MDP of a weighted graph with the constraint of only visiting each vertex once (and not get stuck in infinite loops)?

Many environments are non-Markovian. Sometimes based on the perception of the agent a Markovian environment becomes non-Markovian (woods101 with perception aliasing). If the model assumes a Markovian ...
• 550

Only top scored, non community-wiki answers of a minimum length are eligible