# In AlphaZero, do we need to store the data of terminal states?

I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero.

From the paper:

The data for each time-step $$t$$ is stored as ($$s_t, \pi_t, z_t$$) where $$z_t = \pm r_T$$ is the game winner from the perspective of the current player at step $$t$$. In parallel (Figure 1b), new network parameters $$\Theta_i$$ are trained from data ($$s,\pi, z$$) sampled uniformly among all time-steps of the last iteration(s) of self-play

Regarding the policy at time $$t$$ ($$\pi_t$$), I understood this as the probability distribution of taking some action that is proportional to the visit count to each child node, i.e. during MCTS, given some parent node (state) at time $$t$$, if some child node (subsequent state) $$a$$ is visited $$N_a$$ times and all children nodes are visited $$\sum_b N_b$$ times, then the probability of $$a$$ (and its corresponding move) being sampled is $$\frac{N_a}{\sum_b N_b}$$, and this parametrizes the distribution $$\pi_t$$. Is this correct? If this is the case, then for some terminal state $$T$$, we can't parametrize a distribution because we have no children nodes (states) to visit. Does that mean we don't add ($$s_T, \pi_T, z_T$$) to the training data?

Also, a followup question regarding the loss function:

$$l = (z-v)^2 - \pi^T log\textbf{p} + c||\Theta||^2$$

I'm confused about this $$\pi^T$$ notation. My best guess is that this is a vector of actions sampled from all policies in the $$N$$ X $$(s_t, \pi_t, z_t)$$ minibatch, but I'm not sure. (PS the $$T$$ used in $$\pi^T$$ is different from the $$T$$ used to denote a terminal state if you look at the paper. Sorry for the confusion, I don't know how to write two different looking T's)

In theory, I could imagine some cases where training the value head on terminal game states might be slightly beneficial despite not being strictly necessary; it could enable generalisation to similar game states that are not terminal (but close to being terminal), and speed up learning for those. For example, if you have a game where the goal is to complete a line of $$5$$ pieces, training the value head on terminal states where you actually have a line of $$5$$ pieces and have entirely won the game might generalise and speed up learning for similar game states where you may not yet have $$5$$ pieces in a line, but are very close to that goal. That said, intuitively I really don't feel like this would provide a big benefit (if any), and we could probably also come up with cases where it would be harmful.
In the $$\pi^{\text{T}}$$ notation, $$\pi$$ is a vector (for any arbitrary time step, the time step is not specified here) containing a discrete probability distribution over actions (visit counts of MCTS, normalised into a probability distribution), and the $$\text{T}$$ simply denotes that we take the transpose of that vector. Personally I don't like the notation though, I prefer something like $$\pi^{\top}$$ which is more clearly distinct from a letter $$T$$ or $$\text{T}$$.
Anyway, once you understand that to denote the transpose, you'll see that $$\pi^{\top}\log(\mathbf{p})$$ is a dot product between two vectors, which then ends up being a single scalar.