# Question for the derivation of the probability of a trajectory

I'm studying reinforcement learning now and I'm quite a newbie to this field. I have some questions about how to derive the equation as below.

$$p_{\theta}(s_{1},a_{1},\dots,s_T,a_T)=p(s_1)\prod_{t=1}^T(\pi_\theta(a_t|s_t)p(s_{t+1}|s_t,a_t))$$

The equation above is the probability of a trajectory.

But in my opinion, the equation should be, $$p_{\theta}(s_{1},a_{1},\dots,s_T,a_T)=p(s_1)\prod_{t=1}^T\pi_\theta(a_t|s_t)\prod_{t=1}^{T-1}p(s_{t+1}|s_t,a_t)$$

I don't know why $$p(s_{T+1}|s_T,a_T)$$ emerges in the original equation because there is no $$s_{T+1},a_{T+1}$$ in the trajectory. I think I'm missing something.

Can somebody help me?

• This might be just bayesian factorization. Do you have the image of the trajectory graph? Aug 9 at 1:14
• @desert_ranger Oh I don't have it. But it's just the general trajectory which starts at s_1 and then do the action a_1, so on and forth. I know it's a bayesian factorization, but I still don't know why s_T+1 comes in the first equation.
– Feel
Aug 9 at 8:04