1
You don't have a full reinforcement learning problem, but appear to have a context-free k-armed bandit problem:
The start state at time $t$ is essentially irrelevant to the problem. It does not impact available actions, reward or next state.
The next state at time $t+1$ is only of interest because it determines the reward.
All actions are effectively ...
Only top voted, non community-wiki answers of a minimum length are eligible