The Q function uses the (current and future) states to determine the action that gets the highest reward.
However, in a stochastic environment, the current action (at the current state) does not determine the next state.
How does Q learning handle this? Is the Q function only used during the training process, where the future states are known? And is the Q function still used afterwards, if that is the case?
How does Q learning handle this? Is the Q function only used during the training process, where the future states are known? And is the Q function still used afterwards, if that is the case?
The learned $Q$-function is not only used during training, but also after training (in what we may call "deployment", when we expect a trained agent to behave according to what it has learned).
However, the reliance on future states is only there during training, it is no longer required for deployment.
During training, we use the following $Q$-learning update rule:
$$Q(s, a) \gets (1 - \alpha) Q(s, a) + \alpha \left( R + \gamma \color{red}{\max_{a'} Q(s', a')} \right),$$
where $s'$ is the state we reach after executing $a$ in $s$. Here, the $\color{red}{\text{red}}$ part is the part where we rely on knowledge of the future $s'$. This is available in training because we can simply pick $a$, execute it in $s$, observe $s'$, and only then trigger our update step.
Outside of training (and actually also during training), we also rely on our $Q(s, \cdot)$ function for the selection of actions. We typically select an action $a$ according to $a = \arg\max_a Q(s, a)$; we select the action $a$ that maximises $Q(s, a)$ in our current state $s$. The important thing to note here is that there is no $s'$ term in this description of how we select actions: we do not require knowledge of our future state.
Note: in my answer I decided to answer the question literally as it is written, i.e. I'm explaining how $Q$-learning can still work in the described setting where actions $a$ have no influence whatsoever on the future state reached.
In practice, I would never recommend actually using $Q$-learning in such a setting, and instead refer to Neil Slater's comment about Contextual Multi-Armed Bandit algorithms likely providing a better solution.
