In those cases the uncontrollable variables can be included as part of the observation space alongside other controllable ones, but that feels sub-optimal as the model will need to learn which dimensions are controllable and which aren't as a result of training, even though it is known before-hand.
This is not usually a problem. In normal RL the agent does not attempt to learn how its actions directly impact the next state. So it doesn't really matter which parts of the observation change for which reason. In model-free RL methods, the agent simply needs to receive fairly sampled observations of reward and next state. If some variables are critical, but don't change depending on action, then the main design consideration you need is ensuring enough of a range of different values is experienced by the agent during training*.
The exception would be if you are using a technique that tries to learn the environment model, and directly learn/represent $p(r, s'|s,a)$ or the useful but less complete $p(s'|s,a)$. You might want to learn this model if you are writing a model-based agent with planning capability. You may also want to do this as a kind of regularisation of a shared representation that can predict multiple things (e.g. a neural network could be trained under Actor/Critic with 3 output "heads" - one for action values, one for optimal policy choice and another for predicted next state).
If you are trying to learn the environment model, you can insert your knowldge of which state variables are static or progress mathematically independently of the action choice by not having the neural network attempt to learn those parts of the model, instead adding values calculated according to the fixed rule. So you can directly split the feature vector into "action influenced" and "other" parts, for RL training purposes only bother learning the "action influenced" part, and then recombine the split parts if you actually need a prediction for a planning algorithm. You don't have to do this - if state progression of any feature is complex enough, even if it is not influenced by the action choice, then you may be better off having the nn predict the next value anyway.
Regardless of whether you are trying to get the agent to the learn the environment model or not, then during training you do have the responsibility to control the added variables somehow. Any observational feature which may change expected reward for different actions, needs to be observed at multiple possible values, in order for the agent to learn to behave optimally as the feature changes over time - whether that is fixed in different scenarios, or changes faster than that, independently of the agent's action, the action value predictions and optimal policy predictions need to see a range of combined values in order to train successfully.
The "control" on the added environment variables can be as simple as waiting until multiple possible combinations have been experienced. However, in simulations you may have the ability to provide more direct control by varying scenarios according to a useful distribution for training.
* If all your state variables progress without regard to the action taken, you might be able to simplify the problem from Reinforcement Learning to Contextual Bandit.
