I am working in an environment with 3-dimensional action space. The first two actions are only used at the first timestep and never again. The third action is used at every timestep.
Say, the action is $a = (a_1, a_2, a_3)$. At the start of an episode $i$, the agent uses actions $a_1, a_2$ only at timestep 1. Action $a_3$ is used at every timestep in the episode starting from 1 till the horizon H. The agent receives rewards $r_i$ at each timestep $i$ till the end of the episode.
I am using SAC. Since the actions $a_1, a_2$ only affect the agent's behavior at timestep 1 and are not used at any of the later timesteps, I am not sure if the RL policy will get better at choosing "good" values for $a_1, a_2$.
Will the RL be able to learn a good policy even if it doesn't quite see the effects of the first two actions in the episode data except at the first timestep?