I am working in an reinforcement learning environment with 1-dimensional action space. My action is only used at the first timestep of an episode and never again. In other word the action only affects the agent's behavior at timestep 1 and is not used at any of the later timesteps.
To illustrate my specific case, here is an image of my enviroment:
At step 0 red ball recives the action (force at a certain angle) and starts moving. At each timestep the coordinates of the red and blue ball are computed, but we cannot interfere with the enviroment anymore (the final configuration is already determined by the initail impulse and the physics).
Should I make each step an episode so the action will be always applied at each step ? (i.e., should I consider a step the transition between the initial position of the balls and the their final position?).
I am new to reinforcement learning so I apologize if this is a silly question