Let's say you are training a neural network in an RL setting, where the state (i.e. features/input data) can be the same for multiple successive steps (~typically around 8 steps) of an episode.
For example, an initial state might consist of the following values:
[30, 0.2, 0.5, 1, 0]
And then again the same state could be fed into the neural network for e.g. 6-7 times more, resulting in ultimately the following input arrays:
[[30, 0.2, 0.5, 1, 0], [30, 0.2, 0.5, 1, 0], ..., [30, 0.2, 0.5, 1, 0]]
I know that the value 0 in the feature set depicts that the weight for this feature results in insignificant value.
But what about the repetition of values? How does that affect learning, if it does at all? Any ideas?
Edit: I am going to provide more information as requested in the comments.
The reason I did not provide this information in the first place, is because I thought there would be similarities in such cases across problems/domains of application. But it is also fine to make it more specific.
The output of the network is a probability among two paths. Our network has to select an optimal path based on some gathered network statistics.
I will be using A3C, as similar work in the bibliography has made progress.
The reason the agent is staying in the same state is the fact that the protocol can also make path selection decisions at the same time, without an actual update of network statistics. So in that case, you would have the same RTT for instance.
i. This is a product of concurrency in the protocol
ii. It is expected behavior