sorry if bother you but i have been trying to figure something out for 3 weeks.
I coded some deep rlRL algorithms (DQN and SAC) with tf2/keras to solve an environment where a vehicle needneeds to follow the track and avoid crashing tointo one other vehicle (hasthere is only one other vehicle). Whatever iI do, the agent is able to follow road in one way or another but nearly always crashcrashes into other vehicle. I use some kinematics information for observation. (agentthe agent only controls steering)
My observation is the kinematics of the agent and the other vehicle. This includeincludes coordinates, velocities, trigonometric headings, lateral and longitudinal offsetoffsets to the closest lane, and angular offsetoffsets to the lane.
A reward function could be defined to quantify how close is agent to the center. A negative reward (-1) is provided if the agent crashes. A negative reward is provided if the agent runs out off road.
If there is a crash or if the agent goes off-road, the episode is done.
With this information agent is able to follow the road but as soon as it reaches the other vehicle, it crashes into it. I trained 1000 episodes.
What did i try?
I run my codes in different environments and it works. Code structure is not a problem. I added last actions (t and t-1) info to observations. Hyper parameter tuning. Changed crashing reward to different values. Used Stable-Baselines3's PPO algorithm. (Had the same problem.) Added distance and angular between vehicles to the observation space. I slowed down the discovery rate reduction in DQN. Used PER as buffer in DQN.
- I ran my code in different environments and it works. Code structure is not a problem.
- I added last actions (t and t-1) info to observations.
- Hyper parameter tuning.
- Changed crashing reward to different values.
- Used Stable-Baselines3's PPO algorithm. (Had the same problem.)
- Added the distance and angular between vehicles to the observation space.
- I slowed down the discovery rate reduction in DQN.
- Used PER as buffer in DQN.
None of these solved the crashing problem. Is thereDoes anyone have any suggestions? Any idea could help, i really don't know what to try or ideas to solve this. Thanks a lot of folks. Any idea needed. problem?