I am correct in my understanding that you only provide the agent with the state of the car, i.e. a global x and y position, its angle, velocity, and steering angle?
How does the agent know that it is coming closer to the goal if it is not provided with information about where the goal is? Without this observation of the goal, the agent is operating blindly. That explains why it is so difficult for the agent to reach the goal and impossible when you randomize the starting position.
If my assumptions are correct, the agent takes random actions which are unlikely to reach the goal, but due to the law of large numbers after enough episodes, the agent will reach the goal at random and it can learn to remember this path if given enough reward. But if you then randomize the starting position the agent cannot apply the knowledge it has learned previously because the sequence of actions to reach the goal would now be different. Essentially, there is no correlation between what goal you want the agent to achieve and your state and action space.
To circumvent this problem, I suggest you add additional state information, here are a few suggestions:
- The global x and y position of the goal
- A distance measure measuring the distance from the agent and to the goal. Either the Euclidean distance or Manhattan distance.
- Both of the above
I also support the suggestion of Elfurd: "Training in steps can be useful". This is called curriculum learning and the idea is to present easier training examples to the agent at the beginning of training and steadily increase the difficulty of the environment. In turn, the agent will reach the goal in the easier environments, obtain some reward, and learn. It can then apply what it has learned in the more advanced environments once it progresses through the curriculum.
In your environment, this could be as simple as decreasing the size of the gird world in early training. Or you could spawn the agent close to the goal so that the agent is more likely to reach the goal with just a few random actions, alternatively, you can also randomize the goal close to the starting position of the agent if it has to start from a specified position and then increase the distance to where the goal is sampled.