I have done a lot of research on the internet about Reinforcement Learning and I found encountered methods of Reinforcement Learning: Q-Learning and Deep Q-Learning. And I have developed a vague idea of how these two work.
Before I knew anything about Reinforcement Learning this is how I thought it would work:
Suppose I have 2 virtual players in a game who can shoot each other, one of them is a decent playing hard-coded/pre-coded AI, and the other one is the player I want to train (to shoot the other player and dodge his bullets), the aim of the game would be to get the greatest net score (shots you hit minus shots you took) within 1 minute (a session), and you only have 20 bullets.
You have 3 actions, move left-right (0 = maxleftspeed, 1 = maxrightspeed )
, jump (0=don't jump, 1=jump)
, shoot (0 = don't shoot, 1 = shoot)
.
What I thought was, you could create a basic Feed-Forward Neural Network use the enemy's position and his bullet(s)'s position(s) and bullet direction(s) for the input layer and the action being taken will be given by the (3 nodes in the) output layer.
The untrained player starts off with a randomized algorithm, then (for back-propagation) at the end of each session it modifies one of the parameters by a bit in the neural network, and a new session is started with the slightly modified NN. If this session ends with more points than the previous session, it keeps the changes and makes more changes towards that direction, otherwise, the changes are redone, or possibly reversed. I would visualise this as gradient descent similar to that of Supervised learning.
So my questions are:
- Is something like this already out there? What is it called?
- If nothing like this is out there, could you give me any tips to optimize this method or point out any key points I should keep in minds while carrying this out?
- Since I have written this game, I have control over the speed of the actions, but if I did not, I know this AI would take ages to learn, so is there any way to make the learning faster while still keeping the basic idea in mind?
- How exactly is this different from deep Q-learning (if it is)?
Thanks in advance!