3
$\begingroup$

One of my friends built a version of "Achtung, Die Kurve!", or "Curve Fever" in Python. I was starting to study ML and decided to tackle the game from a learning perspective - write a bot that would crush him in the game. Did some research and found Deep Q learning. Decided to go with that and after a whole lot of throwing around different hyperparameters and layers, I decided I need some help on this. I am new to Deep and Machine Learning in general, so I may have missed things. I was kinda discouraged when I saw that Deep Q is SO impractical currently in the field.

how would you guys tackle this problem? I need some guidance/help building it if someone is up to the task.

|improve this question
$\endgroup$
  • 3
    $\begingroup$ Welcome to ai.se....the best learning happens when you are your own teacher..Try to frame the question in such a way such that their is a specific problem to address...And I am sure someone in this stack will help u.. $\endgroup$ – DuttaA Apr 19 '18 at 11:18
  • 1
    $\begingroup$ @DuttaA I appreciate the Positivity, but... I'm not an implementation expert. I'm more of, per se, a concept aid. I help as much as I can. That amount is usually crippled, as I cannot help much with Python and Java. Wishful thoughts: You don't happen to know any resources for noobs in these languages do you? $\endgroup$ – FreezePhoenix Apr 19 '18 at 14:55
  • 1
    $\begingroup$ @Tech are you fine with embedding the AI in the game? $\endgroup$ – FreezePhoenix Apr 19 '18 at 15:07
  • 1
    $\begingroup$ If so, you need to compile a list of things that would matter to the bot, and embedd a *cough*bug*cough* in the game that reports these stats of your friend when they are asked for. $\endgroup$ – FreezePhoenix Apr 19 '18 at 15:10
  • 1
    $\begingroup$ @Pheo I have done that, yes. I can get the player's position, angle... $\endgroup$ – Tech Apr 19 '18 at 15:25
2
$\begingroup$

Start slowly.

Don't jump straight into Deep Learning, arguably the most complex class in Reinforcement Learning techniques. First work with simpler algorithms, like the original Q-Learning. Define what are good inputs and outputs for your game, and start tuning some hyper-parameters (like future rewards discount factor).

From there, go for Deep Learning. Check other implementations (like DQN, Atari n-step Q-Learning and A3C), and adapt their code to yours, rather than starting from scratch.

|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Thanks for your answer! I didn't use the original Q learning since the environment constantly changes, and I don't see how a Q-table can work with that, but I'll try your suggestions! $\endgroup$ – Tech Apr 20 '18 at 13:16
  • 1
    $\begingroup$ What do you mean, the environment constantly changes? Are you processing image input? Do you mean there is an excessively large state space? You can either pre process the images, or extract features first and use those as input. This is a very common ML step, improving your algorithms input. $\endgroup$ – BlueMoon93 Apr 20 '18 at 16:12
  • 1
    $\begingroup$ I mean that in the game players spawn in random places and as they move, they leave a permanent trail behind them that kills other players if they touch it. My thinking was that Q-tables wouldn't work well with this. Also yes, my input is downized frames in real time. TimeDistributed Conv2D in Keras using 4 frames in 3 channels (different player colors), if that says anything to you. $\endgroup$ – Tech Apr 20 '18 at 19:25
  • $\begingroup$ Yeah, usig raw image pixels as you were probably using works with convolutional layers, but not so well with QTables (too large of a state space). However, you can preprocess the image and obtain simpler states. I.e, extract features from the image and feed a simpler local-perspective grid to QLearning $\endgroup$ – BlueMoon93 Apr 24 '18 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.