4
$\begingroup$

I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments I came across "Blackjack-v0" which is a basic implementation of the game where the state is the hand count of the player and the dealer and if the player has an useable ace. The actions are only hit or stand and the possible rewards 1 if the player wins, -1 if the player loses and 0 when draw.

So that got me thinking what a more realistic environment/model for this game would look like, taking into account the current balance and other factors and has multiple actions like betting 1-10€ and hit or stand.

This brings me to my actual question:

  • As far as I understand neural networks (and I do not very well yet, I guess) the input will be the state and the output the possible actions and how good the network thinks they are/will be. But now there are two different action spaces which apply to different states of the game (betting or playing), so some of the actions are useless. How would be the right way to approach this scenario?

I'm guessing one answer would be to give some kind of negative reward if the network guesses an useless action but in this case I think the reward should be the actual stake (negative reward) and the actual win if any. Therefor this would cause some bias in how the game proceeds as it should start with some amount of balance and end if the balance is 0 or after a specified amount of rounds.

Limiting timesteps wouldn't be an option either I guess because it should be limited to rounds so it won't end after a betting step e.g.

Therefore, for a useless step the reward would be 0 and the state would stay the same but for the network it doesn't matter how many useless steps it takes because it'll make no difference to the actual outcome.

Corollary question:

  • Should be split up into two neural networks? One for betting and one for playing?
$\endgroup$
1
$\begingroup$

There are so many reinforcement learning algorithms. I recommend you this articles:

There are several approaches. A NN can get state and action as an input and expected discounted reward as an output. But you can just not pick an action if this action isn't available and there isn't necessity to give it negative or zero reward just don't learn your net to change it behavior for these actions because they are neither good nor bad.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the links, I'm using a DQN I know there are many algos and many that are better than a DQN but I'm pretty new to this and it was easier to implement, because of more existing examples. I guess I'll have to do a lot more research until I really get all this stuff, but again thanks for the links any resource is good at the moment :) $\endgroup$ – SomeDudeCalledMo Mar 9 '18 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.