1
$\begingroup$

I'm creating an RL application for the game Connect Four.

In general, should I be aiming to create an application that's more generic, which would 'learn' different games, or specific to a particular game (e.g. Connect Four, by assigning greater rewards to certain token positions in the C4 grid)?

Does the difference between the two approaches just come down to adapting their respective reward functions to reward specific achievements or positions (in a board game setting), or something else?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

If what you mean by a generic reinforcement-learning application is an application that can learn any game (or some games), then you can't do it. Why? Because the goal of each game isn't the same, so you have to adapt the rewards depending on the game. If you just want to make an AI for Connect Four, I suggest you to make a specific RL application for that game.

I want to mention another thing: you shouldn't give a reward based on token's position, because it's hard to know what token's position is the best. Instead, just assign a big reward to the winner. That way, you're generalizing your algorithm, and you're avoiding your AI to focus on the wrong goal.

You have to be careful, you don't know for sure what token's position is the best. Try to give a reward ONLY when a player wins the game, because you can know for sure that winning is the best thing to do. It may sound idiot said like that, but that way, you're ensuring you AI to learn by itself what token's position is the best.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Hey! Thanks for trying to contribute to our site :) Btw, I think you should also mention the case of DQN, if you are familiar with it. Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game? $\endgroup$
    – nbro
    Apr 4, 2020 at 1:49
  • 1
    $\begingroup$ Hi Dara, thanks for your reply. Regarding your second paragraph – what's the difference between "you shouldn't give a reward based on token's position" and "giving a score for tokens that are aligned"? Aren't you saying we should give greater rewards when the tokens are in certain positions, this is the same, no? $\endgroup$
    – mason7663
    Apr 4, 2020 at 12:07
  • 1
    $\begingroup$ @mason7663 I think the question I asked above would be a good new post on the site. If you want, you can ask this question(s) in a new post: "Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?". $\endgroup$
    – nbro
    Apr 4, 2020 at 13:50
  • 1
    $\begingroup$ @mason7663 Regarding your follow-up question, are you asking if you should allow the agent to learn the actual reward function? This is also an interesting question that you can ask separately. In some cases, you are given only rewards and not necessarily a definition of the reward function. So this may be interesting to ask. $\endgroup$
    – nbro
    Apr 4, 2020 at 13:52
  • 1
    $\begingroup$ @mason7663 Actually, I think this is a different question and it's not completely clear to me what you are actually asking :P $\endgroup$
    – nbro
    Apr 4, 2020 at 14:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .