1
$\begingroup$

I'm creating an RL application for the game Connect Four.

In general, should I be aiming to create an application that's more generic, which would 'learn' different games, or specific to a particular game (e.g. Connect Four, by assigning greater rewards to certain token positions in the C4 grid)?

Does the difference between the two approaches just come down to adapting their respective reward functions to reward specific achievements or positions (in a board game setting), or something else?

$\endgroup$
1
$\begingroup$

If what you mean by a generic reinforcement-learning application is an application that can learn any game (or some games), then you can't do it. Why? Because the goal of each game isn't the same, so you have to adapt the rewards depending on the game. If you just want to make an AI for Connect Four, I suggest you to make a specific RL application for that game.

I want to mention another thing: you shouldn't give a reward based on token's position, because it's hard to know what token's position is the best. Instead, just assign a big reward to the winner. That way, you're generalizing your algorithm, and you're avoiding your AI to focus on the wrong goal.

You have to be careful, you don't know for sure what token's position is the best. Try to give a reward ONLY when a player wins the game, because you can know for sure that winning is the best thing to do. It may sound idiot said like that, but that way, you're ensuring you AI to learn by itself what token's position is the best.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hey! Thanks for trying to contribute to our site :) Btw, I think you should also mention the case of DQN, if you are familiar with it. Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game? $\endgroup$ – nbro Apr 4 at 1:49
  • 1
    $\begingroup$ Hi Dara, thanks for your reply. Regarding your second paragraph – what's the difference between "you shouldn't give a reward based on token's position" and "giving a score for tokens that are aligned"? Aren't you saying we should give greater rewards when the tokens are in certain positions, this is the same, no? $\endgroup$ – mason7663 Apr 4 at 12:07
  • 1
    $\begingroup$ @mason7663 I think the question I asked above would be a good new post on the site. If you want, you can ask this question(s) in a new post: "Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?". $\endgroup$ – nbro Apr 4 at 13:50
  • 1
    $\begingroup$ @mason7663 Regarding your follow-up question, are you asking if you should allow the agent to learn the actual reward function? This is also an interesting question that you can ask separately. In some cases, you are given only rewards and not necessarily a definition of the reward function. So this may be interesting to ask. $\endgroup$ – nbro Apr 4 at 13:52
  • 1
    $\begingroup$ @mason7663 Actually, I think this is a different question and it's not completely clear to me what you are actually asking :P $\endgroup$ – nbro Apr 4 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.