1
$\begingroup$

So I have some games that I like, and I'd like to create a net that can play them, just for fun. But I don't have their source code, so I can't just pull the information I want and create a state from them. I should have mentioned that most of the games I'd like the net to play are mobile games, and I am using bluestacks for emulation.

What I thought of

  • Finding alternative open-source versions.
  • Taking screenshots of in-game items and then using something like pyautogui to find them on the game window.
  • Using Cheat Engine to extract the info from memory.

Why are those not viable options

  • If there are open-source versions available, they are usually not that good, plus they are not the original game, so it doesn't fit my goal since I want the AI to play the actual game that most people are playing.
  • This is A LOT of work; it only sometimes works, and if there's an update that changes the shapes of the textures, I need to do it all again. (colour itself would be fine, I could use grayscale). And the process of finding images on images is not that fast.
  • This is also a horrible amount of work, and it's just not possible if you need a lot of information.

For example, I want to create a Deep Q-Learning net to play the Subway Surfers game. But I don't have the source code, and no mods allow reading the game info. So how would you guys approach this issue? I don't want to recreate the games myself because of what I mentioned in "Why those are not viable options". Any ideas and or solutions would be well appreciated.

Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I would approach following two parallel paths. I should mention that I greatly prefer developing reinforcement learning systems in which I provide only pixel data and a reward along with the action space. I prefer not to extract or engineer additional features.

  1. Use Pillow to crop out the game score (or whatever other things matter that are visible as text) from the game screen. Build and train a CNN to convert from image to text to extract the data used to build reward information.

  2. Build a reinforcement system that is fed the pixel data with whatever preprocessing, frame stacking, or other things you need for your training environment, leveraging the image to text data from step 1.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Concerning step 1 : It is not clear to me why training a CNN for this would be a better approach than using a pretrained (eventually fine-tuned) OCR system ? $\endgroup$
    – kirua
    Nov 23, 2022 at 13:27
  • $\begingroup$ It's not better or worse. The OP asked how someone might approach it. This is the approach I would take. Of course you can use an OCR system, but you must be mindful of unusual fonts used in a game, so at a minimum transfer learning would most likely be required. $\endgroup$
    – David Hoelzer
    Nov 23, 2022 at 14:44
  • $\begingroup$ Thank you, the second option greatly helped! $\endgroup$
    – Skyr
    Nov 23, 2022 at 17:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .