4
$\begingroup$

Youtube was recently suggesting to me videos of people training NEAT neural networks for video games. I've noticed that often the training process was quite slow (for example in this Trackmania example).

Is there a way (algorithmic approach or an idea) to easily simulate video games, without actually rendering the pixels on screen and making the training much quicker? In addition to that, if you also know of a tool that does that, please, share it with me.


Update:

Following the Trackmania example, I found out the youtuber uses a standard tool called "TMInterface", and the webpage of the tool states it is a "TAS tool" - Tool-assisted speedrun.

I'll investigate how such tools work and whats the idea behind them (and if indeed they do what I think they do). Will update!

$\endgroup$
5
  • 1
    $\begingroup$ If you're asking for a tool, this question is off-topic. If you're looking for an approach/algorithm, then it could be on-topic. Questions that ask for software libraries should be asked on softwarerecs.stackexchange.com or DS SE. We focus on the theoretical/conceptual, philosophical and social aspects of AI (although we have had very few questions on the latter 2 topics recently). See ai.stackexchange.com/help/on-topic. Please, edit your post to clarify that you're not asking for a tool (unless that's an additional request), if you don't want your post to be closed. $\endgroup$
    – nbro
    Jun 5, 2022 at 9:21
  • $\begingroup$ I don't think this is a tool as much as "why the process" sort of question. The deepmind atari, afaict, didn't use the screen, but used a representation in memory. Some versions (starcraft) can also look at other variables in memory like positions of objects. $\endgroup$ Jun 6, 2022 at 16:45
  • $\begingroup$ @EngrStudent How can you access the variables in memory and interact with the game without the game wasting time rendering? $\endgroup$ Jun 6, 2022 at 17:02
  • $\begingroup$ I think some of the answers have them. The screen is a pixel map. Rendering seeks to fill in all the pixels with intensity values. The internal representation that the computer holds would be very inefficient if it was all pixels. The JPEG image is discrete cosine coefficients. If you can expose the learner to the input to the renderer instead of its output, then you can give it a compressed form before that form is burdening the silicon. $\endgroup$ Jun 6, 2022 at 18:13
  • 1
    $\begingroup$ Note that graphics don't have to be rendered in real time and don't have to be rendered to the screen! You may be able to patch the game to render to a framebuffer and run as fast as possible (instead of slowing itself down to 100% normal speed) $\endgroup$
    – user253751
    Jun 7, 2022 at 9:42

2 Answers 2

7
$\begingroup$

It depends on the game environment and on the model being trained.

  • If you are training an agent that uses vision to decide action, then typically you need a copy of the rendered screen:
    • If that comes from a modern game using GPU acceleration to generate output, then you will likely need to leave that as is, and also transfer the rendered output back to the main memory, so it will likely be slower than running the game normally. There may be game settings that help improve speed.
    • If the output is from a game emulator for older games (such as Atari), then it probably won't use a GPU, and you will not need to spend time rendering to the screen. The emulator may be able to run in a mode that doesn't render, whilst still giving in-memory access to the screen buffer. It may also be able to run faster than real time. It will depend on emulator configuration settings that are available to you.
  • If the game is written "natively" in a simulator or semi-native environment like Pygame or Box2D, you should be able to disable rendering and save the time. This is how a lot of OpenAI Gym's environments are set up, and these do run a lot faster when not rendering - a simple flag you can set when starting the environment and/or starting an episode. You may even notice with these environments that putting the view window for the environment behind another window will speed things up considerably (but for automation it is more reliable to disable rendering).

When the environments have been set up via a shared library, like Open AI's Gym, then the internal logic for deciding rendering (or not) will be different for each environment, but there may be a standardised config/method argument to determine whether the game engine is rendered to screen. That could apply to nearly all the environments available in the library. You should check the documentation if for example you are trying out NEAT on some Atari games that have been ready packaged for use with computer agents.

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed explanation! Talking about OpenAI, when they created openAI 5 for the "Dota 2" game, did they create a copy of it without rendering to allow fast training? $\endgroup$ Jun 5, 2022 at 5:18
  • $\begingroup$ @snatchysquid: I don't know, but my understanding is that the RL environment for DoTA is not based on the pixels, but on the object models in the game via an API, which implies the kind of low-level access where that is possible. $\endgroup$ Jun 5, 2022 at 7:29
3
$\begingroup$

Bypassing graphics

As mentioned in Neil Slater's answer, manipulating the engine to bypass graphics rendering to speed up AI simulation can be a valid approach. I have done that myself.

But there are some caveats to consider/work around:

  • The graphics subsystem can be used for core gameplay programming as well. For example, the picking problem (Which object in the scene is the user selecting/clicking on?) can be solved by rendering and evaluating framebuffers.
  • If the game has an FPS limiter, then you need to raise that FPS limit. Otherwise it quickly becomes the new bottleneck.
  • If the game also has real-time timers (i.e. wait five seconds to trigger some event), then your ultra-fast gameplay simulation can cause unintended effects/bugs. This is especially true if you break the aforementioned FPS limit.
  • Sometimes games can experience bugs and crashes when running at high framerates. (Take this PCGamingWiki entry on Peter Jackson's King Kong Gamer's Edition as an example.) With high framerates, gameplay simulation may operate with very small timesteps and glitches in physics and collision detection become more likely. Such effects will only intensify with ultra-high framerates from the accelerated AI simulation.

DLL injection

It is not uncommon for game mods to hijack or replace DLLs, including those of the graphics API. If you cannot manipulate the engine itself, then perhaps you can inject an alternative graphics API implementation that consists of stubs and mockups.

Bypass whatever is irrelevant to gameplay

Depending on the game and the goals of your project, you might take even more drastic steps to accelerate the simulation:

  • Bypass the audio layer as well. Besides graphics, audio is another major output channel that is often irrelevant to AI simulation.
  • Skip through cutscenes and dialogue, like a speedrunner would do.
  • If walking is not essential to gameplay, let the avatar teleport instead of simulating all the walking.
  • Skip animations in general. Always try to fast-forward to the "gameplay outcome" of an action.
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .