I have been playing around with the Pong-v0 game. After few days, the AI was able to beat it. I stopped the script for a moment and ran it again. For my surprise it seems it started all over again.

Where and how does OpenAI store the experience?
How can I continue the learning procedure in another computer?

  • $\begingroup$ I think what I'm looking is to learn about tensorflow' saver function... $\endgroup$ – lepe Sep 28 '17 at 4:00
  • $\begingroup$ Related: discuss.openai.com/t/how-do-i-save-and-load-a-trained-agent/570 $\endgroup$ – lepe Sep 28 '17 at 4:03
  • $\begingroup$ Is learning a function of accessible memory evolution panders to genetic learning and its succesion without any consious specific reference to the accessibility of that memory but then again that could be an addition to the destruction of the intangeable to the material alas.. $\endgroup$ – Bobs May 10 '18 at 21:00
  • $\begingroup$ @Bobs I think we have a winner $\endgroup$ – hisairnessag3 May 10 '18 at 22:09
  • $\begingroup$ @hisairnessag3 why do if feel so much the looser though lol :) $\endgroup$ – Bobs May 10 '18 at 22:18

This is sort of a tricky question to answer, by results you could mean 2 possible things. History, which is a culmination of actions, observations and rewards. Or the value func/policy which is the thing you are trying to improve.

With that being said, you for most instances of RL will only be dealing with the last few events in the history, or sampling uniform randomly from it according to your route of exploration. So I don't think this is what you are talking about.

As far as storing your value func/policy OpenAI offers a way to post and to a certain extent store your result. This Readme will show you how to do so.

Edit: OpenAI does not offer anything to store experience out of the box, as I noted in my comment you could pretty easily whip up some code that writes JSON to some file and saving that would be a store of experience. Although, it looks like you already found something that does this in tensorflow.

OpenAI likely hasn't included it up to this point because they don't put a ton into this framework. I would encourage you check out SerpentAI on Github, as it has a lot of features missing in OpenAI.

Hope I answered your question!

  • $\begingroup$ By results I mean the "experience" or "progress" during learning. I followed the link you posted but I don't see where it explain how to store that experience. I replaced "results" for "experience", I hope its clear now. $\endgroup$ – lepe Sep 28 '17 at 3:09
  • 1
    $\begingroup$ Ah ok, I do not believe OpenAI offers anything to do this out of the box. You could pretty easily whip up code to write JSON and save the file which would have as much data as you told it to save. $\endgroup$ – hisairnessag3 Sep 28 '17 at 6:18
  • $\begingroup$ Thank you. Do you know why it is not included? Can you edit your answer, so it address the question (as you did with your comment), so I can accept it? (otherwise, it will not be clear why I accepted it). $\endgroup$ – lepe Sep 29 '17 at 0:47
  • $\begingroup$ It has been done:) $\endgroup$ – hisairnessag3 Sep 30 '17 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.