1
$\begingroup$

Why can an AI, like AlphaStar, work in StarCraft, although the environment is only partially observable? As far as I know, there are no theoretical results on RL in the POMDP environment, but it appears the core RL techniques are being used in partially observable domains.

$\endgroup$
  • $\begingroup$ See this related but more specific question: Can Q-learning be used in a POMDP?. $\endgroup$ – nbro Apr 3 at 20:30
  • $\begingroup$ It's not even clear if SC2 can be represented as any type of markov model, observable or not but the system still works. $\endgroup$ – FourierFlux Apr 3 at 20:35
  • $\begingroup$ I am not familiar with AlphaStar or StarCraft to explain the success of AlphaStar, but this is not the first time that a certain assumption doesn't probably hold in the real-world problem, but the applied method (that assumes that such an assumption holds) still performs decently or even well. For example, if I recall correctly, naive Bayes makes some assumptions that are just unrealistic, but, in practice, it still works in many cases. Why does it work? I don't know because I am not an expert on naive Bayes and the problems it's been applied to. $\endgroup$ – nbro Apr 3 at 20:39
  • $\begingroup$ However, I can say that, in many cases, people try to make the environment an MDP (or they try to make the Markov property hold) by doing some tricks. For example, a typical trick in RL is to combine different successive frames of a video (or video game) in order to build a state, rather than using only one frame as the state. $\endgroup$ – nbro Apr 3 at 20:40
  • $\begingroup$ Also, note that POMDPs are not exactly what you are thinking of. In POMDP, the agent doesn't know the state it is in, so it maintains a "belief" (i.e. a probability distribution) over the possible states. But your question is still very interesting and it is definitely legitimate! $\endgroup$ – nbro Apr 3 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.