Following on from my other (answered) question:

With regards to the Markov process (chain), if an environment is a board game and its states are the various position the game pieces may be in, how would the transition probability matrix be initialised? How would it be (if it is?) updated?

  • $\begingroup$ Check out this fun project on reddit. It tries to calculate probability of arriving each square in the game of Monopoly. $\endgroup$
    – Enes
    Jul 26 at 7:15

Transition probability matrix cannot be initialized. Your game world has some rules. Since we do not know these rules we can approximate them. To do this, we should run the game over and over and collect the outcome of each action.

Here is a toy example: assume we control a plane. We want the plane to fly forward, but sometimes there may be wind in a random direction. So the plane won't end up in the desired position. Instead, it has some probability to fly left, right and forward. After the data is collected, we can estimate this probabilities.

Another example is estimating n-gram model: assume we want to train a model that will suggest a word in search field. We would analyze a corpus and calculate all 2 word sequences (for bigram model). Then, if a user starts typing I the model would suggest the word am, because I am occurs more frequently then I did, for example, and hence, the transition probability from I to am is greater.

Here is good book on Language Modeling

But your question suggest me that you want something like reinforcement learning. Check out this video wich is a part of a great lectures series on RL by David Silver


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.