4
$\begingroup$

I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully understand how the network worked.

Below is the image of OpenAI five's network OpenAI's model

Basic question

How is the data concatenated/ what is the dimension of the data right after the concatenation before the LSTMs or just before entering the LSTMs? Below are my thoughts which I provide for clarification's sake.

1st Interpretation

In the blue area for units, my initial understanding was that for each visible unit, the output of the max-pool is concatenated along the columns. So, assuming the number of rows is 1(as a 1d max-pool is being applied for each unit), the number of columns is n and there are N visible units, the final size of the matrix when concatenated is $(1,n\cdot N)$ with few extra columns given by the pickups and the like as shown on the left-hand side of the model.

Problem with this interpretation

As the number of units a player can see per each turn is not constant, under this interpretation, I suspect that the fully connected layer after the concatenation layer cannot do its job as matrix multiplication becomes impossible with a variable number of columns.

Possible solution

One possible solution to this is to set a maximum to the number of observed units as $N_{max}$ and pad with constants if some units are not observed. Is this the case?

2nd Interpretation

My 2nd interpretation is that the data is concatenated along the rows. In this case, I can see that the data can pass through a fully connected layer because the number of columns can remain constant. Under this assumption, I decided that right before going through the LSTM, the data is reshaped to (batch size, number of rows, number of columns).

Problems with this interpretation

While I found this interpretation to be more appealing, I noticed that under this train of thought, the LSTM is used just to associate the input data and is not associated with time(the time step for the LSTM is simply the next row of data rather than actual time). I know that this is not especially a problem but I thought that there is no special need to use an LSTM here as in this second interpretation, the order of the data holds no special meaning. But is this the case?

I apologize in advance for any unclear points. Please tell me in the comments and I'll try to clarify as best as I can!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.