I currently have a grid of pixels 20x20. Each pixel can be red green blue or black. So I have one hot-encoded the pixels giving a 20x20x4 array for each screen.
For my Deep-Q Network, I have attached two successive screenshots of the screen together giving a 20x20x4x2 array.
I am trying to build a Convolutional Neural Network to estimate the Q values but I am not sure if my current architecture is a good idea. It currently is as shown below:
def create_model(self):
model = Sequential()
model.add(Conv3D(256, (4, 4,2), input_shape=(20,20,4,2)))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Conv3D(256, (2,2,1), input_shape=self.input_shape))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(64))
model.add(Dense(self.num_actions, activation='linear'))
model.compile(loss='mse', optimizer=Adam(self.learning_rate), metrics=['accuracy'])
return model
Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable? I realise answers may be highly subjective but I'm just looking for someone to point out any immediate flaws in the architecture.