# Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]

I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s. The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.

The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.

This is my attempt so far, having this tutorial as a reference.

Here is the code:

First, create the MLP network

def create_mlp(self, arr, regress=False): # for the position input
# define MLP network
print("Array", arr)
model = Sequential()
model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))

# check to see if the regression node should be added
if regress:

# return our model
return model


Then, the CNN

def create_cnn(self, width, height, depth=1, regress=False): # for the matrix
# initialize the input shape and channel dimension
inputShape = (height, width, depth)
output_nodes = 6e2

# define the model input
inputs = Input(shape=inputShape)

# if this is the first CONV layer then set the input
# appropriately
x = inputs

input_layer = Input(shape=(width, height, depth))
conv1 = Conv2D(100, 3, padding="same", activation="relu", input_shape=inputShape) (input_layer)
flat = Flatten()(pool1)
hidden1 = Dense(200, activation='softmax')(flat) #relu

batchnorm1 = BatchNormalization()(hidden1)
output_layer = Dense(output_nodes, activation="softmax")(batchnorm1)
output_layer2 = Dense(output_nodes, activation="relu")(output_layer)
output_reshape = Reshape((int(output_nodes), 1))(output_layer2)
model = Model(inputs=input_layer, outputs=output_reshape)

# return the CNN
return model


Then, concatenate the two

def _build_model(self):
# create the MLP and CNN models
mlp = self.create_mlp(env.stateSpacePos)
cnn = self.create_cnn(3, len(env.UEs))

# create the input to our final set of layers as the *output* of both
# the MLP and CNN
combinedInput = concatenate([mlp.output, cnn.output])

# our final FC layer head will have two dense layers, the final one
x = Dense(len(env.stateSpacePos), activation="relu")(combinedInput)
x = Dense(1, activation="linear")(x)

# our final model will accept categorical/numerical data on the MLP
# input and images on the CNN input, outputting a single value
model = Model(inputs=[mlp.input, cnn.input], outputs=x)

model.compile(loss="mean_absolute_percentage_error", optimizer=opt)

print(model.summary())

return model


I have an error:

A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]


The line of code that gives the error is:

combinedInput = concatenate([mlp.output, cnn.output])


This is the MLP summary

And this is the CNN summary

I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.

• Programming questions are off-topic here. Please, next time, ask this type of question on Stack Overflow. See ai.stackexchange.com/help/on-topic for more details.
– nbro
Nov 15 '20 at 18:03
• Oh okay, sorry.
– Ness
Nov 15 '20 at 18:07
• Here we focus on the theoretical aspects of AI, including RL. So, if you have a question about the theory of RL, feel free to ask it here :)
– nbro
Nov 15 '20 at 18:08
• Duly noted, thank you ^^
– Ness
Nov 15 '20 at 18:09

Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work. Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.

I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.

def create_mlp(self, arr, regress=False):
# define MLP network
print("Array", arr)
model = Sequential()
model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
**model.add.flatten() ### shape = (None, 1600)**
# check to see if the regression node should be added
if regress:
# return our model
return model


And just remove the reshape layer in create_cnn function. (output shape should be = (None, 600)).

then concatenate two model

combinedInput = concatenate([mlp.output, cnn.output]) ## output shape =(None, 2200)

Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in create_mlp function.

Your code should work this way. You can read this simple one for better understanding.

• Thank you so much :D It worked.
– Ness
Nov 15 '20 at 14:41