Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]

Question

I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s. The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.

The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.

This is my attempt so far, having this tutorial as a reference.

Here is the code:

First, create the MLP network

def create_mlp(self, arr, regress=False): # for the position input
        # define MLP network
        print("Array", arr)
        model = Sequential()
        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
        model.add(Dense((env.rows * env.cols)//2, activation="relu"))
        
        # check to see if the regression node should be added
        if regress:
            model.add(Dense(1, activation="linear"))
            
        # return our model
        return model

Then, the CNN

def create_cnn(self, width, height, depth=1, regress=False): # for the matrix
        # initialize the input shape and channel dimension
        inputShape = (height, width, depth)
        output_nodes = 6e2
        
        # define the model input
        inputs = Input(shape=inputShape)

        # if this is the first CONV layer then set the input
        # appropriately
        x = inputs
        
        input_layer = Input(shape=(width, height, depth))
        conv1 = Conv2D(100, 3, padding="same", activation="relu", input_shape=inputShape) (input_layer)
        pool1 = MaxPooling2D(pool_size=(2,2), padding="same")(conv1)
        flat = Flatten()(pool1)
        hidden1 = Dense(200, activation='softmax')(flat) #relu

        batchnorm1 = BatchNormalization()(hidden1) 
        output_layer = Dense(output_nodes, activation="softmax")(batchnorm1) 
        output_layer2 = Dense(output_nodes, activation="relu")(output_layer) 
        output_reshape = Reshape((int(output_nodes), 1))(output_layer2)
        model = Model(inputs=input_layer, outputs=output_reshape)

        # return the CNN
        return model

Then, concatenate the two

def _build_model(self):
        # create the MLP and CNN models
        mlp = self.create_mlp(env.stateSpacePos)
        cnn = self.create_cnn(3, len(env.UEs))
        
        # create the input to our final set of layers as the *output* of both
        # the MLP and CNN
        combinedInput = concatenate([mlp.output, cnn.output])
        
        # our final FC layer head will have two dense layers, the final one
        # being our regression head
        x = Dense(len(env.stateSpacePos), activation="relu")(combinedInput)
        x = Dense(1, activation="linear")(x)
        
        # our final model will accept categorical/numerical data on the MLP
        # input and images on the CNN input, outputting a single value
        model = Model(inputs=[mlp.input, cnn.input], outputs=x)
        
        opt = Adam(lr=self.learning_rate, decay=self.epsilon_decay)
        model.compile(loss="mean_absolute_percentage_error", optimizer=opt)
        
        print(model.summary())
        
        return model

I have an error:

A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]

The line of code that gives the error is:

combinedInput = concatenate([mlp.output, cnn.output])

This is the MLP summary

And this is the CNN summary

I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.

Answer 1

Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work. Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.

I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.

def create_mlp(self, arr, regress=False): 
        # define MLP network
        print("Array", arr)
        model = Sequential()
        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
        model.add(Dense((env.rows * env.cols)//2, activation="relu"))
        **model.add.flatten() ### shape = (None, 1600)**
        # check to see if the regression node should be added
        if regress:
            model.add(Dense(1, activation="linear"))
        # return our model
        return model

And just remove the reshape layer in create_cnn function. (output shape should be = (None, 600)).

then concatenate two model

combinedInput = concatenate([mlp.output, cnn.output]) ## output shape =(None, 2200)

Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in create_mlp function.

Your code should work this way. You can read this simple one for better understanding.