I'm trying to create simple keras NN which will learn to make addition on numbers between 0 and 10. But I am getting the error:

ValueError: Error when checking model target: expected activation_4 to have shape (None, 19) but got array with shape (100, 1)

here is my code:

from keras.models import Sequential
from keras.layers import Dense, Activation
import numpy as np

keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False)

model = Sequential()
model.add(Dense(output_dim=50, input_dim=2))

model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])

x = []
y = []

for i in range(0, 10):
    for j in range(0, 10):
        x.append((i, j))
        y.append(i + j)

x = np.array(x)
y = np.array(y)

model.fit(x, y, nb_epoch=5, batch_size=32)

how to fix that?

  • $\begingroup$ Interesting question! Please note, though, that questions about the implementation of machine learning are best served at Cross Validated or, in some cases, Data Science. This site is for the social, conceptual, and scientific aspects of AI. For more information on our scope, see the help center and/or Artificial Intelligence Meta. (I'm glad you got an answer here already.) $\endgroup$
    – Ben N
    Mar 6 '17 at 15:55

Try to use the model like this, for example:

model = Sequential() 
model.add(Dense(50, input_shape=(2,))) 
model.add(Dense(50, activation='softmax')) 
model.add(Dense(1, activation='linear')) 
model.compile(optimizer='sgd', loss='mse', metrics=["accuracy"])

This means that first layer will have 50 neurons and can receive data in form of matrix with 2 columns and an unspecified number of rows. So you can prepare your data in this form – 2 numbers for adding in each row.

Dense(50, input_shape=(2,))

At the end, you need a layer with 1 neuron and the 'linear' activation, because you expect one simple number as a result.

Dense(1, activation='linear')

And finally, use 'mse' loss function or something similar. 'categorical_crossentropy' is needed for classification tasks, not regression as needed for you. See: https://keras.io/objectives/


You shouldn't use Softmax as an activation function in intermediate layers. Softmax is used to represent a categorical distribution, and should be applied at the point where one makes a categorical prediction (usually the final layer of the network).

Consider replacing you activation function in all layers except the last one with 'relu' or 'sigmoid'.


Not the answer you're looking for? Browse other questions tagged or ask your own question.