I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.

How can I identify the areas to be corrected to reduce over fitting?

convolutional_model = Sequential()

# 32
convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

# 64
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))


convolutional_model.add(Dense(128, activation='relu'))
convolutional_model.add(Dense(10, activation='softmax'))

convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)
history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])

scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)

enter image description here

  • 1
    $\begingroup$ Please, ask just one question per post! $\endgroup$
    – nbro
    Apr 19, 2019 at 18:25
  • $\begingroup$ Welcome to SE:AI! I'm leaving this question open pending more closevotes, but please consider reducing it to a single question. (You can always ask additional question in separate posts.) $\endgroup$
    – DukeZhou
    Apr 19, 2019 at 19:32
  • 1
    $\begingroup$ Edited the post to reduce number of questions $\endgroup$
    – Maanu
    Apr 20, 2019 at 1:40

1 Answer 1


It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters.

There could be a lot of things that you could consider to reduce over fitting. Some of them are $L_2$ regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer, etc.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .