1
$\begingroup$

I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.

How can I identify the areas to be corrected to reduce over fitting?

convolutional_model = Sequential()

# 32
convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

# 64
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

convolutional_model.add(Flatten())
convolutional_model.add(Dropout(0.5))

convolutional_model.add(Dense(128, activation='relu'))
convolutional_model.add(Dense(10, activation='softmax'))

print(convolutional_model.summary())
convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)
history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])

scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Please, ask just one question per post! $\endgroup$
    – nbro
    Apr 19, 2019 at 18:25
  • $\begingroup$ Welcome to SE:AI! I'm leaving this question open pending more closevotes, but please consider reducing it to a single question. (You can always ask additional question in separate posts.) $\endgroup$
    – DukeZhou
    Apr 19, 2019 at 19:32
  • 1
    $\begingroup$ Edited the post to reduce number of questions $\endgroup$
    – Maanu
    Apr 20, 2019 at 1:40

1 Answer 1

1
$\begingroup$

It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters.

There could be a lot of things that you could consider to reduce over fitting. Some of them are $L_2$ regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer, etc.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .