1
$\begingroup$

I am trying classify CIFAR10. The CNN that I generated over fits when the accuracy reaches ~77%. The code and the plot is given below. I tried DropOut, Batch Normalization and L2 Regularization. But the accuracy does not go beyond ~77.

How can I identify the areas to be corrected to reduce over fitting?

convolutional_model = Sequential()

# 32
convolutional_model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3), kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

# 64
convolutional_model.add(Conv2D(64, (3, 3), activation='relu', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(Conv2D(128, (3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(.0002)))
convolutional_model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))

convolutional_model.add(Flatten())
convolutional_model.add(Dropout(0.5))

convolutional_model.add(Dense(128, activation='relu'))
convolutional_model.add(Dense(10, activation='softmax'))

print(convolutional_model.summary())
convolutional_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=2, patience=8)
history = convolutional_model.fit(X_Train_Part, Y_Train_Part, epochs=200, verbose=2,validation_data=(X_Train_Validate, Y_Train_Validate), callbacks=[es])

scores = convolutional_model.evaluate(X_Test, Y_Test, verbose=2)

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Please, ask just one question per post! $\endgroup$ – nbro Apr 19 '19 at 18:25
  • $\begingroup$ Welcome to SE:AI! I'm leaving this question open pending more closevotes, but please consider reducing it to a single question. (You can always ask additional question in separate posts.) $\endgroup$ – DukeZhou Apr 19 '19 at 19:32
  • 1
    $\begingroup$ Edited the post to reduce number of questions $\endgroup$ – Maanu Apr 20 '19 at 1:40
1
$\begingroup$

It is a difficult task to identify the areas to be corrected to improve accuracy unless that area is looking into your face. By that I mean unless the regularizing parameter has unusually large or small values it is difficult to pin down and identify which regularizer to tweak. You need to do a grid search or random search over the hyper-parameter space to come up with an optimal combination of hyper parameters.

There could be a lot of things that you could consider to reduce over fitting. Some of them are $L_2$ regularizers, Dropout, depth of the network, number of neurons in the layers, the optimizer, etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.