0
$\begingroup$

I am starting to learn about Neural Network and I have come into one problem that I am still learning how to figure it out.

I have a dataset with shape (105,96) (105 samples and 95 first column as features and last column for label of each sample). In short data = (105,95) and label = (105,1).

I have build a model for my dataset:

model = Sequential()
model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(3, activation = 'softmax'))

# Compile model
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])

I use this model on 2 purpose and get a curious result:

1. K-Cross Validation:

I set k = 5, batch_size = 7 and epoch = 150 the result I got is: 94.29% (+-7%)

def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(first_layer_neurons, input_dim=input_neurons, activation='relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
    model.add(Dense(second_layer_neurons, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
    model.add(Dense(classes, activation='softmax'))
    model.summary()
    # Compile model
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

estimator = KerasClassifier(build_fn=baseline_model, epochs= epoch, batch_size = batch, verbose=1)
kfold = KFold(n_splits=k, shuffle=True)
results = cross_val_score(estimator, X, y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))

2. Train-Validation-Test split:

I split dataset to train = (90,95); validation_data = (10,95) and test = (5,95) with epoch = 150 and batch_size = 10 and the result I got is: Train = 99%, Validation = 90% and Test is 100%

model = Sequential()
model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(3, activation = 'softmax'))

# Compile model
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])

history = model.fit(X_train, y_train, validation_split = 0.1, epochs = 150, batch_size = 10)

_ , acc_train = model.evaluate(X_train, y_train)
_ , acc_test = model.evaluate(X_test, y_test)

But I have notice that maybe my model got overfitting because when I plot the Accuracy, Loss Plot and it like this:enter image description here

So my question is how I encounter this problem, my model have a very good result but it seem Overfitting, how I can get over the Overfiting for my model (Dropout or add some hidden layer will help?)

I am appreciate all of answers, comment for my post.

Thank you guys very much.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.