# Different result from k-cross validation model and Train-Validation-Test split model ? (AI fresher question)

I am starting to learn about Neural Network and I have come into one problem that I am still learning how to figure it out.

I have a dataset with shape (105,96) (105 samples and 95 first column as features and last column for label of each sample). In short data = (105,95) and label = (105,1).

I have build a model for my dataset:

model = Sequential()
model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))

# Compile model
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])


I use this model on 2 purpose and get a curious result:

1. K-Cross Validation:

I set k = 5, batch_size = 7 and epoch = 150 the result I got is: 94.29% (+-7%)

def baseline_model():
# create model
model = Sequential()
model.add(Dense(first_layer_neurons, input_dim=input_neurons, activation='relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(second_layer_neurons, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.summary()
# Compile model
return model

estimator = KerasClassifier(build_fn=baseline_model, epochs= epoch, batch_size = batch, verbose=1)
kfold = KFold(n_splits=k, shuffle=True)
results = cross_val_score(estimator, X, y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))



2. Train-Validation-Test split:

I split dataset to train = (90,95); validation_data = (10,95) and test = (5,95) with epoch = 150 and batch_size = 10 and the result I got is: Train = 99%, Validation = 90% and Test is 100%

model = Sequential()
model.add(Dense(200, input_dim = 95, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))
model.add(Dense(30, activation = 'relu', bias_initializer = 'ones', kernel_initializer = 'normal'))

# Compile model
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])

history = model.fit(X_train, y_train, validation_split = 0.1, epochs = 150, batch_size = 10)

_ , acc_train = model.evaluate(X_train, y_train)
_ , acc_test = model.evaluate(X_test, y_test)


But I have notice that maybe my model got overfitting because when I plot the Accuracy, Loss Plot and it like this:

So my question is how I encounter this problem, my model have a very good result but it seem Overfitting, how I can get over the Overfiting for my model (Dropout or add some hidden layer will help?)

I am appreciate all of answers, comment for my post.

Thank you guys very much.