I want to see the effects of changing some training parameters (batch size, learning rate, optimizer...) to the accuracy obtained. The problem is that with the same parameters I get significantlly different results (up to 5%).
I load the same weights before training and I deactivated the shuffle. To my understanding this should be enough to get the same results. The only randomness of the backpropagation comes from the initial weight values, right? What am I missing?
This is how I compiled the model:
model.compile(loss="categorical_crossentropy",
optimizer=keras.optimizers.SGD(lr=0.1, decay=1e-6),
metrics=['accuracy'])
And this is the code that I repeat:
model.load_weights("./saved_weights.h5", by_name=False)
model.fit(x_train, y_train,
batch_size=32,
epochs=1,
initial_epoch=0,
validation_data=(x_test, y_test),
shuffle=False)
I'm using tensorflow 2.0 as a backend in a colabs's CPU.