I want to see the effects of changing some training parameters (batch size, learning rate, optimizer...) to the accuracy obtained. The problem is that with the same parameters I get significantlly different results (up to 5%).
I load the same weights before training and I deactivated the shuffle. To my understanding this should be enough to get the same results. The only randomness of the backpropagation comes from the initial weight values, right? What am I missing?
This is how I compiled the model:
model.compile(loss="categorical_crossentropy", optimizer=keras.optimizers.SGD(lr=0.1, decay=1e-6), metrics=['accuracy'])
And this is the code that I repeat:
model.load_weights("./saved_weights.h5", by_name=False) model.fit(x_train, y_train, batch_size=32, epochs=1, initial_epoch=0, validation_data=(x_test, y_test), shuffle=False)
I'm using tensorflow 2.0 as a backend in a colabs's CPU.