how to benefit from previous training weights in training again to increase accuracy?

Question

I have trained a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%). now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)

Answer 1

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.

Then, there are several methods and considerations:

• You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.

• If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.

• Or, you can use the concept of the selffer network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters

You can read more about the transfer learning concept (or selffer network) to get the most appropriate method for your case. There is also a paper about "incremental training on CNN" that I think is similar with the selffer network but with some modifications.

Hope it helps