2
$\begingroup$

I have trained a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%). now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)

$\endgroup$
2
$\begingroup$

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.

Then, there are several methods and considerations:

  • You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.

  • If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.

  • Or, you can use the concept of the selffer network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters

You can read more about the transfer learning concept (or selffer network) to get the most appropriate method for your case. There is also a paper about "incremental training on CNN" that I think is similar with the selffer network but with some modifications.

Hope it helps

$\endgroup$
  • $\begingroup$ "I assume you've tuned your hyperparameters", if i understood you correctly, yes, i reached this accuracy after trying different learning rates, and adding dropout layers. (i have not change other parameters of vgg). i tried using the previous weights into training the network again by only changing learning rate (learning rate of the new training is smaller than the previous training). but this method did not change the accuracy. $\endgroup$ – norahik Apr 19 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.