1
$\begingroup$

I am fine-tuning a VGG16 model on 20 classes with 500k images I was wondering how do you chose the size of the dense layer (the one before the prediction layer which has a size 20). I would prefer not to do a grid search seeing how long it take to train my model.

Also how many Dense layer should I put after my global average pooling ?

base_model = keras.applications.VGG16(weights='imagenet', include_top=False)

  x = base_model.output
  x = GlobalAveragePooling2D()(x)
  x = Dense(???, activation='relu')(x)
  x = Dropout(0.5, name='drop_fc1')(x)
  prediction_layer = Dense(class_number, activation='softmax')(x)

I haven't see particular rules about how its done, are there any ? Is it link with the size of the convolution layer ?

$\endgroup$
2
$\begingroup$

It's depend more on number of classes. For 20 classes 2 layers 512 should be more then enough. If you want to experiment you can try also 2 x 256 and 2 x 1024. Less then 256 may work too, but you may underutilize power of previous conv layers.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Is it based on your own experience something else ? and why would 2 layers of 512 be better than for example a layer of 4096 ? $\endgroup$ – Hadrien Berthier Mar 20 '19 at 11:22
  • $\begingroup$ Just common practice. Historically 2 dense layers put on top of VGG/Inception. It works, so everyone use it. Intuition behind 2 layers instead of 1 bigger is that it provide more nonlinearity. But it's not proven. Also the number of weights would be quite different. It's different balance between weights and input/output pipe size $\endgroup$ – mirror2image Mar 20 '19 at 13:22
  • $\begingroup$ okay thanks :) I was wondering because I saw that a lot but didn't see anything written about $\endgroup$ – Hadrien Berthier Mar 20 '19 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.