1
$\begingroup$

I would like to use the bottleneck layer of U-Net (the last layer of the encoder) to calculate the similarity between two images. For that, I have to somehow flatten the last layer of the encoder. In my opinion, there are two approaches:

  1. Take the last layer which in my case is $4 \times 4 \times 16$ and flatten it to 1D

  2. Add a dense before the decoder and then reshape the dense 1D layer into 3D

For the second case, I am not sure how this would affect the network. Arbitrarily reshaping a 1D array into a 3D tensor. Could that introduce weird artifacts? Does someone have experience in a similar problem?

Improve this question
$\endgroup$
  • $\begingroup$ If I understood you correctly it's the same (architecturally). Decoder will reshape dense layer into 3d tensor anyway in U-net $\endgroup$ – mirror2image Jun 19 '19 at 4:48
  • $\begingroup$ I don't think that's true. In the paper the bottleneck layer is of dimension 30 x 30 x 1024. $\endgroup$ – oezguensi Jun 19 '19 at 22:33
  • $\begingroup$ Anyway shouldn't be a problem. Dense layer is just 1x1xN convolution layer and however you down/upsample it shouldn't change behavior too much. $\endgroup$ – mirror2image Jun 20 '19 at 4:50
  • $\begingroup$ I think it only doesn't change things if - when shaped into 3D again - the same pixels end up in the same spot. Otherwise the spatial information gets lost. $\endgroup$ – oezguensi Jun 20 '19 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.