1
$\begingroup$

Why does DeconvNet (Zeiler, 2014) use ReLU in the backward pass (after unpooling)? Are not the feature maps values already positive due to the ReLU in the forward pass? So, why do the authors apply the ReLU again coming back to the input?

update: I better explain my problem:

given an input image $x$ and ConvLayer $CL$ composed of:

  1. a convolution
  2. an activation function ReLU
  3. a pool operation

$f$ is the output of ConvLayer given an input $x$, i.e. $f=CL(x)$.

So, the Deconv target is to "reverse" the output $f$ (the feature map) to restore an approximate version of $x$. To this aim, the authors define a function $CL^{-1}$ composed of 3 subfunctions:

a. unpool

b. activation function ReLU (useless in my opinion, because $f$ is already positive due to the application of the 2. step in $CL(f)$)

c. transposed convolution.

In other words $x\simeq CL^{-1}(f)$ where $CL^{-1} (f) = transpconv(relu(unpool(f)))$. But, if $f$ is the output computed as $f=CL(x)$, it is already positive, so the b. step is useless.

This is what I understood from the paper. Where I wrong?

Improve this question
$\endgroup$
2
  • $\begingroup$ While computing error (training phase) its just like any other layer. Error can be negative, hence the ReLU, I guess. $\endgroup$
    – Karan Shah
    Commented Apr 3, 2020 at 10:15
  • $\begingroup$ but in the paper seems that $CL^{-1}$ is applied to $f$ (which is the feature map computed in the forward pass) $\endgroup$
    – volperossa
    Commented Apr 11, 2020 at 12:06

0

Reset to default

You must log in to answer this question.