4
$\begingroup$

I was reading the paper Learning to Prune Filters in Convolutional Neural Networks, which is about pruning the CNN filters using reinforcement learning (policy gradient). The paper says that the input for the pruning agent (the agent is a convolutional neural network) is a 2D array of shape (N_l, M_l), where N_l is the number of filters and M_l = m x h x w (m, l and h are filter dimensions), and the output is an array of actions (each element is 0 (unnecessary filter) or 1 (necessary)) and says in order to approximate gradients we have to sample the output M times (using the REINFORCE algorithm).

Since I have one input, how can I sample the output distribution multiple times (without updating the CNN parameters)?

If I'm missing something, please, tell me where I'm wrong

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I'm not sure what do you mean by one input. The input to the pruning agent is always the same, it's the convolutional layer $W$ of dimension $m \times h \times w$. The layer is taken from baseline model that is pretrained. The input doesn't change it's always the same. The output of the pruning agent is an array of probabilities to prune a specific filter. For example if you have $3$ filters in a layer, the output of the pruning agent will be array of $3$ elements . Let's say its \begin{equation} y = [0.1, 0.6, 0.7] \end{equation} Each of these elements represents probability of pruning filter $i$ in layer $W$. So $0.1$ would be probability to prune filter $1$, $0.6$ to prune filter $2$ and $0.7$ to prune filter $3$. Let's say you sample this distribution $2$ times and you get: $[0, 1, 1], [0, 0, 1]$. That means you would make 2 different models from the original baseline model. First model would have 2nd and 3rd filter pruned in layer $W$, and second model would have 3rd filter pruned. The you run those 2 new models on your train and validation set, calculate objective function $R$. Then you update parameters $\theta$ of your pruning agent based on $R$. The original weights of layer $W$ stay untouched. Then you do another inference of the pruning model $\pi$ with updated parameters $\theta$ (the input is still original $W$). You will get another array of probabilities and you keep repeating previous steps that i described until parameters $\theta$ converge. When they converge you make final pruning.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for your answer! but how can I get these two arrays [0,1,1],[0,0,1] ?in order to sample I normally would do something like action=model.predict(W), if I sample another time I will get the same action array, won't I ? please tell me what I'm missing. $\endgroup$
    – Habib-Allah
    Commented Apr 27, 2020 at 12:55
  • $\begingroup$ That's not what sampling means, sampling translated to code (Python) is something like this : random.random() < prob. It's like flipping a coin. If random number is lower than probability that means you set array element to 0 (or 1 depending what you want to do). $\endgroup$
    – Brale
    Commented Apr 27, 2020 at 13:11
  • $\begingroup$ In the paper (Algorithm 1) they say sample the output distribution & then write Ai=π(θ,W). I'm confused i think. anyway thank you I will try what you said. $\endgroup$
    – Habib-Allah
    Commented Apr 27, 2020 at 13:26
  • $\begingroup$ Yes, that's exactly what I said in the previous comment. $\endgroup$
    – Brale
    Commented Apr 27, 2020 at 13:42
  • $\begingroup$ So this Ai=π(θ,W) means sampling the output distribution randomly ? I thought Ai=π(θ,W) means do something like Ai=agent.predict(W). Thanks :) $\endgroup$
    – Habib-Allah
    Commented Apr 27, 2020 at 13:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .