I was reading the paper Learning to Prune Filters in Convolutional Neural Networks, which is about pruning the CNN filters using reinforcement learning (policy gradient). The paper says that the input for the pruning agent (the agent is a convolutional neural network) is a 2D array of shape
(N_l, M_l), where
N_l is the number of filters and
M_l = m x h x w (
h are filter dimensions), and the output is an array of actions (each element is 0 (unnecessary filter) or 1 (necessary)) and says in order to approximate gradients we have to sample the output
M times (using the REINFORCE algorithm).
Since I have one input, how can I sample the output distribution multiple times (without updating the CNN parameters)?
If I'm missing something, please, tell me where I'm wrong