1
$\begingroup$

I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks.

Now, I want to find the non-zero connections that they have in common. Any idea how to visualize this or compute this?

I am using Python 3.7 and TensorFlow 2.0.

After the convergence of a neural network following the random weight initialization, some weights/connections increase (magnitude), while other weights decrease. You can then prune the smallest magnitude weights. I want to compare the remaining weights for say two networks having the same level of sparsity of say 50%. The idea is to have an idea of which weights were pruned away and which weights/connections remain.

$\endgroup$
3
  • $\begingroup$ The paper Visualizing the Hidden Activity of Artificial Neural Networks (by Rauber et al.) could be useful. I haven't yet read it. If I read it, I may provide an answer to this question later. $\endgroup$
    – nbro
    May 21, 2020 at 22:01
  • $\begingroup$ edited question to add more information $\endgroup$
    – Arun
    May 22, 2020 at 1:44
  • $\begingroup$ Thanks! Now, it's definitely clearer! $\endgroup$
    – nbro
    May 22, 2020 at 1:45

0

You must log in to answer this question.