0
$\begingroup$

I have been learning how to create ResNext neural networks, and am confused how input works with cardinality. In this answer, it seems that it's saying that the data is added together, which I assumed meant some sort of aggregation like element-wise summation, but I've ready other things that say that ResNext undergoes split-transform-merge, the same that inception network uses. I also was looking at this Github repo that is a ResNext model in PyTorch, and while I don't completely understand it, it seemed like it just had multiple convolutions working on different sections of the data, instead of running multiple copies of the data through separate convolutions. This is the section I was looking at:

        width_ratio = out_channels / (widen_factor * 64.)
        D = cardinality * int(base_width * width_ratio)
        self.conv_reduce = nn.Conv2d(in_channels, D, kernel_size=1, stride=1, padding=0, bias=False)
        self.bn_reduce = nn.BatchNorm2d(D)
        self.conv_conv = nn.Conv2d(D, D, kernel_size=3, stride=stride, padding=1, groups=cardinality, bias=False)
        self.bn = nn.BatchNorm2d(D)
        self.conv_expand = nn.Conv2d(D, out_channels, kernel_size=1, stride=1, padding=0, bias=False)
        self.bn_expand = nn.BatchNorm2d(out_channels)

What I would have expected to see (and I am very naive about this), would be concatenating the data to itself cardinality times, and then using cardinality groups. So, does ResNext send identical copies of data through separate branches, or split it into sections and run those sections through separate branches?

$\endgroup$
3
  • $\begingroup$ Your second link doesn't work on my end. It seems to be a Bing link -- if it's search result, can you replace it with a link to the original website? $\endgroup$ Commented Jan 31 at 1:09
  • $\begingroup$ Also, for the third link it'd be more clear if you added the specific snippet of code that you're referring to in your question. $\endgroup$ Commented Jan 31 at 1:10
  • $\begingroup$ My bad, sorry. I just copied the link from the edge search results after I found it, but apparently that doesn't actually copy the link. I fixed it and added some code. $\endgroup$
    – eop3
    Commented Jan 31 at 3:38

1 Answer 1

1
$\begingroup$

I realized that groups in PyTorch works along the same dimension as channels, I thought it worked along the same dimension as the data. I believe the GitHub example does run the same data through each group, it just has an initial convolution to increase the channel length.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .