Popular deep learning frameworks have different default data formats (PyTorch with NCHW, TensorFlow with NHWC), which lead to published pretrained models with weights that may have an incompatible data format. Many searches and a lot of disappointing finds after, I believe there is currently no simple way to convert trained weights from one data format to the other (only PyTorch might have something in that regard), but I can't understand why this is the case.
Let us consider the following example (I'll use pseudo-python syntax for convenience):
filters = np.random.random([5,5,1,10], dtype=np.float32) # convolution filters for a 2D 5x5 conv on an input with 1 channel, output has 10 channels. image_nhwc = np.random.random([1,28,28,1], dtype=np.float32) # one image in NHWC data format image_nchw = np.transpose(image_nhwc.copy(), (0,3,1,2)) # same image in NCHW format
Ignoring for the moment padding issues, strides, dilations, etc, and assuming there is a
conv2d method that implements the simple convolution of
image (and takes the data format as a parameter), I believe there ought to be some permutation
filters such that:
out_nchw = conv2d(image_nchw, filters, 'NCHW') out_nhwc = conv2d(image_nhwc, perm(filters), 'NHWC') assert np.allclose(out_nhwc == np.transpose(out_nchw, (0,2,3,1)))
i.e, the result of the two convolutions is the same after adapting the data format of the outputs to be the same (think of it as "I have trained weights for the NCHW model, applying
perm I get the trained weights for the NHWC equivalent model").
For how hard I think about this, however, I can't come up with what
perm would be.
So, the question finally: Am I right in thinking that
perm ought to exist? If so, how does one calculate it?