0
$\begingroup$

Popular deep learning frameworks have different default data formats (PyTorch with NCHW, TensorFlow with NHWC), which lead to published pretrained models with weights that may have an incompatible data format. Many searches and a lot of disappointing finds after, I believe there is currently no simple way to convert trained weights from one data format to the other (only PyTorch might have something in that regard), but I can't understand why this is the case.

Let us consider the following example (I'll use pseudo-python syntax for convenience):

filters = np.random.random([5,5,1,10], dtype=np.float32) # convolution filters for a 2D 5x5 conv on an input with 1 channel, output has 10 channels.

image_nhwc = np.random.random([1,28,28,1], dtype=np.float32) # one image in NHWC data format
image_nchw = np.transpose(image_nhwc.copy(), (0,3,1,2)) # same image in NCHW format

Ignoring for the moment padding issues, strides, dilations, etc, and assuming there is a conv2d method that implements the simple convolution of filters on image (and takes the data format as a parameter), I believe there ought to be some permutation perm of filters such that:

out_nchw = conv2d(image_nchw, filters, 'NCHW')
out_nhwc = conv2d(image_nhwc, perm(filters), 'NHWC')

assert np.allclose(out_nhwc == np.transpose(out_nchw, (0,2,3,1)))

i.e, the result of the two convolutions is the same after adapting the data format of the outputs to be the same (think of it as "I have trained weights for the NCHW model, applying perm I get the trained weights for the NHWC equivalent model").

For how hard I think about this, however, I can't come up with what perm would be.

So, the question finally: Am I right in thinking that perm ought to exist? If so, how does one calculate it?

$\endgroup$
1
  • 1
    $\begingroup$ If anyone reading has suggestions for better tags for this, please add them or suggest the edit, I couldn't come up with anything better than this... $\endgroup$ – GPhilo Jan 17 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.