I was reproducing the findings of a research article in which I discovered that they had switched the Channel dimension from last to first. To clarify this concept, I went through A Gentle Introduction to Channels-First and Channels-Last Image Formats . The author of this link stated:
When represented as three-dimensional arrays, the channel dimension for the image data is last by default, but may be moved to be the first dimension, often for performance-tuning reasons.
There are two ways to represent the image data as a three dimensional array. The first involves having the channels as the last or third dimension in the array. This is called “channels last“. The second involves having the channels as the first dimension in the array, called “channels first“.
Channels Last. Image data is represented in a three-dimensional array where the last channel represents the color channels, e.g. [rows][cols][channels].
Channels First. Image data is represented in a three-dimensional array where the first channel represents the color channels, e.g. [channels][rows][cols].
.
We are aware of when the channel was last used and the manner in which kernels were applied, theoretically. However, I'm curious as to when the dimensions of the channel come first. How kernels will process. More precisely:
Assume we have [rows, columns, channels] -> [2,4,3] image dimensions. We may say we have three data channels, each with two rows and four columns, correct?
Alternatively, if we assume that channel dimensions are first means [channels, rows, columns] -> [3,2,4]. In other words, we now have four data channels, each with three rows and two columns, am I correct? If I am taking correctly than this is quite confusing because we are completely modifying our image.
Question:
What is the benefit of shifting the channel dimension first, and how will the kernels move on it?
For more detail check the code:
input_layer = tf.keras.Input(shape=input_shape, name="Time_Series_Activity")
con_l1 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu", data_format='channels_first')(input_layer)
Summary of Code
Layer (type) Output Shape Param #
=================================================================
Time_Series_Activity (InputL [(None, 1, 30, 52)] 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 64, 26, 52) 384
_________________________________________________________________
