The concept of channels comes from communications and has little to do with neural networks specifically. We see the concept in AI literature because AI deals with media streams and files.
The concept is this. A media format has channels to allow the independent passage of multiple signals through a single mechanism. That's it.
For example, some streamed broadcast or multicast may have channels for red, green, blue, alpha, left audio, and right audio. In printing, four color is standardized to cyan, magenta, yellow, and black (or key). In jet flight control there are channels for rudder, aileron, elevator, throttle, landing gear, and break positions.
Automated vehicles would have steering, accelerator, break, horn, and two turn signals for control channels and data acquisition channels including an accelerometer, two microphones, probably some strain gauges, four radial encoders, three color channels for each of two cameras, and some thermistors.
In the context of convolution, a kernel may be applied in various ways to individual channels or groups of channels either in parallel or in aggregation. When you turn up the bass on a digital music player, a kernel designed to boost the lower frequencies is applied to both left and right audio channels. Missile countermeasures have kernels with built in hardware trigonometry to track and steer in compound angles with nanosecond response time. Remote control toys have channels through which joystick signals are sent through kernels to coordinate propulsion signals to wheels or props.
For computer vision, consider the five dimensions involved in a data set of 1,000 video examples with resolution of 1,920 x 1,080 progressive 32 bit pixels at 29.97 fps, each example with 9,010 frames.
- Example index
- Frame number
- Vertical index
- Horizontal index
This would fit into an array of bytes dimensioned [1,000][9,010][1,080][1,920]. How the kernel deals with one or more channels and whether the channels interact or are kept independent has to do with the expected purpose of the layer within the deep learning design, and that is dependent upon what is expected in the video input and what is hoped for as output after the network is trained.
- The AI system topology,
- The configuration and dimensions of layers,
- The dimensions of kernels,
- How those kernels deal with the channels, and
- What criteria are used to determine what is back propagated
is a coordinated set of design decisions the AI engineer must make based on theory and probably demonstrate in a POC later on.
The mapping of 1 channel to grayscale and 3 channels to color is an oversimplification. Sometimes color is RGB at the input and output (3 and 3 channels respectively) but averaged to grays through some layers for the sake of computational thrift whereas a normalized representation of color balance is processed in a separate parallel stream. Other times the alpha channel is needed or an alpha and beta channel, so there may be 4 or 5 channels respectively to handle green or blue screen movie making techniques or to support the learning of scene layering.
The AI engineer must first
- Decide what is wanted,
- What must be learned to get it,
- Design a process to achieve it, and
- Select appropriate learning techniques to tune the parameterized process to learn what is necessary.
After such design activity is reasonably complete, the system topology and the dimension count and sizes become clear. The use of channels to fully represent information and what operations must be performed by kernels on those channels also become apparent.