How do 3 channels affect a network when detecting human skin (CNN)?

Question

Yeah I know, best title ever. Anyway,

I want to make a neural network which is fed with frames coming from an usb camera. Don't wanna be so specific, so I'm just gonna say that the network's goal is to classify human hand gestures, therefore I need to make sure it can effectively learn how the hand moves around.

My problem is that I've no idea about what happens when having 3 channels instead of 1, I only know that (for 3 channels) it does 3 separate convolution operations with the same kernel, resulting actually in 3 separate layers. How do this 3 channels affect the network? Does it learn from the movement 3 parallel times, then it mixes toghether this 3 "separate movements"? Do I need to make it single channel to help him detect the hand?

PS: the text is problably confusing, but that's because I'm confused to, that's why I'm asking.

Answer 1

3 channels will give you more information about the color of the object and surroundings which might be important in certain cases. For example if you want to classify blue cars and red cars then color of the object is very important. Since your problem is to classify hand gestures then color might not be that relevant. You're not very interested in the color of the hand you only care about it's position so there is a good chance that grayscale images might be enough. You should try with grayscale first and if that works good, if not, try with 3 channels.