1
$\begingroup$

I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs?

It sounds that my neural network will have too many inputs if I do it that way. Is that a problem? What is the recommended way of feeding an image through a neural network?

| improve this question | |
$\endgroup$
1
$\begingroup$

If you are using a fully connected network (aka an MLP) and images with one channel (grey scale) and 100 x 100 = 10,000 pixels, then yes, MLP would have 10k inputs and 10k x N 1 trainable weights in the first layer (as noted by Neil Slater). If you have a color image with 3 channels, e.g. RGB, then you can expect 3 times as many weights because there are 3 times as many values used to represent the image.

A convolutional neural network is a common architecture for analyzing images. For a 100x100 (10k) pixel image, the input layer might have 3x3x1x32 = 288 weights (for 1 channel) or 3x3x3x32 = 864 weights in the first layer, much less than 10k x N 1 from a fully connected network. This would transform your image into a 98x98x32 size image. The main point is that you would have 3x3 weights per input channel per output channel at each layer, instead of 10k weights per input channel per output channel. CNNs also give you some invariance properties that are usually nice in machine learning with images.

For images in general, having a lot of weights is normal. ImageNet (linked above) has 60 million weights to train. Typically special hardware, like a GPU is used to handle this many weights. If you are using just a CPU, your model may not train well in any reasonable amount of time, i.e. years.

| improve this answer | |
$\endgroup$
  • $\begingroup$ A fully-connected network would actually have $10000 \times N^1$ weights where $N^1$ is the number of neurons in the first hidden layer. Assuming your CNN example is expanding number of channels from 3 to 32, its 288 weights when fully converted into a $3 \times 10000$ input feeding a $32 \times 10000$ first hidden layer, would have 9.6 billion weights to train! $\endgroup$ – Neil Slater Aug 10 '19 at 9:00
  • $\begingroup$ edited to incorporate the $10000 \times N^1$. @NeilSlater how do you get 9.6 billion? $\endgroup$ – bjschoenfeld Aug 14 '19 at 19:23
  • $\begingroup$ 3 channels times 10,000 pixels input times 32 channels times 10,000 "feature pixels" in first layer. $\endgroup$ – Neil Slater Aug 14 '19 at 19:55
  • 1
    $\begingroup$ I see. You are saying that while a CNN layer would have 288 (or really 864) weights, while a fully connected layer would have 9.6 billion. A good point! $\endgroup$ – bjschoenfeld Aug 14 '19 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.