I am confused as to how neural networks consider the different features by that have access to at the input layer.

Consider this example: I have three features: an image, a dollar amount, and a rating. However, since the one feature is an image, I need to represent it with very high dimensionality, for example with 128x128=16,384 pixel values. (I am just using 'image' as an example, my question holds for any feature that needs high dimensional representation: word counts, one-hot encodings, etc.)

Will the 16,384 'features' representing the image completely overwhelm the other 2 features that are the dollar amount and rating? Ideally, I would think the network would consider each of the three true features relatively equally. Would this issue naturally resolve itself in the training process? Would training become much more difficult of a task?


As stated in your example dataset the three features are an image, price, rating. Now you want to build a model that uses all of these features and the simplest way to do is to feed them directly into the neural network but its inefficient and fundamentally flawed. Due to the following reasons :

  • In the first dense layer neural network will try to combine raw pixel value linearly with price and rating which will produce features that are meaning less for inference.
  • It could perform well just by optimize the cost function but the model performance will be nowhere as good as it could perform with a good architecture.

So the neural network doesn't care if the data is raw pixel value, price or rating it would just optimize it to produce the desired output. That is why it is necessary to design suitable architecture for the given problem.

Possible architecture for your given example :

  1. Seprate your raw features i.e pixel value and high level data i.e price and rating
  2. Stack 2-3 dense layer for raw features (to find complex pattern in images)
  3. Stack 1-2 dense layer for high level features
  4. Combine them together in a dense layer

If you want to de emphasize the importance of image just connect the first dense layer 16,384 to another layer having less connections say 1024 and have more connections from the high level data say 2048

  1. (raw features) -> Dense Layer(16,384) -> Dense Layer(1024)
  2. (high level features) -> Dense Layer(2048)
  3. combine 1 and 2 with another Dense Layer()
| improve this answer | |

The answer by ssh is correct. Your results could be further improved i) by extracting image features by a convolutional (instead of fully connected) architecture, and ii) by exploiting transfer learning.

To exploit transfer learning you i) pick some widely used model, eg. ResNet-18, ii) initialize it with ImageNet pretrained parameters, iii) replace its fully connected layer (the one that produces 1000-D softmax input) with your own randomly initialized fully connected layer. If you are interested, have a look at detailed instructions.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.