4
$\begingroup$

Let's say you have an input which can take one of 10 different unique values. How would you encode it?

  1. Have input length 10 and one-hot encode it.

  2. Have 1 input but normalise the value between the input range.

Would the end result be the same?

$\endgroup$
3
$\begingroup$

As with many general questions about how to represent features, or best learn from them, the answer is "it depends".

  • If there is a natural sequence to the separate items, and that sequence is informative somehow for the prediction, then the classes may work best as a single feature which takes discrete values, scaled to the network. A good example might be predicting house prices where one of the features is property tax bands (in locations which have these such as UK Council Tax).

  • If the classes have no natural sequence to them relating to the problem, then one-hot-encoding is usually a better interpretation. An example of this might be predicting the price of a car based on the manufacturer.

In both cases, a neural network with enough layers and connections - and enough data - could resolve a more difficult representation, and make little effective difference between the representations in practice. However, if you know something useful about the feature, it is normal to choose the most "natural" representation based on that knowledge, and you will often see a small improvement by making the correct choice. In the first case, the neural network can benefit from requiring less parameters, and learn more efficiently.

If you are not sure, then choosing one-hot versus a single scaled input should be an experiment which you perform as part of your hyper parameter tuning, along with any other feature engineering that you are not sure of.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.