What will happen if I concatenate one-hot-encoded categorical features along with continuous numerical features?

Question

Here is one row from my data:

H  7.042 5.781 5.399  -9.118   5.488  7.470

The first column is a categorical class. The rest of them are continuous numerical features.

I encoded the classes using one-hot-encoding labels and concatenated them with the numerical features list:

    1   0  1  7.042 5.781 5.399  -9.118   5.488  7.470

Then I used this list for training.

1. Is this a valid technique?
2. Am I achieving anything really useful here?

My supervisor says that I made a mistake by using labels as features. I am trying to understand what I did wrong. In my view, the mistake seems to be: when I am using labels as features, I must provide those labels as input whenever I want to use the trained model. Therefore, this trained model is practically useless. Am I correct?

• Cross posted in stats.SE

Answer 1

Your approach is totally valid (especially considering you only have a single categorical feature).

If you're training a neural network though make sure to standardize your numerical features in order to be in the range [0, 1].

Also, in general, and again only if you're training a neural network, you might keep in mind that there's also another valid concatenation approach, happening not at the input layer level but in a hidden layer. This is common in natural language processing, where the text features can be quite larger in number than the numerical ones. By processing separately numerical and text features we can ensure that both are mapped to a fixed size dense layer, so when the concatenation occurs, we give same importance to both types of features regardless of their respective amounts. Another advantage of this approach is that when using one hot encoding the resulting features are sparse, and by keeping them separated you can treat them accordingly (for example using a loss function for sparse features only on the one hot feature block).