1
$\begingroup$

Here is one row from my data:

H  7.042 5.781 5.399  -9.118   5.488  7.470

The first column is a categorical class. The rest of them are continuous numerical features.

I encoded the classes using one-hot-encoding labels and concatenated them with the numerical features list:

    1   0  1  7.042 5.781 5.399  -9.118   5.488  7.470

Then I used this list for training.

  1. Is this a valid technique?
  2. Am I achieving anything really useful here?

My supervisor says that I made a mistake by using labels as features. I am trying to understand what I did wrong. In my view, the mistake seems to be: when I am using labels as features, I must provide those labels as input whenever I want to use the trained model. Therefore, this trained model is practically useless. Am I correct?

$\endgroup$
2
  • $\begingroup$ when you say 'features' do you refer specifically to continuous values as opposed to one hot encoded ones? $\endgroup$ Feb 24, 2022 at 8:01
  • $\begingroup$ @EdoardoGuerriero, Yes. see the edit. $\endgroup$
    – user366312
    Feb 24, 2022 at 8:41

1 Answer 1

1
$\begingroup$

Your approach is totally valid (especially considering you only have a single categorical feature).

If you're training a neural network though make sure to standardize your numerical features in order to be in the range [0, 1].

Also, in general, and again only if you're training a neural network, you might keep in mind that there's also another valid concatenation approach, happening not at the input layer level but in a hidden layer. This is common in natural language processing, where the text features can be quite larger in number than the numerical ones. By processing separately numerical and text features we can ensure that both are mapped to a fixed size dense layer, so when the concatenation occurs, we give same importance to both types of features regardless of their respective amounts. Another advantage of this approach is that when using one hot encoding the resulting features are sparse, and by keeping them separated you can treat them accordingly (for example using a loss function for sparse features only on the one hot feature block).

$\endgroup$
1
  • $\begingroup$ OP's post on stats.SE clarifies in a comment that the intention is to use the label to predict the label. stats.stackexchange.com/questions/565623/… In other words, the proposed procedure is entirely tautological. I think this answer assumes the typical case, where a person encodes categorical features and uses them to predict the class label. $\endgroup$
    – Sycorax
    Oct 8 at 19:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .