# Is categorical encoding a type of word embedding?

Word embedding refers to the techniques in which a word is represented by a vector. There are also integer encoding and one-hot encoding, which I will collectively call categorical encoding.

I can see no fundamental difference between the categorical encoding and word embedding at a fundamental level. They may be different at an application level.

Is it true that categorical encoding is a type of word embedding? And are different names solely due to the task in which apply the technique?

A one-hot vector contains one element that is 1 and all other elements are 0. So, for example, the vector $$[0, 0, 1, 0]$$ is a one-hot vector, while the vector $$[0, 2, 0.2, 0]$$ is not. (Given that the sum of all elements in the one-hot vector is equal to 1 and all elements are in the range $$[0, 1]$$, a one-hot vector is a probability vector, although this may be irrelevant.) To represent your objects as one-hot vectors, you first need to know how many objects you have (in the context of NLP, this is often the vocabulary size). Let's say you have $$N$$ distinct elements (e.g. words), then, in order for two objects not to be mapped to the same one-hot vector, you need to have one-hot vectors of $$N$$ dimensions. So, if $$N$$ is very large, this can be a disadvantage of one-hot encoding, although, in principle, you only need to store the index of the $$1$$ for each one-hot vector.