Do individual dimensions in vector space have meaning?

Question

Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction).

It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. For example "king + woman - man = queen".

These terms are all N-dimensional vectors. Now, suppose, for simplicity, that $N=3$, $\text{king} = [0, 1, 2], \text{woman} = [1, 1, 0], \text{man} = [2, 2, 2], \text{queen} = [-1, 0, 0]$, then the expression above can be written as $[0, 1, 2] + [1, 1, 0] - [2, 2, 2] = [-1, 0, 0]$.

In this (contrived) example, the last dimension (king/man=2, queen/woman=0) suggests a semantic concept of gender. Aside from semantics, a given dimension could "mean" a part of speech, first letter, or really any feature or set of features that the algorithm might have latched onto. However, any perceived "meaning" of a single dimension might well just be a simple coincidence.

If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? Or is this purely a "random" artefact of the algorithm, with only the full N-dimensional vector distances mattering?

Answer 1

Do individual dimensions in vector space have meaning?

IIRC, some dimensions are interpretable, but in general this is not the case. Also it is debatable as to wether it is actually learning the actual representation or just an approximation of it. But in any case its not very reliable outside from some edge cases.

If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information?

Yes, but as to what that information entails in terms of "meaning" is lesser clear. You could say that if in a certain dimension the distance between two vectors is 0, then you have am estimate of the real distance that is better than guessing.