3 SVD Based Methods

For this class of methods to find word embeddings (otherwise known as word vectors), we first loop over a massive data set and accumulate word co-occurrence counts in some form of a matrix X and then perform Singular Value Decomposition on X to get a USV^T decomposition. We then use the rows of U as the word embeddings for all words in our dictionary. Let us discuss a few choices of X.

Above is the excerpt from the standford univ cs224n lecture 1 notes.

Above USV refer to what? There's no prior explanation about it so I ask here.


USV^T refers to the result of the singular value decomposition (SVD).

An m times n matrix X can be written with the help of three matrices

X = USV^T,

where U is an m times m unitary matrix, S is a diagonal m times n matrix with real entries called singular values, and V is a unitary n times n matrix. The ^T is the Hermitean transpose. SVD has applications, e.g, in optimization problems, principal component analysis, etc. Wikipedia has quite a long article.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.