When it comes to normal layman terms "latent space" means it cannot be accessed, thus we have no direct control over it. We can only manipulate it indirectly, while "Embeddings" can be obtained directly. We can use deterministic operations or transformations to convert an object into its corresponding embedding space.
There is no marked difference between these 2 terms as far as Machine Learning is concerned. If we look at this famous paper on Variational Autoencoders, we can see the words has been used interchangeably.
More specifically, I would consider the word (in the context of Machine Learning only) latent as a more general term than Embedding. Embeddings will refer to a more specific object (in context of ML), for example the embedding of $word_1$ is $embedding_1$. Whereas, we can use the term latent to describe broader terms like latent space, latent representation, latent variables (latent variables of a word is same as an embedding of a word).
After digging some more I found some what of a formal definition of Latent Variables in Deep Learning by Goodfellow:
- Latent Variables - A latent variable is a random
variable that we cannot observe directly. The component identity variable $c$ of the
mixture model provides an example. Latent variables may be related to $x$ through
the joint distribution, in this case, $P(x, c) = P(x | c)P(c)$. The distribution $P(c)$
over the latent variable and the distribution $P(x | c)$ relating the latent variables
to the visible variables determines the shape of the distribution $P(x)$, even though
it is possible to describe $P(x)$ without reference to the latent variable.
Also a paper cited by Goodfellow while discussing embeddings has the following excerpt:
Following the success of user/item clustering or matrix factorization techniques in collaborative filtering to represent non-trivial similarities between the connectivity patterns of entities in single relational data, most existing methods for multi-relational data have been designed within the framework of relational learning from latent attributes, as pointed out by; that is, by learning and
operating on latent representations (or embeddings) of the constituents (entities and relationships).
So clearly these are somewhat interchangeable terms.
But my interpretation would be that embeddings are helpful more explicitly (more visible, latent variables are meant to be hidden), that is we can construct a new data-set from it and use various ML methods on it, whereas latent variables are something not useful explicitly (it is a part of a bigger problem we are trying to solve).
EDIT: In the context of HMM's the term better suitable is hidden state and not latent space. Thus, in a HMM (from Wiki) The adjective hidden refers to the state sequence through which the model passes, not to the parameters of the model; the model is still referred to as a hidden Markov model even if these parameters are known exactly.