I read the answers for that question What kind of word embedding is used in the original transformer?. It says that transforms like bert start the first word embedding layer with random values.
Initializing the first word embedding layer in transformers with random values works fine but Wouldn't initializing transformers with pre-trained word embedding speed up the training of transformers?
Isn't starting with pre-trained word embedding(vectors that have semantic meaning) is better than starting from scratch?
I am not talking about the performance. I am talking about the speed of the training.