How does transformer leverage GPU to be trained faster than RNNs?

I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multiple GPUs, and why does that accelerate its training?

How does a transformer leverage the GPU to be trained faster than RNNs?

I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multiple GPUs, and why does that accelerate its training?

How does transformer leverage GPU which trains faster than RNN? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multi-gpu and accelerates its training?

How does transformer leverage GPU to be trained faster than RNNs? 

I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multiple GPUs, and why does that accelerate its training?