# Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic weighting component in the process.

However, in Transformers (self-attention based encoder-decoder type architecture to be specific) we get different outputs with the same prompts (assuming $$T > 0$$). This doesn't make sense to me because the set of weights are always static, so the probability distribution produced should be the same; this simple decoding should yield the same output.

However, in practice, we observe that it is not actually the case. Any reasons why?