For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic weighting component in the process.
However, in Transformers (self-attention based encoder-decoder type architecture to be specific) we get different outputs with the same prompts (assuming $T > 0$). This doesn't make sense to me because the set of weights are always static, so the probability distribution produced should be the same; this simple decoding should yield the same output.
However, in practice, we observe that it is not actually the case. Any reasons why?