LLMs are understood to generate non-deterministic outputs.
Are there LLMs out there that are capable to producing deterministic outputs for any given input given fixed parameters (like e.g temperature)?
I heard that llama.cpp - if run on a CPU instead of a GPU appears to generate deterministic outputs.
Any LLM that exists could easily be modified to be deterministic. At the present, they sample from a probability distribution for the next word. It is a trivial change to make, so that instead of sampling from this distribution, they always pick the word deemed most likely.
This has nothing to do with running on a GPU vs CPU: the non-determinism is not a function of hardware but software.
In many LLMs, this tradeoff is governed by a temperature parameter (it is called temperature for historical reasons) that parameterizes how much randomness will be in the LLM.
Reference: LLM Text Generation: Why is Determinism so Hard to Achieve?
To answer your question "No there arent any".
Executing an LLM such as llama.cpp on a CPU may indeed lead to more predictable outcomes due to the way it handles floating-point calculations. To see the difference for yourself, simply obtain a copy of llama.cpp and adjust the temperature and seed parameters as needed, or apply both if relevant, and conduct your tests. The same experiment can be replicated with a model running on a GPU to compare results.
LLMs are mathematical functions executed on floating-point calculations. The software and hardware implementations cause non-deterministic side-effects.
In most cases, these non-deterministic side-effects are desirable to produce outcomes that converge towards better overall performance.
That being said removing the non-deterministic behavior is some cases can be incredibly difficult since this behavior may be buried in some framework. For example, weight initialization is performed by selecting random small values for regularization in neural network implementations. This is a desirable effect to prevent vanishing or exploding gradients, but that is not to say that those random values couldn't be explicitly reused in some use case if it was absolutely necessary.
When people talk about using CPU, vs GPU, it typically references computational speed increases but GPUs include a myriad of computational optimizations and some GPU optimizations for picking random numbers do exist and may be used. There are also floating point anomalies in some computation environments but that should not be attributed towards being "non-deterministic".
