Skip to main content
edited title
Link

How can ana LLM learn and optimise on it's own processing speed

Source Link

How can an LLM learn and optimise on it's own processing speed

Is it possible to give a large-language-model access to the time it takes to give it's answers and ask it to optimize on that?

An example prompt would be:

From now on, please measure the time it takes for you to give me an answer and optimize that time so that it stays under 500 milliseconds. It is alright if you give me incorrect information because of this constraint.

We can assume that the human brain optimizes for processing speed to lower it's energy usage. How can this be applied to a LLM?