Timeline for How can a LLM optimise on it's own processing speed
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 20, 2023 at 5:36 | history | edited | Fabian Zeindl | CC BY-SA 4.0 |
edited title
|
| Dec 17, 2023 at 6:33 | comment | added | Fabian Zeindl | Yes, that would be limiting the output. But is the time needed for an LLMs to answer really linearly depending on the number of inputs tokens and output tokens. Or are there some loops. Kind of how humans are able to reason, arrive at a conclusion, see that the have an edge case and spend more time thinking about the edgecase? My request would be for the LLMs to output "there's an edge case I would need to analyze deeper if I had more time." | |
| Dec 16, 2023 at 23:11 | comment | added | Alberto | if you add "considering that on average every word you process takes X ms" it will probably already work, because you are pretty much just telling it to produce responses of length < 500/X | |
| Dec 16, 2023 at 11:47 | answer | added | Ari Setiawan | timeline score: 1 | |
| S Dec 16, 2023 at 7:15 | review | First questions | |||
| Dec 19, 2023 at 20:19 | |||||
| S Dec 16, 2023 at 7:15 | history | asked | Fabian Zeindl | CC BY-SA 4.0 |