A simple typo can split a single token for a common word into several tokens, not only making the prompt longer, but also creating a combination of tokens that was rare in the training set. I wonder how big the effect of typos in the prompt is.

It would also be interesting to know whether using more common synonyms for words improves the understanding of the prompt, or whether replacing words with synonyms of identical meaning that are common in other types of text has a strong influence on the type of response, or whether the model simply associates the synonym with the other word and behaves the same way.

Finally, I wonder if a simple spell checker based on tokenization would be a significant way to improve the response. Is there notable research on these topics?



You must log in to answer this question.