As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?

  • 2
    $\begingroup$ i dont think neuroscience is far enough to the point it can answer that.... beamsearch is an effective and memory constrained method of approximating the sequence that maximizes the joint probability $\endgroup$
    – mshlis
    Jun 2 '19 at 19:56
  • $\begingroup$ I think text generation uses for more different approaches than just this one algorithm. There is a lot of NLP happening which does not involve neural networks/deep learning approaches. $\endgroup$ Jun 3 '19 at 8:29
  • $\begingroup$ @OliverMason for example ... ? $\endgroup$ Jun 5 '19 at 17:35
  • $\begingroup$ Any work in text generation in the past 60 years. $\endgroup$ Jun 6 '19 at 8:17
  • $\begingroup$ @OliverMason would you mind being more specific? For example, any state of the art method which does not use beam search? Some concrete references/links will be appreciated. $\endgroup$ Jun 6 '19 at 21:38

I thought the answer might be no.

enter image description here

In this 2020 ICLR paper: The Curious Case of Neural Text Degeneration, researchers found that beam search text is less surprising compared to human natural language. And they proposed a nucleus sampling method which generates more human like text.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.