2
$\begingroup$

As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?

$\endgroup$
5
  • 2
    $\begingroup$ i dont think neuroscience is far enough to the point it can answer that.... beamsearch is an effective and memory constrained method of approximating the sequence that maximizes the joint probability $\endgroup$
    – mshlis
    Commented Jun 2, 2019 at 19:56
  • $\begingroup$ I think text generation uses for more different approaches than just this one algorithm. There is a lot of NLP happening which does not involve neural networks/deep learning approaches. $\endgroup$ Commented Jun 3, 2019 at 8:29
  • $\begingroup$ @OliverMason for example ... ? $\endgroup$ Commented Jun 5, 2019 at 17:35
  • $\begingroup$ Any work in text generation in the past 60 years. $\endgroup$ Commented Jun 6, 2019 at 8:17
  • $\begingroup$ @OliverMason would you mind being more specific? For example, any state of the art method which does not use beam search? Some concrete references/links will be appreciated. $\endgroup$ Commented Jun 6, 2019 at 21:38

1 Answer 1

3
$\begingroup$

I thought the answer might be no.

enter image description here

In this 2020 ICLR paper: The Curious Case of Neural Text Degeneration, researchers found that beam search text is less surprising compared to human natural language. And they proposed a nucleus sampling method which generates more human like text.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .