As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ i dont think neuroscience is far enough to the point it can answer that.... beamsearch is an effective and memory constrained method of approximating the sequence that maximizes the joint probability $\endgroup$– mshlisJun 2, 2019 at 19:56
-
$\begingroup$ I think text generation uses for more different approaches than just this one algorithm. There is a lot of NLP happening which does not involve neural networks/deep learning approaches. $\endgroup$– Oliver MasonJun 3, 2019 at 8:29
-
$\begingroup$ @OliverMason for example ... ? $\endgroup$– Pablo MessinaJun 5, 2019 at 17:35
-
$\begingroup$ Any work in text generation in the past 60 years. $\endgroup$– Oliver MasonJun 6, 2019 at 8:17
-
$\begingroup$ @OliverMason would you mind being more specific? For example, any state of the art method which does not use beam search? Some concrete references/links will be appreciated. $\endgroup$– Pablo MessinaJun 6, 2019 at 21:38
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I thought the answer might be no.
In this 2020 ICLR paper: The Curious Case of Neural Text Degeneration, researchers found that beam search text is less surprising compared to human natural language. And they proposed a nucleus sampling method which generates more human like text.
answered Feb 19, 2020 at 12:34