6
$\begingroup$

In section 3.6 of the OpenAI GPT-2 paper it mentions summarising text based relates to this, but the method is described in very high-level terms:

To induce summarization behavior we add the text TL;DR: after the article and generate 100 tokens with Top-k random sampling (Fan et al., 2018) with k=2 which reduces repetition and encourages more abstractive summaries than greedy decoding. We use the first 3 generated sentences in these 100 tokens as the summary.

Given a corpus of text, in concrete code terms (python preferred), how would I go about generating a summary of it?

Improve this question
$\endgroup$
  • $\begingroup$ I think the referenced article explains clearly how to do it: aclweb.org/anthology/P18-1082 $\endgroup$ – OmG Mar 3 '19 at 13:36
  • 1
    $\begingroup$ Perhaps we should tag questions about GPT-2 as “AGI”. According to the latest information in the Internet, it's more than only a simple text-summary tool. The underlying technology, Generative Adversarial Networks, works with co-evolution which makes it hard to determine the limits. $\endgroup$ – Manuel Rodriguez Mar 3 '19 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.