Questions tagged [text-generation]
The text-generation tag has no usage guidance.
20
questions
0
votes
0
answers
34
views
What is the best way to generate handwritten text documents?
I am new to generative models. I was wondering if it would be better to generate an image of a handwritten text document as a whole (which I don't know how exactly is done), or first generate ...
- 235
0
votes
0
answers
47
views
What's the best way to feed stories to a neural network?
I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
- 1
3
votes
2
answers
89
views
How can GPT-3 be used for designing electronic circuits from text descriptions?
I was wondering if it is possible to use GPT-3 to translate text description of a circuit to any circuit design language program, which in turn can be used to make the circuit.
If it is possible, what ...
- 235
0
votes
1
answer
28
views
Semantic-based evaluation of translations instead of BLEU
I have a text generation model and I want to evaluate its output by comparing it to a set of gold human-annotated references.
I went through machine-translation metrics and I found that BLEU is used ...
- 123
1
vote
0
answers
41
views
How to generate text descriptions from keywords?
I wonder how can I build a neural network which will generate text description from given tag/tags. Let's assume I have created such data structure:
...
1
vote
1
answer
43
views
What would be the state of the art image captioning deep learning model?
I saw a couple of architectures, like CNN-LSTM, with and without attention model, use of Glove vector, self-critical models, etc. I am overwhelmed looking at different notebooks and architectures, ...
- 13
2
votes
0
answers
194
views
T5 or BERT for sentence correction/generation task?
I have sentences with some grammatical errors , with no punctuations and digits written in words... something like below:
As you can observe, a proper noun , winston isnt highlighted with capital in ...
- 121
0
votes
1
answer
272
views
Are there any good alternatives to an LSTM language model for text generation?
I have a trained LSTM language model and want to use it to generate text. The standard approach for this seems to be:
Apply softmax function
Take a weighted random choice to determine the next word
...
1
vote
2
answers
96
views
Making generated texts from "data-to-text" more variable
I am diving in data-to-text generation for long articles (> 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs,...
- 63
0
votes
1
answer
108
views
Transformer Language Model generating meaningless text
I currently learning on Transformers, so check my understanding I tried implementing a small transformer-based language model and compare it to RNN based language model. Here's the code for ...
- 141
3
votes
1
answer
48
views
Is there a complement to GPT/2/3 that can be trained using supervised learning methods?
This is a bit of a soft question, not sure if it's on topic, please let me know how I can improve it if it doesn't meet the criteria for the site.
GPT models are unsupervised in nature and are (from ...
- 131
1
vote
0
answers
27
views
Is the number of bidirectional LSTMs in seq2seq model equal to the maximum length of input text/characters?
I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/.
It ...
- 151
1
vote
0
answers
60
views
Importance sampling eq. 5 in paper "Residual Energy-based Models for Text Generation"
In the paper "Residual Energy-Based Models for Text Generation" (arXiv), on page 5, they write that equation 5 is an instance of importance sampling.
Equation 5 is:
$$ P(x_t \mid x_{<t}) = P_{LM}(...
- 41
1
vote
0
answers
110
views
How can I use GPT-2 to modify seed text of one form into a different form (LENGTH INVARIANT) whilst retaining meaning?
I am currently starting a research project whereby I am trying to convert text of one form into another.
i.e. If I were to write a seed sentance of the form "Scientists have finally achieved the ...
- 111
2
votes
0
answers
41
views
How should I design a reward function for a NLP problem where two models interoperate?
I would like to design a reward function. I am training two models from the first model that classify set of texts (paragraphs and keywords) and I also got some hidden states. The second model is ...
- 21
2
votes
0
answers
156
views
Pretrained Models for Keyword-Based Text Generation
I'm looking for an implementation that allows me to generate text based on a pre-trained model (e.g. GPT-2).
An example would be gpt-2-keyword-generation (click here for demo). As the author notes, ...
- 129
2
votes
0
answers
23
views
How to tell if two hotel reviews addressing the same thing
I am playing with a large dataset of hotel reviews, which contains both positive and negative reviews (the reviews are labeled). I want to use this dataset to perform textual style transfer - given a ...
3
votes
2
answers
404
views
How to use LSTM to generate a paragraph
A LSTM model can be trained to generate text sequences by feeding the first word. After feeding the first word, the model will generate a sequence of words (a sentence). Feed the first word to get the ...
- 1,183
8
votes
1
answer
6k
views
How do I use GPT-2 to summarise text?
In section 3.6 of the OpenAI GPT-2 paper it mentions summarising text based relates to this, but the method is described in very high-level terms:
To induce summarization behavior we add the text <...
- 354
23
votes
3
answers
31k
views
Can BERT be used for sentence generating tasks?
I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words.
...
- 333