Questions tagged [natural-language-generation]

For questions about the generation of human-readable text from structured data. This is related to natural-language-processing (NLP), and statistical NLP models can be used in NLG too. However, NLG also applies to templating, report writing and scripted conversations.

Filter by
Sorted by
Tagged with
1 vote
1 answer
90 views

Why do current language models no longer generate to long or short texts?

One of the biggest strengths of ChatGPT is that it generates fitting text with respect to the input query. It usually stays on topic, anwers the question completely and especially does not start ...
1 vote
3 answers
69 views

Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?

Imagine we have some sort of "next token predictor," either with transformer architecture, LSTM, or just a HMM (though the terminology I use here will be less aligned to HMMs, I believe the ...
2 votes
2 answers
2k views

How does a LLM (transformer) pick words from its vocabulary?

I have a very rough understanding of the "attention/self attention" mechanism of transformer models and how this can be used to process a set of word vectors provided as an input/prompt to ...
3 votes
1 answer
89 views

How can I improve this toy Graph Neural Network Generative Language model [closed]

Background I'm an undergraduate student with research interests in a field of physics that has significant overlap with graph theory, and a functioning knowledge of how simple neural nets work and how ...
0 votes
1 answer
120 views

Sentence generation for limited vocabulary

I need to make a sentence generator for a limited set of vocabulary (about 600 words). The requirements are: It must use only the words that are on the list, and never go beyond that; It must produce ...
0 votes
0 answers
54 views

How to build a Generative AI based on insights

I have a huge amount of documents that I do some preprocessing for them to extract insights. For instance, thousands of companies' profiles are used to extract the top N industries that exist in Japan....
2 votes
2 answers
4k views

What temperature would you recommend for the chatgpt api?

I believe that it is recommended to have a tiny bit of temperature with GPT 3 even for noncreative tasks like 0.2 or something (I am not entirely sure why). Last I checked, and if I remember correctly,...
4 votes
0 answers
3k views

How is ChatGPT maintaining context?

It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT ...
4 votes
1 answer
144 views

What puts the "chat" in a system like ChatGPT?

So I read Wolfram's What Is ChatGPT Doing … and Why Does It Work? but it left one really big question in my mind. His summary [if it could be called that!] really emphasizes that the core model is ...
1 vote
0 answers
639 views

Why doesn't ChatGPT ask questions?

As far as I understand ChatGPT has been trained on a vast array of data, and it does understand questions; but it seems to never ask. Even if a person would ask clarifying questions (that I assume are ...
2 votes
1 answer
44 views

What are the most effective methods and tools for summarizing long-form content like articles, editorials, and discussion threads for an app?

With users expecting instantaneous information and no compromise on in-depth details, app developers are challenged to condense long-form content such as articles, editorials, and discussion threads ...
1 vote
0 answers
187 views

How do I preload a conversational AI assistant like GPT-3 with complex relational data to draw on?

I'm exploring options to build a virtual assistant type of product. Creating good dialog is mostly solved with GPT-3 or even DialoGPT. My main question is how do I add larger amounts of relational ...
1 vote
0 answers
200 views

Can you train GPT-J to use a specific list of words and prioritise them?

Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this? Say you're using GPT-J to write a story, you might wish to mention ...
0 votes
1 answer
553 views

BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word. I ...
9 votes
3 answers
5k views

Can AI write good jokes yet?

Just watched a recent WIRED video on virtual assistants' performance on telling jokes. They're composed by humans, but I'd like to know if AI has gotten good enough to write some.
3 votes
1 answer
2k views

How are certain machine learning models able to produce variable-length outputs given variable-length inputs?

Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable ...
2 votes
1 answer
2k views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
1 vote
2 answers
236 views

Making generated texts from "data-to-text" more variable

I am diving in data-to-text generation for long articles (> 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs,...
2 votes
1 answer
131 views

I have 5000 html files (structured text), how can I generate a new one that "resembles" those?

I don't know anything about ML or NLP, but I was asked by someone to create brand new statutes (written laws) that resemble the ones currently in effect in my country. I have already gathered the laws,...
1 vote
0 answers
173 views

What is the best way to generate German paraphrases?

What is the best method to generate German paraphrases? The state-of-the-art are seq2seq transformer models, like T5, but they only work for English sentences. I found the multilingual MT5 model, but ...
1 vote
0 answers
21 views

Can unsupervised models learn something from cat vocalizations?

I love cats, and over the years have noticed that they have recurrent patterns of vocalizations. For example, upon seeing a bird, a cat may start chittering, but the same cat would never chitter at ...
0 votes
0 answers
25 views

Is there any neural network model that can perform multiple NLP steps at once?

I realize most NLP algorithms have multiple steps. (e.g. OCR/speech rec > syntax > semantics > response logic > semantic output > natural language output) Is there any NN model that can ...
3 votes
1 answer
5k views

How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation?

How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation? My idea is to come up with a system that is empowering the user’s imagination to create ...
2 votes
1 answer
1k views

How can a system like Jarvis understand the commands and take actions?

I am looking to make an AI like Jarvis. A perfect real-life example of this type of system is the simple AI that Mark Zuckerberg has recently built. Here is a description of how his AI works. From ...
2 votes
1 answer
171 views

Are there any meaningful books entirely written by an artificial intelligence?

Are there any meaningful books entirely written by an artificial intelligence? I mean something with meaning, unlike random words or empty books. Something that can be charactersed as fiction ...