Questions tagged [natural-language-generation]

For questions about the generation of human-readable text from structured data. This is related to natural-language-processing (NLP), and statistical NLP models can be used in NLG too. However, NLG also applies to templating, report writing and scripted conversations.

Filter by
Sorted by
Tagged with
1 vote
3 answers
70 views

Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?

Imagine we have some sort of "next token predictor," either with transformer architecture, LSTM, or just a HMM (though the terminology I use here will be less aligned to HMMs, I believe the ...
BigMistake's user avatar
1 vote
1 answer
96 views

Why do current language models no longer generate to long or short texts?

One of the biggest strengths of ChatGPT is that it generates fitting text with respect to the input query. It usually stays on topic, anwers the question completely and especially does not start ...
Ricu's user avatar
  • 11
3 votes
1 answer
97 views

How can I improve this toy Graph Neural Network Generative Language model [closed]

Background I'm an undergraduate student with research interests in a field of physics that has significant overlap with graph theory, and a functioning knowledge of how simple neural nets work and how ...
MomentumEigenstate's user avatar
0 votes
1 answer
131 views

Sentence generation for limited vocabulary

I need to make a sentence generator for a limited set of vocabulary (about 600 words). The requirements are: It must use only the words that are on the list, and never go beyond that; It must produce ...
Slavus's user avatar
  • 103
0 votes
0 answers
54 views

How to build a Generative AI based on insights

I have a huge amount of documents that I do some preprocessing for them to extract insights. For instance, thousands of companies' profiles are used to extract the top N industries that exist in Japan....
Minions's user avatar
  • 123
4 votes
1 answer
145 views

What puts the "chat" in a system like ChatGPT?

So I read Wolfram's What Is ChatGPT Doing … and Why Does It Work? but it left one really big question in my mind. His summary [if it could be called that!] really emphasizes that the core model is ...
natevw's user avatar
  • 141
1 vote
0 answers
675 views

Why doesn't ChatGPT ask questions?

As far as I understand ChatGPT has been trained on a vast array of data, and it does understand questions; but it seems to never ask. Even if a person would ask clarifying questions (that I assume are ...
EmmanuelMess's user avatar
2 votes
2 answers
2k views

How does a LLM (transformer) pick words from its vocabulary?

I have a very rough understanding of the "attention/self attention" mechanism of transformer models and how this can be used to process a set of word vectors provided as an input/prompt to ...
MLBeginner's user avatar
2 votes
2 answers
4k views

What temperature would you recommend for the chatgpt api?

I believe that it is recommended to have a tiny bit of temperature with GPT 3 even for noncreative tasks like 0.2 or something (I am not entirely sure why). Last I checked, and if I remember correctly,...
userrandrand's user avatar
2 votes
1 answer
44 views

What are the most effective methods and tools for summarizing long-form content like articles, editorials, and discussion threads for an app?

With users expecting instantaneous information and no compromise on in-depth details, app developers are challenged to condense long-form content such as articles, editorials, and discussion threads ...
Anshu Kumar Gupta's user avatar
4 votes
0 answers
3k views

How is ChatGPT maintaining context?

It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT ...
Kay999's user avatar
  • 41
1 vote
0 answers
194 views

How do I preload a conversational AI assistant like GPT-3 with complex relational data to draw on?

I'm exploring options to build a virtual assistant type of product. Creating good dialog is mostly solved with GPT-3 or even DialoGPT. My main question is how do I add larger amounts of relational ...
Sam7's user avatar
  • 111
1 vote
0 answers
200 views

Can you train GPT-J to use a specific list of words and prioritise them?

Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this? Say you're using GPT-J to write a story, you might wish to mention ...
Learningallday's user avatar
0 votes
1 answer
566 views

BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word. I ...
Mucida's user avatar
  • 101
2 votes
1 answer
2k views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
neel g's user avatar
  • 146
1 vote
0 answers
175 views

What is the best way to generate German paraphrases?

What is the best method to generate German paraphrases? The state-of-the-art are seq2seq transformer models, like T5, but they only work for English sentences. I found the multilingual MT5 model, but ...
lattenjoe's user avatar
1 vote
0 answers
21 views

Can unsupervised models learn something from cat vocalizations?

I love cats, and over the years have noticed that they have recurrent patterns of vocalizations. For example, upon seeing a bird, a cat may start chittering, but the same cat would never chitter at ...
Alex Stone's user avatar
2 votes
1 answer
134 views

I have 5000 html files (structured text), how can I generate a new one that "resembles" those?

I don't know anything about ML or NLP, but I was asked by someone to create brand new statutes (written laws) that resemble the ones currently in effect in my country. I have already gathered the laws,...
Biganon's user avatar
  • 121
3 votes
1 answer
2k views

How are certain machine learning models able to produce variable-length outputs given variable-length inputs?

Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable ...
Python Test's user avatar
0 votes
0 answers
25 views

Is there any neural network model that can perform multiple NLP steps at once?

I realize most NLP algorithms have multiple steps. (e.g. OCR/speech rec > syntax > semantics > response logic > semantic output > natural language output) Is there any NN model that can ...
stackypants's user avatar
1 vote
2 answers
242 views

Making generated texts from "data-to-text" more variable

I am diving in data-to-text generation for long articles (> 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs,...
Carol.Kar's user avatar
2 votes
1 answer
171 views

Are there any meaningful books entirely written by an artificial intelligence?

Are there any meaningful books entirely written by an artificial intelligence? I mean something with meaning, unlike random words or empty books. Something that can be charactersed as fiction ...
Joe Jobs's user avatar
  • 121
3 votes
1 answer
5k views

How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation?

How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation? My idea is to come up with a system that is empowering the user’s imagination to create ...
user108180's user avatar
9 votes
3 answers
5k views

Can AI write good jokes yet?

Just watched a recent WIRED video on virtual assistants' performance on telling jokes. They're composed by humans, but I'd like to know if AI has gotten good enough to write some.
user avatar
2 votes
1 answer
1k views

How can a system like Jarvis understand the commands and take actions?

I am looking to make an AI like Jarvis. A perfect real-life example of this type of system is the simple AI that Mark Zuckerberg has recently built. Here is a description of how his AI works. From ...
Noah Chalifour's user avatar