Questions tagged [gpt-3]
Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive large language model that uses deep learning to produce human-like text. Introduced in May 2020, by OpenAI, with 175 billion parameters.
31
questions
0
votes
0
answers
11
views
Update OpenAI embedding based on own domain corpus
I have a large domain corpus. Is there anyway to update the word/sentence/document embedding obtained from OpenAI embedding API based on my own domain corpus? There may be some words in my corpus that ...
- 101
0
votes
0
answers
199
views
Extract multiple records from raw text using open AI API function calls
Trying to extract records from pdf. The plan was to convert pdf data to raw text and use open AI api calls to extract the data in a desired format (csv). This is because the pdf data is very ...
0
votes
0
answers
180
views
How is OpenAI embeddings obtained
There is OpenAI embedding API https://platform.openai.com/docs/guides/embeddings. How is this embedding related to the GPT3.5 transformer model architecture? Is it the vectors learned from the input ...
- 101
4
votes
1
answer
2k
views
What's the difference between GPT3.5 and InstructGPT?
I read about the different model series in GPT3.5 here - https://platform.openai.com/docs/models/gpt-3-5
At the beginning of the page, it mentions to look at https://platform.openai.com/docs/model-...
- 41
0
votes
0
answers
82
views
As a user, can I fine-tune end-to-end with GPT-3 if I want to add custom layers?
I am looking at the OpenAI API, and it seems that the only way to get embeddings for a specific text is to use the embedding API. In other words, I can only use the embeddings as fixed features in my ...
- 111
1
vote
1
answer
111
views
Repainting a picture in the style of some painter (or of another picture)
It sounds like a straight-forward task for DALL-E (and GPT?) to present a painting and ask to repaint it "in the style of Leonardo da Vinci". Like one can present texts and ask to rewrite ...
8
votes
2
answers
898
views
Is GPT-4 based on GPT-3 or was it trained from the scratch?
To me it looks like GPT-4 is based on GPT-3.
On the other hand, there were rumors that training of GPT-3 was done with errors, but re-train was impossible due to the costs.
- 321
1
vote
0
answers
2k
views
Tool for detecting AI Generated code?
Can you recommend any tools for detecting AI generated code? I am aware of AI-generated text detectors such as ZeroGPT and OpenAI-Classifier, but I am specifically interested in detecting AI-generated ...
- 111
1
vote
0
answers
55
views
How much do we know about the architectures of the Codex (prototype) models?
The transformer model Codex by OpenAI was introduced in a 2021 paper. The paper does not give complete information about the architecture. Below I've quoted all the passages in the paper that give ...
- 252
0
votes
1
answer
315
views
How is InstructGPT a fine-tuned version of GPT-3 and at the same time has fewer parameters than the original GPT3?
I am reading the paper "Training language models to follow instructions with human feedback"
It says:
Our labelers provide demonstrations of the desired behavior on the input prompt ...
1
vote
0
answers
45
views
Computation required for GPT model to choose likely word from n-options where n < total vocabulary size
Let’s imagine two different use cases for a LLM/GPT-3.
Predicting the next most likely word in a sequence using all ~50k words in its dictionary (i.e. the standard method of prompting a LLM)
Checking ...
- 11
9
votes
2
answers
3k
views
Are GPT-3.5 series models based on GPT-3?
In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained:
We trained this model using Reinforcement Learning from Human Feedback
(RLHF), ...
- 193
0
votes
0
answers
120
views
Could GPT3 be integrated with GOFAI programming to improve its ability to acquire new knowledge?
In my limited experience with AI chatbots (Replika, Sophie, etc.) they immediately fail the Turing test if I try to teach them some simple fact, or ask them to recall something that I said a few ...
- 1
4
votes
0
answers
3k
views
How is ChatGPT maintaining context?
It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT ...
- 41
0
votes
0
answers
173
views
Does ChatGPT's limited understanding of novel inputs reveal limitations in language's ability to convey meaning?
As a language model, ChatGPT generate responses based on patterns they have learned from the data they were trained on. However, it may not always provide a perfect or complete answer to novel or ...
- 1,094
9
votes
1
answer
1k
views
What causes ChatGPT to generate responses that refer to itself as a bot or LM?
ChatGPT occasionally generates responses to prompts that refer to itself as a "bot" or "language model."
For instance, when given a certain input (the first paragraph of this ...
- 192
1
vote
1
answer
3k
views
Fine-tune GPT-Neo with prompt and completion?
I'm new to AI and machine learning.
To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
- 111
1
vote
0
answers
95
views
Privacy implications of storing and transmitting GPT-3 embeddings?
We are exploring implementing a feature where a user might enter "which product has optional all wheel drive" into a search input, which would be transformed to GPT-3 embeddings, and ...
- 11
2
votes
1
answer
980
views
If GPT-3 is trained on predicting the next token, how is it able to take commands?
From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the ...
- 31
1
vote
1
answer
823
views
Papers on Prompt Engineering
I am into AI in general and NLP in particular. Besides, I have a background in philosophy, and the new LLMs like GPT-3 seem to have exciting capabilities. I want to study prompt engineering (for ...
- 123
1
vote
0
answers
507
views
How can I train GPT-3 (or similar) to search for answers my text file
My company has large sets of standards that are quite dry to read through and find answers to questions. How can I have an AI perform the search?
The text files are too long to submit with every ...
- 11
-2
votes
1
answer
598
views
Are there any transcripts of GPT-3 arguing that it is not conscious?
There have been a lot of transcripts showing GPT-3 arguing that it is self-conscious. In response, it was pointed out that GPT-3 can argue anything and pretend to be anything, given appropriate ...
- 125
1
vote
0
answers
167
views
How do I preload a conversational AI assistant like GPT-3 with complex relational data to draw on?
I'm exploring options to build a virtual assistant type of product.
Creating good dialog is mostly solved with GPT-3 or even DialoGPT.
My main question is how do I add larger amounts of relational ...
- 111
1
vote
0
answers
199
views
Can you train GPT-J to use a specific list of words and prioritise them?
Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this?
Say you're using GPT-J to write a story, you might wish to mention ...
4
votes
1
answer
875
views
How to get GPT-3 to translate a specific word in a sentence?
I just gave GPT-3 the following prompt (in the playground, using text-davinci-001 with default settings):
...
- 151
1
vote
1
answer
228
views
Are there any other language models besides GPT-3 that can be used to create chatbots with a specific identity and environment?
I've recently been looking into language models but ran into a small question out of curiosity. For some language models like GPT-3, it is possible to generate dialogue, with the model basing its ...
- 113
31
votes
1
answer
33k
views
What is the "temperature" in the GPT models?
What does the temperature parameter mean when talking about the GPT models?
I know that a higher temperature value means more randomness, but I want to know how randomness is introduced.
Does ...
- 433
4
votes
3
answers
5k
views
How can GPT-3 be used for designing electronic circuits from text descriptions?
I was wondering if it is possible to use GPT-3 to translate text description of a circuit to any circuit design language program, which in turn can be used to make the circuit.
If it is possible, what ...
- 265
15
votes
1
answer
22k
views
What language is the GPT-3 engine written in?
I know that the API is python based, but what's the gpt-3 engine written in mostly? C? C++? I'm having some trouble finding this info.
- 275
2
votes
2
answers
131
views
How do transformers understand data and answer custom questions?
I recently heard of GPT-3 and I don't understand how the attention models and transformers encoders and decoders work. I heard that GPT-3 can make a website from a description and write perfectly ...
1
vote
0
answers
29
views
What is the meaning of "Our current objective weights every token equally and lacks a notion of what is most important to predict" in the GPT-3 paper?
On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function:
Our current objective weights every token equally and lacks a notion of what is most important to ...
- 111