Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [gpt-3]

Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive large language model that uses deep learning to produce human-like text. Introduced in May 2020, by OpenAI, with 175 billion parameters.

Filter by
Sorted by
Tagged with
1 vote
1 answer
408 views

What is accelerated years in describing the amount of the training time?

As described in this article, it was written that GPT-3 took 405 V100 years to train in 2020. I'm a bit confused about this definition, does that mean the process was accelerated like using a V100 GPU ...
Marziye Hasanshahi's user avatar
0 votes
0 answers
15 views

Update OpenAI embedding based on own domain corpus

I have a large domain corpus. Is there anyway to update the word/sentence/document embedding obtained from OpenAI embedding API based on my own domain corpus? There may be some words in my corpus that ...
Salty Gold Fish's user avatar
0 votes
0 answers
274 views

Extract multiple records from raw text using open AI API function calls

Trying to extract records from pdf. The plan was to convert pdf data to raw text and use open AI api calls to extract the data in a desired format (csv). This is because the pdf data is very ...
viraj jadhav's user avatar
1 vote
0 answers
278 views

How is OpenAI embeddings obtained

There is OpenAI embedding API https://platform.openai.com/docs/guides/embeddings. How is this embedding related to the GPT3.5 transformer model architecture? Is it the vectors learned from the input ...
Salty Gold Fish's user avatar
4 votes
1 answer
2k views

What's the difference between GPT3.5 and InstructGPT?

I read about the different model series in GPT3.5 here - https://platform.openai.com/docs/models/gpt-3-5 At the beginning of the page, it mentions to look at https://platform.openai.com/docs/model-...
Arya's user avatar
  • 41
1 vote
1 answer
164 views

Repainting a picture in the style of some painter (or of another picture)

It sounds like a straight-forward task for DALL-E (and GPT?) to present a painting and ask to repaint it "in the style of Leonardo da Vinci". Like one can present texts and ask to rewrite ...
Hans-Peter Stricker's user avatar
8 votes
2 answers
1k views

Is GPT-4 based on GPT-3 or was it trained from the scratch?

To me it looks like GPT-4 is based on GPT-3. On the other hand, there were rumors that training of GPT-3 was done with errors, but re-train was impossible due to the costs.
Anixx's user avatar
  • 331
1 vote
0 answers
2k views

Tool for detecting AI Generated code?

Can you recommend any tools for detecting AI generated code? I am aware of AI-generated text detectors such as ZeroGPT and OpenAI-Classifier, but I am specifically interested in detecting AI-generated ...
HeavenHM's user avatar
  • 111
1 vote
0 answers
69 views

How much do we know about the architectures of the Codex (prototype) models?

The transformer model Codex by OpenAI was introduced in a 2021 paper. The paper does not give complete information about the architecture. Below I've quoted all the passages in the paper that give ...
Jack M's user avatar
  • 262
0 votes
1 answer
343 views

How is InstructGPT a fine-tuned version of GPT-3 and at the same time has fewer parameters than the original GPT3?

I am reading the paper "Training language models to follow instructions with human feedback" It says: Our labelers provide demonstrations of the desired behavior on the input prompt ...
DanielTheRocketMan's user avatar
1 vote
0 answers
48 views

Computation required for GPT model to choose likely word from n-options where n < total vocabulary size

Let’s imagine two different use cases for a LLM/GPT-3. Predicting the next most likely word in a sequence using all ~50k words in its dictionary (i.e. the standard method of prompting a LLM) Checking ...
Derek's user avatar
  • 11
10 votes
2 answers
3k views

Are GPT-3.5 series models based on GPT-3?

In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained: We trained this model using Reinforcement Learning from Human Feedback (RLHF), ...
iMad's user avatar
  • 203
0 votes
0 answers
128 views

Could GPT3 be integrated with GOFAI programming to improve its ability to acquire new knowledge?

In my limited experience with AI chatbots (Replika, Sophie, etc.) they immediately fail the Turing test if I try to teach them some simple fact, or ask them to recall something that I said a few ...
Gaylen's user avatar
  • 1
4 votes
0 answers
3k views

How is ChatGPT maintaining context?

It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT ...
Kay999's user avatar
  • 41
0 votes
0 answers
193 views

Does ChatGPT's limited understanding of novel inputs reveal limitations in language's ability to convey meaning?

As a language model, ChatGPT generate responses based on patterns they have learned from the data they were trained on. However, it may not always provide a perfect or complete answer to novel or ...
Faizy's user avatar
  • 1,104
9 votes
1 answer
1k views

What causes ChatGPT to generate responses that refer to itself as a bot or LM?

ChatGPT occasionally generates responses to prompts that refer to itself as a "bot" or "language model." For instance, when given a certain input (the first paragraph of this ...
Obie 2.0's user avatar
  • 192
1 vote
1 answer
3k views

Fine-tune GPT-Neo with prompt and completion?

I'm new to AI and machine learning. To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
SoftTimur's user avatar
  • 111
1 vote
0 answers
102 views

Privacy implications of storing and transmitting GPT-3 embeddings?

We are exploring implementing a feature where a user might enter "which product has optional all wheel drive" into a search input, which would be transformed to GPT-3 embeddings, and ...
jahghAAB's user avatar
2 votes
1 answer
1k views

If GPT-3 is trained on predicting the next token, how is it able to take commands?

From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the ...
Andrew Tang's user avatar
1 vote
1 answer
1k views

Papers on Prompt Engineering

I am into AI in general and NLP in particular. Besides, I have a background in philosophy, and the new LLMs like GPT-3 seem to have exciting capabilities. I want to study prompt engineering (for ...
Mehdi Abbassi's user avatar
1 vote
0 answers
520 views

How can I train GPT-3 (or similar) to search for answers my text file

My company has large sets of standards that are quite dry to read through and find answers to questions. How can I have an AI perform the search? The text files are too long to submit with every ...
Jon 310's user avatar
  • 11
-2 votes
1 answer
607 views

Are there any transcripts of GPT-3 arguing that it is not conscious?

There have been a lot of transcripts showing GPT-3 arguing that it is self-conscious. In response, it was pointed out that GPT-3 can argue anything and pretend to be anything, given appropriate ...
michau's user avatar
  • 125
1 vote
0 answers
196 views

How do I preload a conversational AI assistant like GPT-3 with complex relational data to draw on?

I'm exploring options to build a virtual assistant type of product. Creating good dialog is mostly solved with GPT-3 or even DialoGPT. My main question is how do I add larger amounts of relational ...
Sam7's user avatar
  • 111
1 vote
0 answers
200 views

Can you train GPT-J to use a specific list of words and prioritise them?

Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this? Say you're using GPT-J to write a story, you might wish to mention ...
Learningallday's user avatar
4 votes
1 answer
904 views

How to get GPT-3 to translate a specific word in a sentence?

I just gave GPT-3 the following prompt (in the playground, using text-davinci-001 with default settings): ...
Jonas Sourlier's user avatar
1 vote
1 answer
231 views

Are there any other language models besides GPT-3 that can be used to create chatbots with a specific identity and environment?

I've recently been looking into language models but ran into a small question out of curiosity. For some language models like GPT-3, it is possible to generate dialogue, with the model basing its ...
Jelle's user avatar
  • 113
36 votes
1 answer
41k views

What is the "temperature" in the GPT models?

What does the temperature parameter mean when talking about the GPT models? I know that a higher temperature value means more randomness, but I want to know how randomness is introduced. Does ...
Tom Dörr's user avatar
  • 483
4 votes
3 answers
6k views

How can GPT-3 be used for designing electronic circuits from text descriptions?

I was wondering if it is possible to use GPT-3 to translate text description of a circuit to any circuit design language program, which in turn can be used to make the circuit. If it is possible, what ...
Aether's user avatar
  • 265
16 votes
1 answer
23k views

What language is the GPT-3 engine written in?

I know that the API is python based, but what's the gpt-3 engine written in mostly? C? C++? I'm having some trouble finding this info.
Otherness's user avatar
  • 285
2 votes
2 answers
134 views

How do transformers understand data and answer custom questions?

I recently heard of GPT-3 and I don't understand how the attention models and transformers encoders and decoders work. I heard that GPT-3 can make a website from a description and write perfectly ...
DragonflyRobotics's user avatar
1 vote
0 answers
29 views

What is the meaning of "Our current objective weights every token equally and lacks a notion of what is most important to predict" in the GPT-3 paper?

On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function: Our current objective weights every token equally and lacks a notion of what is most important to ...
exteral's user avatar
  • 111