Questions tagged [open-ai]
For questions related to the openAI, including the Gym toolkit.
88
questions
5
votes
2
answers
276
views
Is GPT-4 based on GPT-3 or was it trained from the scratch?
To me it looks like GPT-4 is based on GPT-3.
On the other hand, there were rumors that training of GPT-3 was done with errors, but re-train was impossible due to the costs.
- 201
0
votes
0
answers
17
views
Using embedings to answer questions
i have simple code. text which is indexed in pinecone. when person asks quesiton i'm searching for relevant part from this index and then givin this part to open ai to answer question based on this ...
1
vote
1
answer
113
views
Creating a support chat bot for my business
I am trying to create a kind of Support Bot to answer my clients about specific technical details about WordPress plugins that I sell.
The goal is that the /completitions api would be feeded a prompt ...
0
votes
0
answers
69
views
When would you use prompting vs. fine tuning?
I would like to hear your thoughts on when is it appropriate to use prompting vs. fine-tuning.
Does one make more sense for specific tasks than the other.
Kindly elaborate.
4
votes
2
answers
401
views
Would self-hosting ChatGPT be feasible, w.r.t. computation costs?
Suppose the pre-trained, current date (2023-02-04) ChatGPT model was released open source, would it be feasible for regular users to interact with the model on a self-hosted computer?
Assumptions
I ...
- 223
4
votes
2
answers
762
views
Are GPT-3.5 series models based on GPT-3?
In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained:
We trained this model using Reinforcement Learning from Human Feedback
(RLHF), ...
- 143
0
votes
1
answer
558
views
How to assess if OpenAI's ChatGPT chatbot has a human in the loop? [closed]
I've asked a question and given a couple answers that propose the OpenAI ChatGPT chatbot has humans in the loop (HITL), and that explains the chatbot's extraordinary abilities.
I've been repeatedly ...
- 337
0
votes
0
answers
25
views
What is a good strategy for breaking up content into prompts and completions for OpenAI fine tuning?
I want to train a fine-tuned openai model to know more about specific Judo throws and training methodologies. I have a bunch of documents I have written on Judo throws that I would like to use for ...
- 101
5
votes
2
answers
337
views
InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used?
InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used?
$$ \operatorname{loss}(\theta) = -\frac{1}{\binom{K}{2}}E_{(x,y_w,y_l)\sim D}[\log(\sigma(r_{\theta}(x, y_w) - ...
- 161
2
votes
0
answers
2k
views
How is ChatGPT maintaining context?
It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT ...
- 21
0
votes
0
answers
895
views
What are the similarities and differences between ChatGPT and YouChat?
I have recently tried the both, and it looks to me that the both have similar capabilities.
YouChat says ChatGPT is more advanced.
YouChat has connection to the Internet, and according to it, ...
- 201
4
votes
1
answer
637
views
Why does ChatGPT create fake code?
ChatGPT has been a big thing lately. It also makes a lot of mistakes. For example, it creates fake functions of a package and tells it as it works for real. I was wondering how that works. Why is it ...
- 155
0
votes
0
answers
2k
views
Is there a way to determine if code has been autogenerated by an AI source?
Lately, our Junior developer has been producing code fixes very quickly. At first, I was excited for him as I thought he was starting to retain information and proving he was able to research and fix ...
-1
votes
1
answer
51
views
What is the Need for AI Developers since we have AI APIs Now?
I am new to AI and passionate about it. I have a fair knowledge of deep learning and I am into computer vision. I am eyeing NLP as well.
I just want to know the need for an AI developer in computer ...
- 101
0
votes
0
answers
158
views
Does ChatGPT's limited understanding of novel inputs reveal limitations in language's ability to convey meaning?
As a language model, ChatGPT generate responses based on patterns they have learned from the data they were trained on. However, it may not always provide a perfect or complete answer to novel or ...
- 1,004
2
votes
1
answer
3k
views
How do ChatGPT content filters work? (If not chatGPT then in general) [closed]
I first tried ChatGPT few days ago. And every day that goes by it seems more and more content filters are introduced. I can still make it do stuff if I "jailbreak it" but I feel like the ...
- 121
4
votes
2
answers
3k
views
How does ChatGPT respond to novel prompts and commands?
So I understand how a language model could scan a large data set like the internet and produce text that mimicked the statistical properties of the input data, eg completing a sentence like "eggs ...
- 143
7
votes
4
answers
43k
views
OpenAI ChatGPT gives a network error at long responds. How can I fix it? [closed]
When OpenAI's ChatGPT replies with a very long answer, it will return a network error. When you check the network console, the POST request will fail with a ...
- 187
0
votes
0
answers
50
views
Embedding layers/entities in openAI's Hide and seek paper
I've recently come across a youtube video about openAI's hide and seek paper (https://openai.com/blog/emergent-tool-use/) and got really fascinated about the paper itself. But as I digging in the ...
1
vote
1
answer
118
views
OpeanAI Gym. Train problem: invalid values [closed]
I have a problem with my reinforcement learning model.
I am trying to simulate an electric battery storage. To keep it as simple as possible, the efficiency of charge, storage and discharge are 100%. ...
- 13
0
votes
0
answers
206
views
Problem with OpenAI gym in VSCode
I downloaded OpenAI Gym in windows from the console, it did not give me any error, but when I run the file nothing is shown as can be seen in the terminal. So I don't know what to do or what to ...
- 1
0
votes
0
answers
21
views
How to utilize real-world data to improve performance of RL Agent?
I'm still a newbie in RL, so please forgive me if the answer is obvious or the question totally dumb.
I'm currently trying to train a robot RL agent in OpenAI gym with a custom environment. This alone ...
- 1
0
votes
0
answers
49
views
Dall-E Question (probably silly)
I have started to read the Dall-E (1) paper and I have a quick question that will help me a lot. I know the basis of Transformers but only for NLP tasks (Text-to-text).
So when i read : "The ...
0
votes
0
answers
198
views
Why DDPG losses don't decrease while the reward grows?
I've noticed that training a DDPG agent in the Reacher-v2 environment of OpenAI Gym, the losses of both actor and critic first decrease but after a while start increasing but the episode mean reward ...
- 41
1
vote
2
answers
155
views
RL solutions for OpenAI Gym environments?
Is there any place where people share their agent's settings for solving OpenAI Gym Environments?
For example, I'd like to know what are good parameters for a DDPG agent to learn the task in Reacher-...
- 41
-1
votes
1
answer
512
views
what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]
I haven't been able to understand the output that OpenAI gym return for observation from this snippet
...
1
vote
1
answer
114
views
How to take gradient of log policy when actions are negative?
I am currently trying to train BipedalWalker of OpenAI gym by using policy gradient approach. My action space contains 4 continuous actions, all ranging [-1.0, 1.0]. In this case, how can we calculate ...
- 51
1
vote
0
answers
1k
views
Can GPT-3 write an entire program if provided with detailed UML and ER Diagrams?
We already have tools like Denigma.app & Figstack.com (AI-powered code explanation in English), Codesee.io (Codebase and dependency mapping).
There's also a tool called Enzyme.so which converts ...
- 11
3
votes
1
answer
344
views
How to get GPT-3 to translate a specific word in a sentence?
I just gave GPT-3 the following prompt (in the playground, using text-davinci-001 with default settings):
...
- 133
1
vote
1
answer
182
views
Are there any other language models besides GPT-3 that can be used to create chatbots with a specific identity and environment?
I've recently been looking into language models but ran into a small question out of curiosity. For some language models like GPT-3, it is possible to generate dialogue, with the model basing its ...
- 113
2
votes
1
answer
327
views
Delayed state observation or caching action in OpenAI gym. Can it still learn?
I am planning to use OpenAI gym for my experiment in real life.
In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
- 55
3
votes
1
answer
226
views
Why adding a baseline doesn't affect the policy gradient?
On the OpenAI's Spinning Up, they justify the fact that adding a baseline $b(s_t)$ in the policy gradient doesn't change its gradient by saying that this is
an immediate consequence of the EGLP Lemma
...
1
vote
0
answers
244
views
Too slow search using MCTS in OpenAI Atari games
I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying.
Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
- 11
1
vote
1
answer
201
views
Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]
When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
- 113
0
votes
0
answers
156
views
Open AI Taxi - Agent fails to learn an effective policy
I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
- 103
0
votes
1
answer
119
views
Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning
I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0.
However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
- 371
1
vote
0
answers
27
views
What is the meaning of "Our current objective weights every token equally and lacks a notion of what is most important to predict" in the GPT-3 paper?
On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function:
Our current objective weights every token equally and lacks a notion of what is most important to ...
- 111
1
vote
0
answers
110
views
How should I simulate this Markov Decision Process?
I am working on solving a problem on nodes in a graph communicating with each other. They try to estimate a central state using Kalman consensus filter, with the connections described by the graph's ...
- 11
1
vote
2
answers
1k
views
Simple DQN too slow to train [closed]
I have been trying to solve the OpenAI lunar lander game with a DQN taken from this paper
https://arxiv.org/pdf/2006.04938v2.pdf
The issue is that it takes 12 hours to train 50 episodes so something ...
- 111
1
vote
0
answers
212
views
How to deal with KerasRL DDPG algorithm getting stuck in a local optima?
I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
- 36
1
vote
1
answer
685
views
How do I create a custom gym environment based on an image?
I am trying to create my own gym environment for the A3C algorithm (one implementation is here). The custom environment is a simple login form for any site. I want to create an environment from an ...
- 21
1
vote
0
answers
39
views
What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?
I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
- 11
6
votes
2
answers
852
views
My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem
I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
- 371
2
votes
0
answers
183
views
Most of state-action pairs remain unvisited in the q-table
In building my first Q-learning algorithm for OpenAI gym's CartPole problem, many of my states remain unvisited. I believe it is the reason that my agent does not learn.
Can I be told of the reasons I ...
- 371
1
vote
0
answers
179
views
OpenAI gym's CartPole problem system does not learn
My OpenAI CartPole-v0 problem's implementation using basic Q-learning does not learn at all.
I am a beginner and have implemented my first ever Q-learning from scratch after learning from tutorials.
...
- 371
23
votes
1
answer
11k
views
What exactly are the "parameters" in GPT-3's 175 billion parameters and how are they chosen/generated?
When I studied neural networks, parameters were learning rate, batch size etc. But even GPT3's ArXiv paper does not mention anything about what exactly the parameters are, but gives a small hint that ...
- 451
4
votes
1
answer
530
views
Why is GPT-3 such a game changer?
I've been hearing a lot about GPT-3 by OpenAI, and that it's a simple to use API with text in text out and has a big neural network off 175B parameters.
But how did they achieve this huge number of ...
- 41
0
votes
1
answer
636
views
How can I change observation states' values in OpenAI gym's cartpole environment?
I am learning with the OpenAI gym's cart pole environment.
I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$
-\...
- 371
1
vote
1
answer
481
views
How do I test an LSTM-based reinforcement learning model using any Atari games in OpenAI gym?
I am writing a couple of different reinforcement learning models based on Rainbow DQN or some PG models. All of them internally use an LSTM network because my project is using time series data.
I ...
- 31
1
vote
1
answer
162
views
Is there a general file type associated with AI projects?
This is a general question.
Is there a general file type associated with AI projects?
Photoshop = .psd
Excel = csv
Artificial Intelligence = ?
- 11