Questions tagged [open-ai]
For questions related to the openAI, including the Gym toolkit.
65
questions
-3
votes
0
answers
15
views
Can I export a fine-tuned open-ai model? [closed]
I would like to export it to test on some benchmarks and more!
-1
votes
0
answers
10
views
(How can) I test my openai fine-tuned model against standard Question Answering benchmarks? [closed]
I would like to automate the evaluation of my model but I do not know how to. Open to other ideas than benchmarks. Also I may want to remove human subjectivity as much as possible.
-1
votes
1
answer
208
views
what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]
I haven't been able to understand the output that OpenAI gym return for observation from this snippet
...
1
vote
1
answer
83
views
How to take gradient of log policy when actions are negative?
I am currently trying to train BipedalWalker of OpenAI gym by using policy gradient approach. My action space contains 4 continuous actions, all ranging [-1.0, 1.0]. In this case, how can we calculate ...
- 41
0
votes
0
answers
56
views
Can GPT-3 write an entire program if provided with detailed UML and ER Diagrams?
We already have tools like Denigma.app & Figstack.com (AI-powered code explanation in English), Codesee.io (Codebase and dependency mapping).
There's also a tool called Enzyme.so which converts ...
- 1
3
votes
1
answer
100
views
How to get GPT-3 to translate a specific word in a sentence?
I just gave GPT-3 the following prompt (in the playground, using text-davinci-001 with default settings):
...
- 133
1
vote
1
answer
45
views
Are there any other language models besides GPT-3 that can be used to create chatbots with a specific identity and environment?
I've recently been looking into language models but ran into a small question out of curiosity. For some language models like GPT-3, it is possible to generate dialogue, with the model basing its ...
- 113
2
votes
1
answer
154
views
Delayed state observation or caching action in OpenAI gym. Can it still learn?
I am planning to use OpenAI gym for my experiment in real life.
In my experiment design, by the limits of a real-life scenario, I can only receive the state information or the rewards about 2-3 ...
- 55
2
votes
1
answer
102
views
Why adding a baseline doesn't affect the policy gradient?
On the OpenAI's Spinning Up, they justify the fact that adding a baseline $b(s_t)$ in the policy gradient doesn't change its gradient by saying that this is
an immediate consequence of the EGLP Lemma
...
1
vote
0
answers
112
views
Too slow search using MCTS in OpenAI Atari games
I'm recently using Monte Carlo Tree Search in OpenAi Gym Atari, but the result isn't satisfying.
Without render, the game lasts about 180 steps ( env.step() was called this much time ) with random ...
- 11
1
vote
1
answer
76
views
Why does the Atari Gym Amidar environment only move after a certain number of episodes? [closed]
When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to ...
- 113
0
votes
0
answers
90
views
Open AI Taxi - Agent fails to learn an effective policy
I'm trying to solve the openai gym taxi problem (v3) using deep q learning. I've already had some success with the q-table approach, but for the life of me cannot manage to train a NN to learn a ...
- 103
0
votes
1
answer
66
views
Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning
I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0.
However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) ...
- 331
1
vote
0
answers
26
views
What is the meaning of "Our current objective weights every token equally and lacks a notion of what is most important to predict" in the GPT-3 paper?
On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function:
Our current objective weights every token equally and lacks a notion of what is most important to ...
- 111
1
vote
0
answers
73
views
How should I simulate this Markov Decision Process?
I am working on solving a problem on nodes in a graph communicating with each other. They try to estimate a central state using Kalman consensus filter, with the connections described by the graph's ...
- 11
0
votes
2
answers
742
views
Simple DQN too slow to train [closed]
I have been trying to solve the OpenAI lunar lander game with a DQN taken from this paper
https://arxiv.org/pdf/2006.04938v2.pdf
The issue is that it takes 12 hours to train 50 episodes so something ...
- 101
1
vote
0
answers
155
views
How to deal with KerasRL DDPG algorithm getting stuck in a local optima?
I am using KerasRL DDPG to try to learn a policy on my own custom environment, but the agent is stuck in a local optima although I am adding the OrnsteinUhlenbeck randomization process. I used the ...
- 36
1
vote
1
answer
461
views
How do I create a custom gym environment based on an image?
I am trying to create my own gym environment for the A3C algorithm (one implementation is here). The custom environment is a simple login form for any site. I want to create an environment from an ...
- 21
1
vote
0
answers
33
views
What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?
I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
- 11
5
votes
2
answers
553
views
My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem
I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
- 331
2
votes
0
answers
100
views
Most of state-action pairs remain unvisited in the q-table
In building my first Q-learning algorithm for OpenAI gym's CartPole problem, many of my states remain unvisited. I believe it is the reason that my agent does not learn.
Can I be told of the reasons I ...
- 331
1
vote
0
answers
113
views
OpenAI gym's CartPole problem system does not learn
My OpenAI CartPole-v0 problem's implementation using basic Q-learning does not learn at all.
I am a beginner and have implemented my first ever Q-learning from scratch after learning from tutorials.
...
- 331
9
votes
1
answer
4k
views
What exactly are the "parameters" in GPT-3's 175 billion parameters and how are they chosen/generated?
When I studied neural networks, parameters were learning rate, batch size etc. But even GPT3's ArXiv paper does not mention anything about what exactly the parameters are, but gives a small hint that ...
- 311
4
votes
1
answer
278
views
Why is GPT-3 such a game changer?
I've been hearing a lot about GPT-3 by OpenAI, and that it's a simple to use API with text in text out and has a big neural network off 175B parameters.
But how did they achieve this huge number of ...
- 41
0
votes
1
answer
463
views
How can I change observation states' values in OpenAI gym's cartpole environment?
I am learning with the OpenAI gym's cart pole environment.
I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$
-\...
- 331
1
vote
1
answer
358
views
How do I test an LSTM-based reinforcement learning model using any Atari games in OpenAI gym?
I am writing a couple of different reinforcement learning models based on Rainbow DQN or some PG models. All of them internally use an LSTM network because my project is using time series data.
I ...
- 31
1
vote
1
answer
102
views
Is there a general file type associated with AI projects?
This is a general question.
Is there a general file type associated with AI projects?
Photoshop = .psd
Excel = csv
Artificial Intelligence = ?
- 11
0
votes
1
answer
56
views
How to add more than 1 agent in one generation with Q Learning
Sometimes the agent learns a bit slow and you want to have multiple agents in one generation. And at each episode you'll draw on the screen only the best of them or all of them. How is that possible?
...
- 15
3
votes
1
answer
144
views
What should the action space for the card game Crib be?
I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if ...
- 33
1
vote
1
answer
206
views
Can we combine Off-Policy with On-Policy Algorithms?
On-Policy Algorithms like PPO directly maximize the performance objective or an approximation of it. They tend to be quite stable and reliable but are often sample inefficient. Off-Policy Algorithms ...
- 431
1
vote
0
answers
51
views
State-of-the-art algorithms not working on a custom RL environment
I'm trying to train a RL agent on a custom, highly stochastic environment (MDP). In order to do so I'm using existing implementations of state-of-the-art RL algorithms as provided by Stable Baselines. ...
- 129
1
vote
1
answer
1k
views
OpenAI Gym: Multiple actions in one step
I'm trying to design an OpenAI Gym environment in which multiple users/players perform actions over time. It's round based and each user needs to take an action before the round is evaluated and the ...
- 183
1
vote
0
answers
150
views
How do i start building an autoclick bot for pubg mobile?
I want to make a bot which clicks the fire button on the mobile screen upon seeing an enemies head.
In pubg mobile which is an android game you have to control the fire button and the aim along with ...
- 11
1
vote
1
answer
377
views
OpenAI spinning up convolutional networks with PPO [closed]
I am using pytorch version of PPO and I have image input that I need to process with convolutional neural networks, are there any examples on how to set up the network? I know that stable baselines ...
- 111
5
votes
1
answer
694
views
How to define an action space when an agent can take multiple sub-actions in a step?
I'm attempting to design an action space in OpenAI's gym and hitting the following roadblock. I've looked at this post which is closely related but subtly different.
The environment I'm writing needs ...
4
votes
1
answer
55
views
Is a neural network able to optimize itself for speed?
I am experimenting with OpenAI Gym and reinforcement learning. As far as I understood, the environment is waiting for the agent to make a decision, so it's a sequential operation like this:
...
- 201
2
votes
2
answers
322
views
Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful
I have written code in OpenAI's gym to simulate a random playing in Montezuma's Revenge where the agent randomly samples actions from the action space and tries to play the game. A success for me is ...
- 91
5
votes
1
answer
1k
views
How powerful is OpenAI's Gym and Universe in board games area? [closed]
I'm a big fan of computer board games and would like to make Python chess/go/shogi/mancala programs. Having heard of reinforcement learning, I decided to look at OpenAI Gym.
But first of all, I would ...
- 63
0
votes
1
answer
55
views
Why is this deep Q agent constantly learning just one action?
I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doing wrong? Here is the Github repository with ...
- 99
1
vote
1
answer
529
views
Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines?
I would like to use OpenAI Gym to solve a continuing environment, that is, a problem with a single, never-ending episode (please note I don't mean a continuous environment with continuous state and ...
- 177
1
vote
0
answers
56
views
Same implementation, but agent is not learning in Retro Pong Environment
I tried to implement the exact same python coding by Andrej Karpathy to train RL agent to play Pong, except that I migrated the environment from Gym to Retro.
Everything is the same except the action ...
- 21
3
votes
1
answer
447
views
Can OpenAI simulations be used in real world applications?
I know that classical control systems have been used to solve the problem of the inverted pendulum - inverted pendulum.
But I've seen that people have also used machine learning techniques to solve ...
0
votes
1
answer
70
views
Do algorithms like OpenAI's "think up strategies"?
I was discussing with a friend whether current AI does anything remotely similar to 'thinking' and he argued that AIs that play games must think up strategies.
While thinking may not be precisely ...
- 146
1
vote
0
answers
27
views
How am I getting same results 30 times faster than in original HER paper?
I am reproducing the results from Hindsight Experience Replay by Andrychowicz et. al. In the original paper they present the results below, where the agent is trained for 200 epochs.
200 epochs * 800 ...
- 243
3
votes
3
answers
3k
views
Has anyone been able to solve OpenAI's hardcore bipedal walker with their implementation of DDPG?
As the question suggests, I'm trying to see if I can solve OpenAI's hardcore version of their gym's bipedal walker using OpenAI's DDPG algorithm.
Below is a performance graph from my latest attempt, ...
- 143
2
votes
0
answers
93
views
What is a high performing network architecture to use in a PPO2 MlpLnLstmPolicy RL model?
I am playing around with creating custom architectures in stable-baselines. Specifically I am training an agent using a PPO2 model.
My question is, are there some rules of thumb or best practices in ...
- 185
1
vote
0
answers
146
views
Understanding policy update in PPO2
I have a question regarding the functionality of the PPO2 algorithm together with the Stable Baselines implementation:
From the original paper I know that the policy parameters $\theta$ are updated K-...
- 81
0
votes
1
answer
1k
views
OpenAI Spinning Up: Breakout-v0 example [closed]
I haven't been able to find any assistance / examples which could help me implement OpenAI's Spinning Up resource to solve Atari's Breakout-v0 game in the OpenAI gym.
I simply want to know why the ...
- 143
1
vote
1
answer
517
views
OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)
I'm about to create an OpenAI Gym environment for a flight simulator. I'm wondering, how to cope with the fact, that the result and reward for some action needs a considerable time to advance through ...
- 151
1
vote
1
answer
602
views
Advice on creating a new environment using OpenAI Gym
I'm looking for some general advice here before I dive in.
I'm interested in creating a new environment for OpenAI gym to provide some slightly more challenging continuous control problems than the ...
- 121