Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
76
questions
0
votes
0answers
23 views
How to implement or avoid masking for transformer?
When it comes to using Transformers for image captioning is there any reason to use masking?
I currently have a resnet101 encoder and am trying to use the features as the input for a transformer model ...
1
vote
0answers
24 views
Pytorch Deep q network not learning and step not stepping towards target
I am trying to create a simple deep q network for rl with conv2d layers. I can’t figure out what I am doing wrong, and the only thing I can see that doesn’t seem right is when I get the model ...
0
votes
1answer
18 views
Why would the reward of A3C with LSTM suddenly drop off after many episodes?
I am training an A3C with stacked LSTM.
During initial training, my model was giving descent +ve reward. However, after many episodes, its reward just goes to zero and is continuing for a long time. ...
0
votes
0answers
12 views
How to use an image tensor for caption generation with Transformer-XL or BERT?
I am fairly new to transformers and deep learning in general so please be kind,
I am currently working on a project that will caption images using either Transformer-XL or BERT, however, I am not sure ...
1
vote
0answers
14 views
is it ok to take random actions while training a3c as in below code
i am trying to train an A3C algorithm but I am getting same output in the multinomial function.
can I train the A3C with random actions as in below code.
can someone expert comment.
...
0
votes
1answer
49 views
Feeding YOLOv4 image data into LSTM layer?
How would one extract the feature vector from a given input image using YOLOv4 and pass that data into an LSTM to generate captions for the image?
I am trying to make an image captioning software in ...
0
votes
0answers
17 views
Class activation maps for 3D Convolutional neural network?
I have implemented a 3D convolutional neural network and I was not able to find resources for interpretation for my model. I have found some techniques such as GradCam and GradCam++ but these generate ...
0
votes
0answers
22 views
Why does my loss value of autoencoder in PyTorch is negative?
I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension.
The idea is kind of simple, SDNE uses the ...
0
votes
1answer
22 views
Get Neural Network to predict a tag/class on a certain word using the surrounding words as context [PyTorch]?
I am somewhat a novice at the topic of Neural Netoworks and PyTorch.
I am trying to create a model that takes a word (that I have modified very slightly) and a 'window' of context around it and ...
0
votes
0answers
12 views
Automated Scoring (non-english language) Using BERT
i'm a student and i'm new to NLP.
I want to build an Automated Scoring system which is in Indonesian Language using BERT. The system is expected to be able to measure the similarity of an answer(e.g: ...
0
votes
0answers
14 views
Descent Training episodes for LSTM + TD3
I am building an AI with TD3 and lstm in both actor and critic.
By LSTM size is 5,5 with 3 layers and hidden layers with 400 and 300 neurons respectively.
I have states dimension of 5 with each value ...
0
votes
0answers
20 views
TD3+lstm predicting the same output for varying states
I have a model with TD3 + lstm in both actor and critic.
I am trying to make it learn to predict some specific actions based on the environment conditions.
However i see that the AI predicts very ...
0
votes
1answer
65 views
Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?
In the concept of the vanilla policy gradient algorithm, is it possible for our trajectory size to be fixed?
For example, my environment is the space of embedded images (using a pre-trained encoder to ...
1
vote
0answers
44 views
Is it good practice to save NLP Transformer based pre-trained models into file system in production environment
I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers.
I have saved the pretrained model into the file directory in dev environment. ...
1
vote
0answers
45 views
Classification or regression for deep Q learning
DQN implemented at https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py uses the mean square error loss function for the neural network ...
1
vote
1answer
57 views
Why is the policy loss the mean of $-Q(s, \mu(s))$ in the DDPG algorithm?
I am trying to implement the DDPG algorithm based on this paper.
The part that confuses me is the actor network's update.
I don't understand why the policy loss is simply the mean of $-Q(s, \mu(s))$, ...
0
votes
0answers
25 views
How do I get my DCGAN to generate a number of fake images?
I have a Deep Convolutional Generative Adversarial Network (DCGAN) that trains on the CIFAR dataset. When I finish the training (100k epochs), how can I make my network generate 1000 fake images? I ...
0
votes
1answer
53 views
Input tensor shape order for RNN (PyTorch)
I am confused as to why the sequence length is the first dimension of the input tensor for an RNN when batch size is the first dimension for any other kind of network (Linear/CNN/etc.).
This makes me ...
0
votes
1answer
28 views
How is it possible to get the output size of `n` Consecutive Convolutional layers? [closed]
Given network architecture, what are the possible ways to define fully connected layer fc1 to have a generalized structure such as ...
1
vote
0answers
40 views
In layman's terms, what is stochastic computation graph?
I'm going through the distributions package on PyTorch's documentation and came across the term stochastic computation graph. In layman's terms, what is it?
1
vote
0answers
44 views
Policy Gradient on Tic-Tac-Toe not working
I wanted to implement the Policy Gradient on Tic-Tac-Toe.
I tried to use the code that worked for any environment like CartPole-v0 to my Tic-Tac-To game. But it is not learning. There are no errors. ...
2
votes
1answer
263 views
Why isn't my implementation of A2C for the the atari pong game converging?
I have two different implementations with PyTorch of the Atari Pong game using A2C algorithm. Both implementations are similar, but some portion are different.
https://colab.research.google.com/drive/...
2
votes
2answers
173 views
Advantage computed the wrong way?
Here is the code written by Maxim Lapan. I am reading his book (Deep Reinforcement Learning Hands-on). I have seen a line in his code which is really weird. In the accumulation of the policy gradient $...
1
vote
0answers
40 views
Benchmarking SAC on Pybullet
So far I have seen TD3 and DDPG benchmarks on Pybullet environments, but I am looking for SAC benchmarks on Pybullet too, anyone can help?
1
vote
0answers
37 views
What are the procedures to get RL paper results? [closed]
I finished working on a new algorithm in Reinforcement Learning, I need to compare it to some well-known algorithms. That's why I need to know the step-by-step procedures that RL researchers usually ...
1
vote
1answer
115 views
Once the environments are vectorized, how do I have to gather immediate experiences for the agent?
My main purpose right now is to train an agent using the A2C algorithm to solve the Atari Breakout game. So far I have succeeded to create that code with a single agent and environment. To break the ...
1
vote
1answer
90 views
What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?
One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$
For me, $\theta$ are just ...
1
vote
0answers
32 views
What is the difference between FC and MLP in as used in PointNet?
I am trying to understand the PointNet network for dealing with point clouds and struggling with understanding the difference between FC and MLP:
"FC is fully connected layer operating on each ...
1
vote
0answers
23 views
How can I use GPT-2 to modify seed text of one form into a different form (LENGTH INVARIANT) whilst retaining meaning?
I am currently starting a research project whereby I am trying to convert text of one form into another.
i.e. If I were to write a seed sentance of the form "Scientists have finally achieved the ...
1
vote
0answers
153 views
Replace epsilon greedy action selection and the standard DQN by an Independent Gaussian Noise Network Model
Here is my code
Recently, I solved the game of Atari Breakout using a classic DQN model. The convergence of the mean reward slowly improved during three days. I was interested in learning a method ...
1
vote
1answer
35 views
Why are the rewards of my RL agent for the Atari Breakout game decreasing after a certain number of episodes?
The agent is trying to master the Atari Breakout game.
Here is my code
Is that normal that reward_100 decreased that much after it hits 4.5? Is there a way to ...
1
vote
1answer
40 views
2
votes
0answers
46 views
Why does GAN loss converge to log(2) and not -log(2)?
In Goodfellow's paper, he says:
Hence, by inspecting Eq. 4 at $D^*_G (\mathbf{x}) = \frac{1}{2}$, we find $C(G) = \log \frac{1}{2}+ \log \frac{1}{2} = − \log 4$. To see that this is the best ...
0
votes
1answer
92 views
Can you train Transformers sequentially?
I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code
...
1
vote
0answers
35 views
Is it possible to do token classification using a model such as GPT-2?
I am trying to use PyTorch's transformers as a part of a research project to do sentiment analysis of several types of review data (laptop and restaurant).
To do this, my team is taking a token-...
1
vote
1answer
44 views
time-series prediction : loss going down, then stagnates with very high variance
I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and ...
1
vote
0answers
47 views
What are the pros and cons of Keras, PyTorch and Caffe for computer vision?
I have tried to get the basic grasp of the following deep learning frameworks with python:
Keras
Pytorch
Caffe
However, I have lately noticed that people in the computer vision community care less ...
0
votes
0answers
25 views
Which hidden state should I use for a trajectory when incorporating LSTM into RL?
I'm trying to wrap my head around using LSTM in an RL algorithm like actor-critic or PPO. I've found this Github code which presents this in a very simple manner, however I have a very limited ...
1
vote
0answers
30 views
Pytorch deep learning models and tabular data representation
I have quite a naive question regarding Pytorch deep learning models and tabular data representation. So, assume I have a dictionary of tables. Each table has some number of columns: categorical and ...
4
votes
1answer
116 views
Is there a reason to use TensorFlow over PyTorch for research purposes?
I've been using PyTorch to do research for a while and it seems to be quite easy to implement new things with. Also, it is easy to learn and I didn't have any problem with following other researchers ...
2
votes
0answers
68 views
What is the equivalent PyTorch version of tensorflow lite
Update
Checked the PyTorch Mobile which is designed to Android and iOS. Although according to the document, it says it can build for ARM CPUs, but there isn't any documentation mention about how to ...
1
vote
1answer
53 views
RealNVP gives wrong probabilities
I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long)...
2
votes
0answers
46 views
Why does the BatchNormalization layer produce different outputs during training and inference?
I modified resnet50 architecture to get a regression network. I just add batchnorm1d and ReLU layers just before the fully connected layer. During the training, the output of batchnorm1d layer is ...
4
votes
1answer
108 views
What are the differences between TensorFlow and PyTorch? [closed]
What are the differences between TensorFlow and PyTorch, both in terms of performance and functionality?
0
votes
1answer
43 views
Why is this deep Q agent constantly learning just one action?
I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doing wrong? Here is the Github repository with ...
1
vote
1answer
45 views
Why does PyTorch use a different formula for the cross-entropy?
In my understanding, the formula to calculate the cross-entropy is
$$
H(p,q) = - \sum p_i \log(q_i)
$$
But in PyTorch nn.CrossEntropyLoss is calculated using this ...
1
vote
0answers
62 views
FastAi How to turn off crop in ImageDataBunch
I just trained my birds model. It works fine when I was testing it with close pictures.
But when I moved the pictures further away my camera, the model was not able to detect birds.
My guess is in ...
1
vote
0answers
39 views
Why is my variational auto-encoder generating random noise?
This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
1
vote
0answers
26 views
How can I train a neural network to find the hyper-parameters with which the data was generated?
I have 10000 tuples of numbers (x1, x2, y) generated from the equation: y = np.cos(0.583 * x1) + np.exp(0.112 * x2). I want to ...
1
vote
0answers
46 views
Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?
I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this paper. As a reference, I am also using this Github repository (another implementation).
My model/idea is pretty ...
