Questions tagged [pytorch]
For questions related to the deep learning PyTorch framework.
48
questions
0
votes
1answer
26 views
1
vote
0answers
25 views
Why does GAN loss converge to log(2) and not -log(2)?
In Goodfellow's paper, he says:
Hence, by inspecting Eq. 4 at $D^*_G (\mathbf{x}) = \frac{1}{2}$, we find $C(G) = \log \frac{1}{2}+ \log \frac{1}{2} = − \log 4$. To see that this is the best ...
0
votes
1answer
50 views
Can you train Transformers sequentially?
I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code
...
0
votes
0answers
17 views
Understanding log_prob for Normal distribution in pytorch [migrated]
I'm currently trying to solve Pendulum-v0 from the openAi gym environment which has a continuous action space. As a result, I need to use a Normal Distribution to sample my actions. What I don't ...
1
vote
0answers
16 views
Error: AttributeError: module 'transformers' has no attribute 'TFBertModel' [migrated]
I am applying transfer learning with the python framework (PyTorch). I am getting the below error, when loading a PyTorch pre-trained model in Google Colab. After changing the code 1 to be as code 2, ...
1
vote
0answers
22 views
Is it possible to do token classification using a model such as GPT-2?
I am trying to use PyTorch's transformers as a part of a research project to do sentiment analysis of several types of review data (laptop and restaurant).
To do this, my team is taking a token-...
0
votes
1answer
27 views
time-series prediction : loss going down, then stagnates with very high variance
I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and ...
1
vote
0answers
34 views
What are the pros and cons of Keras, PyTorch and Caffe for computer vision?
I have tried to get the basic grasp of the following deep learning frameworks with python:
Keras
Pytorch
Caffe
However, I have lately noticed that people in the computer vision community care less ...
0
votes
0answers
12 views
Which hidden state should I use for a trajectory when incorporating LSTM into RL?
I'm trying to wrap my head around using LSTM in an RL algorithm like actor-critic or PPO. I've found this Github code which presents this in a very simple manner, however I have a very limited ...
0
votes
0answers
16 views
Pytorch deep learning models and tabular data representation
I have quite a naive question regarding Pytorch deep learning models and tabular data representation. So, assume I have a dictionary of tables. Each table has some number of columns: categorical and ...
4
votes
1answer
98 views
Is there a reason to use TensorFlow over PyTorch for research purposes?
I've been using PyTorch to do research for a while and it seems to be quite easy to implement new things with. Also, it is easy to learn and I didn't have any problem with following other researchers ...
2
votes
0answers
34 views
What is the equivalent PyTorch version of tensorflow lite
Update
Checked the PyTorch Mobile which is designed to Android and iOS. Although according to the document, it says it can build for ARM CPUs, but there isn't any documentation mention about how to ...
1
vote
0answers
20 views
RealNVP gives wrong probabilities
I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long)...
1
vote
0answers
28 views
Why does the BatchNormalization layer produce different outputs during training and inference?
I modified resnet50 architecture to get a regression network. I just add batchnorm1d and ReLU layers just before the fully connected layer. During the training, the output of batchnorm1d layer is ...
3
votes
1answer
80 views
What are the differences between TensorFlow and PyTorch?
What are the differences between TensorFlow and PyTorch, both in terms of performance and functionality?
0
votes
1answer
29 views
Why is this deep Q agent constantly learning just one action?
I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doning wrong? Here is the Github repository with ...
1
vote
1answer
37 views
Why does PyTorch use a different formula for the cross-entropy?
In my understanding, the formula to calculate the cross-entropy is
$$
H(p,q) = - \sum p_i \log(q_i)
$$
But in PyTorch nn.CrossEntropyLoss is calculated using this ...
0
votes
0answers
29 views
FastAi How to turn off crop in ImageDataBunch
I just trained my birds model. It works fine when I was testing it with close pictures.
But when I moved the pictures further away my camera, the model was not able to detect birds.
My guess is in ...
1
vote
0answers
27 views
Why is my variational auto-encoder generating random noise?
This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
1
vote
0answers
23 views
How can I train a neural network to find the hyper-parameters with which the data was generated?
I have 10000 tuples of numbers (x1, x2, y) generated from the equation: y = np.cos(0.583 * x1) + np.exp(0.112 * x2). I want to ...
1
vote
0answers
33 views
Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?
I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this paper. As a reference, I am also using this Github repository (another implementation).
My model/idea is pretty ...
2
votes
0answers
129 views
How can I train YOLO with the COCO dataset?
I am trying to implement the original YOLO architecture for object detection, but I am using the COCO dataset. However, I am a bit confused about the image sizes of COCO. The original YOLO was trained ...
1
vote
0answers
10 views
Can Grad CAM feature maps be used for Training?
I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf
Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
0
votes
0answers
53 views
Focal loss for imbalanced multi class classification in Pytorch
I want an example code for Focal loss in PyTorch for a model with three class prediction. My model outputs 3 probabilities.
...
1
vote
0answers
46 views
How to train LSTM score prediction with very little data? (Bounty to be added)
I am trying to make a text score prediction network, and my dataset have 500 samples only. I know there is a public dataset called the ASAP Dataset. I have tested my model ...
1
vote
1answer
62 views
How is the policy gradient's derivative work?
I am trying to understand policy gradient method using a pytorch implementation and this tutorial.
My first question is about the end result of this gradient derivation,
mainly in this equation
$\...
3
votes
1answer
41 views
Training network with 4 GPUs performance is not exactly 4 times over one GPU why?
Training neural network with 4 GPUs using pyTorch, performance is not even 2 times (btw 1 & 2 times) compare to using one GPU. From Nvidia-smi we see GPU usage is for few milliseconds and next 5-...
0
votes
1answer
46 views
Is this learning rate schedule increasing the learning rate?
I was reading a PyTorch code then I saw this learning rate scheduler:
...
0
votes
1answer
110 views
Confused with backprop in pytorch with BCE loss
I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.
...
2
votes
0answers
616 views
How to transfer learn Darknet YOLOv3
I've started getting into object detection in image. I have YOLOv3 neural network with Darknet framework. The network is pre-trained from COCO data set. Now I need to do some transfer learning in ...
3
votes
1answer
81 views
Issue at training simple RNN for word generation
After completing Coursera course from Andrew Ng, I wanted to implement again simple RNN for generating dinosaurs name based on a text file containing around 800 dinosaurs name.
This is done with ...
0
votes
0answers
41 views
Training, validation loss and accuracy yolov3?
This is a version of Yolo V3 implemented in PyTorch – YOLOv3 in PyTorch
I am trying to use transfer learning to train this yolov3 implementation following the directions given in this post. This is ...
1
vote
0answers
32 views
Initial LSTM hidden state and cell
If we use LSTMCell from torch:
The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds?
...
0
votes
0answers
23 views
Super Resolution CNN generates black dots on output images
I have been trying to train a CNN for the super-resolution task based on the work of Dong et al., 2015 [1]. The network structure built in PyTorch is as follows:
...
1
vote
1answer
119 views
Are the training loss and validation loss plotted per sample or per batch?
I am using a CNN to train on some data, where training size = 21700 samples, and test size is 653 samples, and say I am using a batch_size of 500 (I am accounting for samples out of batch size as well)...
1
vote
1answer
76 views
LSTM text classifier shows unexpected cyclical pattern in loss
I'm training a text classifier in PyTorch and I'm experiencing an unexplainable cyclical pattern in the loss curve. The loss drops drastically at the beginning of each epoch and then starts rising ...
0
votes
0answers
25 views
How to implement Multiple Neural network architecture, connected in parallel and series in Keras or Pytorch
Hello Dear StackExchange members,
I want to make a deep network as shown in the image. I want each 'network 1 to look at the specific part of the input and I don't want to divide my input beforehand ...
-1
votes
1answer
73 views
Which is better to start deep learning and understand it in depth (and not just a simple overview) - pytorch or tensorflow 2.0?
I am beginning to learn deep learning. I recently spoke with an expert in the field. He suggested that I start with pytorch because of these reasons:
Keras abstracts the stuff a lot that we will not ...
0
votes
0answers
14 views
Limits for a bottleneck
I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-...
1
vote
0answers
31 views
Add a layer derivative in the loss function
I am writing a NN in pytorch and I want to add the derivative of the output with respect to one of the inner layers in the loss. Here is a simple example of what I mean:
...
2
votes
1answer
48 views
How to add some data input in a CNN?
There is this problem I have encountered, I was trying to classify the pixels from input image into classes, sort of like segmentation, using a encoder-decoder CNN. The “interested” pixels usually ...
1
vote
1answer
43 views
How to properly optimize shared network between actor and critic?
I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment.
When I share the encoder with the actor ...
1
vote
1answer
127 views
Heavy loss and inaccurate answer in pytorch
As my first AI model I have decided to make an AI model to predict multiplication of two numbers EX - [2,4] = [8]. I wrote the following code, but the loss is very high, around thousands, and it's ...
1
vote
1answer
189 views
LSTM is not converging
I am writing my first LSTM network and I would really appreciate if someone can tell me if it is right (the loss seems to go down very slowly and before playing around with hyper parameters I want to ...
2
votes
1answer
848 views
How to export Pytorch Deep Neural Networks trained model to C++ program to use it from C++? [closed]
I want to build a DNN model that I will later integrate into a C++ program.
I heard that PyTorch model is hard to load it on C++ and the integration requires extra code, and it's complicated.
I have ...
3
votes
1answer
59 views
Mnist CNN Architecture
In this tutorial from Jeremy Howard: What is torch.nn really? he has an example towards the end where he creates a CNN for mnist. In nn.Conv2d he makes the ...
0
votes
0answers
42 views
How to make episode ending “good” in reinforcement learning?
TL;DR: read the bold. The rest are details
I am trying to implement Reinforcement Learning:An Introduction, section 13.5 myself:
on OpenAi's cartpole
The algorithm seems to be learning something ...
2
votes
1answer
461 views
Why isn't my model learning satisfactorily?
The problem to solve is non-linear regression of a non-linear function. My actual problem is to model the function "find the max over many quadratic forms": ...
