Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
150
questions
-1
votes
0
answers
22
views
Why I cannot use CrossEntropyLoss for an input size (7,7,2048) and a target size (7)? [closed]
I have an output tensor with size (7,7,2048) and a target tensor with size (7). When I am trying to calculate cross-entropy loss with this code: loss = nn.CrossEntropyLoss(output, targets), this error ...
- 9
0
votes
0
answers
11
views
Comparison of model performances in Pytorch and Tensorflow implementations of Differential privacy [closed]
I'm exploring differential privacy implementations of tensorflow-privacy and pytorch Opacus on Titanic dataset and other linear regression datasets. Tensorflow implementations of differential privacy ...
- 121
0
votes
0
answers
20
views
Computing d1 and d2 for weighted cross entropy in U-Net paper [closed]
I am trying to implement the original U-Net paper. They are using weighted cross entropy and i get it but i couldn't find how can i calculate the d2 value. I can use scipy's binary_transform_edt ...
- 1
1
vote
2
answers
75
views
How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?
To generate synthetic dataset using a trained VAE, there is confusion between two approaches:
Use learned latent space: z = mu + (eps * log_var) to generate (...
- 183
1
vote
0
answers
8
views
DDPG agent with convolutional layers for feature extraction [closed]
I'm trying to come up with a definition of the critic for a DDPG agent in PyTorch using a CNN as a feature extractor. It is pretty straight forward for the actor model. However, for the critic model I ...
1
vote
2
answers
40
views
What should I think about when designing a custom loss function?
I'm trying to get my toy network to learn a sine wave.
I output (via tanh) a number between -1 and 1, and I want the network to minimise the following loss, where ...
- 113
2
votes
0
answers
23
views
How to create a loss function that penalizes duplicate indices in the output tensor?
We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
- 121
1
vote
0
answers
21
views
How can I produce crossbred images out of two datasets?
I'm very new to AI and deep learning. So my question is going to be very basic.
I'm trying to understand which approach I would need to use to cross-breed set of images. Let's say I'm having dataset ...
2
votes
1
answer
92
views
When exactly does the split into different heads in Multi-Head-Attention occur?
I am confused by the Multi-Head part of the Multi-Head-Attention used in Transformers. My question concerns the implementations in Pytorch of nn.MultiheadAttention and its forward method ...
- 23
0
votes
0
answers
15
views
Using an RNN for predicting columns of characters
I'm making an RNN using pytorch to learn from columns of tiles (each tile represented by a text character) and predict the next column of tiles. The training sequences are from maps of level data ...
- 1
0
votes
1
answer
68
views
How can I reduce the loss? Why do I have the high loss and why do I have the gradient?
I want to classify some images (there are about 200.000 images) with a CNN. But I get a very high loss, see figures:
Loss over the hole training run
Loss for each epoch
It's confused me, that there ...
0
votes
0
answers
12
views
How is state size obtained/calculated?
I am looking at the Generator portion of the Pytorch tutorial for Generative Adversarial Networks and I am confused as to how the last 2 dimensions are obtained for what is ...
- 101
0
votes
1
answer
53
views
1D Sequence Classification with self-supervised learning
I am working on a multi-class classification task on long one-dimensional sequences. The sequence length may vary in the range $[512, 30720]$, and there is one feature associated each time-step in the ...
- 101
0
votes
0
answers
40
views
Training a u-net for multi-landmark heatmap regression producing the same heatmap for each channel
I’m training a U-Net (model below) to predict 4 heatmaps (gaussian centered around a keypoint, one in each channel). Each channel is for some reason outputting the same result, an example is given of ...
- 1
0
votes
0
answers
17
views
Matching a reconstructed 3D Face Model and a 2D Image
I'm working on a project where I should replace the face in the original video with a reconstructed face. I read a lot of articles about keypoints matching and deepfake but there are no pretrained ...
- 1
2
votes
1
answer
99
views
Does a colour consistency loss in neural networks (cycleGAN) make sense?
My neural network takes an image as an input and outputs another image. It's the generator of a cycleGAN.
I would like to add (to the discriminator loss, the ...
0
votes
0
answers
12
views
Can you use both copy mechanism and BPE?
I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques:
BPE
Copy mechanism
It appears to me they are two orthogonal approaches.
Can we combine the two, i.e., we use ...
- 51
1
vote
0
answers
25
views
How to define a custom layer in Pytorch [closed]
I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
- 11
0
votes
1
answer
52
views
GANs inputs normalized and generator only outputs in [-1; 1]
I'm currently coding a GAN on the dataset MNIST.
I'm using the following code to transform my data:
...
0
votes
0
answers
18
views
Why doesn't torch use a nonlinearity in its RNN implementation?
The RNN example implementation and the RNN tutorial from pytorch doesn't use a nonlinarity in the hidden layer.
Shouldn't the network have at least one nonlinear activation to be able to learn ...
- 240
0
votes
1
answer
77
views
Which approach can I use to generate forged signatures from real ones?
I am in internship period and I'm working on a signature verification problem.
This process needs real and forged signatures. All I have are the real signatures (like 30 signatures per person), and I ...
0
votes
0
answers
19
views
How to interpret Transformer output
In this (https://towardsdatascience.com/a-detailed-guide-to-pytorchs-nn-transformer-module-c80afbc9ffb1) article the author says, that the output of the ...
0
votes
0
answers
126
views
Why is my Advantage Actor Critic agent learning and then forgetting (i.e. the loss drops to zero)?
I am writing my own version of the A2C algorithm in PyTorch, and for the most part my algorithm is learning the simple environments (like CartPole-v1). Unfortunately during the training process, the ...
- 46
2
votes
2
answers
238
views
How can the input order of pairs into a neural network not matter (i.e. symmetry)?
Let me explain, suppose we are building a neural network that predicts if two items are similar or not. This is a classification task with hard labels (0, 1) of examples of similar and dissimilar ...
- 59
1
vote
1
answer
67
views
Why doesn't the high precision of neural network weights improve accuracy?
Consider the following paragraph from the subsubsection 3.5.2: A dtype for every occasion chapter named It starts with a tensor from the textbook titled Deep Learning with PyTorch by Eli Stevens et al....
- 3,099
0
votes
1
answer
35
views
What does it mean by "lazy mean" here?
Consider the following paragraph, taken from 3.4: Named Tensors of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the calculation of the mean for RGB channels of an RGB ...
- 3,099
1
vote
0
answers
29
views
How to compute the loss for a sequence labeling task without the Softmax distribution?
For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (...
- 123
0
votes
0
answers
21
views
How does Stack-Augmented Recurrent Nets in work?
I am new to RNN/LSTM and I am working on a project about language modeling. I just got familiarized with simple RNN and LSTM. However, these simple models did not achieve the performance I want. Since ...
- 121
1
vote
2
answers
54
views
How to make NN distinguish between two types of functions (data)?
I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict ...
- 113
0
votes
1
answer
168
views
Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression
Below are the two tensors
...
- 111
1
vote
1
answer
198
views
What exactly is embedding layer used in RNN encoders?
I am reading about RNN encoders. I came across the following line from this code. And I am facing difficulty in understanding the theoretical details regarding it.
...
- 3,099
0
votes
0
answers
134
views
ignoring instances or masking by zero in multitask learning model
For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
- 123
0
votes
0
answers
24
views
Proper loss function for regression with uniform target distribution
I'm doing some simulations and I would like to estimate a real number that is uniformly distributed between minValue and maxValue...
- 101
0
votes
0
answers
31
views
What loss function should be used for negative log likelihood labels
I am trying to build a ranking CNN model for document - query pairs using MS Marco dataset and python pytorch. My supervisor suggested to use the same CNN to extract feature vector for document and ...
- 101
3
votes
1
answer
97
views
Why do we multipy context_size with embedding_dim? (PyTorch)
I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-...
- 145
3
votes
0
answers
46
views
How are partial derivatives calculated in a computational graph?
I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at ...
- 131
3
votes
1
answer
178
views
What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?
In PyTorch, transformer (BERT) models have an intermediate dense layer in between attention and output layers whereas the BERT and Transformer papers just mention the attention connected directly to ...
1
vote
1
answer
136
views
Multi label classification on non binary labels with pytorch
I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).
Would be the best approach to hard ...
- 11
1
vote
1
answer
37
views
Is there any recommended way to perform pooling in this context?
Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
- 3,099
0
votes
0
answers
68
views
Without using data augmentation gives results better than using data augmentation
I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images).
I am doing transfer learning (pre-trained ...
- 1
2
votes
1
answer
275
views
What exactly happens in gradient clipping by norm?
Consider the following description regarding gradient clipping in PyTorch
...
- 3,099
0
votes
1
answer
65
views
How to pass multiple vectors and numeric features as input to the neural network?
I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
0
votes
0
answers
46
views
How to increase accuracy of image orientation classification (Left, Right, Center)?
I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this:
The images are "Left", "...
- 101
1
vote
1
answer
72
views
Joined vs Separate optimizer for Actor-Critic
Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
- 143
1
vote
1
answer
182
views
Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]
I have trained a classification network with PyTorch lightning where my training step looks like below:
...
- 25
0
votes
0
answers
32
views
Is it normal that we get different AUC results after running with various seeds?
We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch.
We saw that choosing ...
1
vote
0
answers
39
views
Are there any benefits of adding attention to linear layers?
Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
- 13
0
votes
1
answer
86
views
How do CNNs handle inputs of different sizes and shapes?
I am new to deep learning so feel free to correct me where I am wrong.
Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
- 65
-1
votes
1
answer
55
views
How to re-training an AI model to have smaller input image size
I need a PyTorch Model which can do road segmentation on OAK-D camera.
The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
- 181
0
votes
1
answer
615
views
Deep Q-Learning with multiple discrete actions
I am working on a DQN project with Pytorch, where I should choose multiple discrete actions, each in a range, say, (0, 15). I am wondering how I can model it, such ...
- 1