Questions tagged [pytorch]

For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.

150 questions
Filter by
Sorted by
Tagged with
22 views

Why I cannot use CrossEntropyLoss for an input size (7,7,2048) and a target size (7)? [closed]

I have an output tensor with size (7,7,2048) and a target tensor with size (7). When I am trying to calculate cross-entropy loss with this code: loss = nn.CrossEntropyLoss(output, targets), this error ...
• 9
11 views

Comparison of model performances in Pytorch and Tensorflow implementations of Differential privacy [closed]

I'm exploring differential privacy implementations of tensorflow-privacy and pytorch Opacus on Titanic dataset and other linear regression datasets. Tensorflow implementations of differential privacy ...
• 121
20 views

Computing d1 and d2 for weighted cross entropy in U-Net paper [closed]

I am trying to implement the original U-Net paper. They are using weighted cross entropy and i get it but i couldn't find how can i calculate the d2 value. I can use scipy's binary_transform_edt ...
• 1
1 vote
75 views

How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?

To generate synthetic dataset using a trained VAE, there is confusion between two approaches: Use learned latent space: z = mu + (eps * log_var) to generate (...
• 183
1 vote
8 views

DDPG agent with convolutional layers for feature extraction [closed]

I'm trying to come up with a definition of the critic for a DDPG agent in PyTorch using a CNN as a feature extractor. It is pretty straight forward for the actor model. However, for the critic model I ...
1 vote
40 views

What should I think about when designing a custom loss function?

I'm trying to get my toy network to learn a sine wave. I output (via tanh) a number between -1 and 1, and I want the network to minimise the following loss, where ...
• 113
23 views

How to create a loss function that penalizes duplicate indices in the output tensor?

We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
• 121
1 vote
21 views

How can I produce crossbred images out of two datasets?

I'm very new to AI and deep learning. So my question is going to be very basic. I'm trying to understand which approach I would need to use to cross-breed set of images. Let's say I'm having dataset ...
92 views

I am confused by the Multi-Head part of the Multi-Head-Attention used in Transformers. My question concerns the implementations in Pytorch of nn.MultiheadAttention and its forward method ...
15 views

Using an RNN for predicting columns of characters

I'm making an RNN using pytorch to learn from columns of tiles (each tile represented by a text character) and predict the next column of tiles. The training sequences are from maps of level data ...
68 views

How can I reduce the loss? Why do I have the high loss and why do I have the gradient?

I want to classify some images (there are about 200.000 images) with a CNN. But I get a very high loss, see figures: Loss over the hole training run Loss for each epoch It's confused me, that there ...
12 views

How is state size obtained/calculated?

I am looking at the Generator portion of the Pytorch tutorial for Generative Adversarial Networks and I am confused as to how the last 2 dimensions are obtained for what is ...
53 views

1D Sequence Classification with self-supervised learning

I am working on a multi-class classification task on long one-dimensional sequences. The sequence length may vary in the range $[512, 30720]$, and there is one feature associated each time-step in the ...
• 101
40 views

Training a u-net for multi-landmark heatmap regression producing the same heatmap for each channel

I’m training a U-Net (model below) to predict 4 heatmaps (gaussian centered around a keypoint, one in each channel). Each channel is for some reason outputting the same result, an example is given of ...
17 views

Matching a reconstructed 3D Face Model and a 2D Image

I'm working on a project where I should replace the face in the original video with a reconstructed face. I read a lot of articles about keypoints matching and deepfake but there are no pretrained ...
99 views

Does a colour consistency loss in neural networks (cycleGAN) make sense?

My neural network takes an image as an input and outputs another image. It's the generator of a cycleGAN. I would like to add (to the discriminator loss, the ...
12 views

Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
1 vote
25 views

How to define a custom layer in Pytorch [closed]

I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
• 11
52 views

GANs inputs normalized and generator only outputs in [-1; 1]

I'm currently coding a GAN on the dataset MNIST. I'm using the following code to transform my data: ...
18 views

Why doesn't torch use a nonlinearity in its RNN implementation?

The RNN example implementation and the RNN tutorial from pytorch doesn't use a nonlinarity in the hidden layer. Shouldn't the network have at least one nonlinear activation to be able to learn ...
• 240
77 views

Which approach can I use to generate forged signatures from real ones?

I am in internship period and I'm working on a signature verification problem. This process needs real and forged signatures. All I have are the real signatures (like 30 signatures per person), and I ...
19 views

How to interpret Transformer output

In this (https://towardsdatascience.com/a-detailed-guide-to-pytorchs-nn-transformer-module-c80afbc9ffb1) article the author says, that the output of the ...
126 views

Why is my Advantage Actor Critic agent learning and then forgetting (i.e. the loss drops to zero)?

I am writing my own version of the A2C algorithm in PyTorch, and for the most part my algorithm is learning the simple environments (like CartPole-v1). Unfortunately during the training process, the ...
• 46
238 views

How can the input order of pairs into a neural network not matter (i.e. symmetry)?

Let me explain, suppose we are building a neural network that predicts if two items are similar or not. This is a classification task with hard labels (0, 1) of examples of similar and dissimilar ...
• 59
1 vote
67 views

Why doesn't the high precision of neural network weights improve accuracy?

Consider the following paragraph from the subsubsection 3.5.2: A dtype for every occasion chapter named It starts with a tensor from the textbook titled Deep Learning with PyTorch by Eli Stevens et al....
• 3,099
35 views

What does it mean by "lazy mean" here?

Consider the following paragraph, taken from 3.4: Named Tensors of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the calculation of the mean for RGB channels of an RGB ...
• 3,099
1 vote
29 views

How to compute the loss for a sequence labeling task without the Softmax distribution?

For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (...
• 123
21 views

How does Stack-Augmented Recurrent Nets in work?

I am new to RNN/LSTM and I am working on a project about language modeling. I just got familiarized with simple RNN and LSTM. However, these simple models did not achieve the performance I want. Since ...
• 121
1 vote
54 views

How to make NN distinguish between two types of functions (data)?

I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict ...
168 views

Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

Below are the two tensors ...
• 111
1 vote
198 views

What exactly is embedding layer used in RNN encoders?

I am reading about RNN encoders. I came across the following line from this code. And I am facing difficulty in understanding the theoretical details regarding it. ...
• 3,099
134 views

For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
• 123
24 views

Proper loss function for regression with uniform target distribution

I'm doing some simulations and I would like to estimate a real number that is uniformly distributed between minValue and maxValue...
• 101
31 views

What loss function should be used for negative log likelihood labels

I am trying to build a ranking CNN model for document - query pairs using MS Marco dataset and python pytorch. My supervisor suggested to use the same CNN to extract feature vector for document and ...
• 101
97 views

Why do we multipy context_size with embedding_dim? (PyTorch)

I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-...
• 145
46 views

How are partial derivatives calculated in a computational graph?

I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at ...
• 131
178 views

What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?

In PyTorch, transformer (BERT) models have an intermediate dense layer in between attention and output layers whereas the BERT and Transformer papers just mention the attention connected directly to ...
1 vote
136 views

Multi label classification on non binary labels with pytorch

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ). Would be the best approach to hard ...
1 vote
37 views

Is there any recommended way to perform pooling in this context?

Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
• 3,099
68 views

Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images). I am doing transfer learning (pre-trained ...
• 1
275 views

What exactly happens in gradient clipping by norm?

Consider the following description regarding gradient clipping in PyTorch ...
• 3,099
65 views

How to pass multiple vectors and numeric features as input to the neural network?

I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
46 views

How to increase accuracy of image orientation classification (Left, Right, Center)?

I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this: The images are "Left", "...
• 101
1 vote
72 views

Joined vs Separate optimizer for Actor-Critic

Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
• 143
1 vote
182 views

Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]

I have trained a classification network with PyTorch lightning where my training step looks like below: ...
• 25
32 views

Is it normal that we get different AUC results after running with various seeds?

We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch. We saw that choosing ...
1 vote
39 views

Are there any benefits of adding attention to linear layers?

Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
• 13
86 views

How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
55 views

How to re-training an AI model to have smaller input image size

I need a PyTorch Model which can do road segmentation on OAK-D camera. The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
• 181