# Questions tagged [pytorch]

For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.

150 questions
Filter by
Sorted by
Tagged with
22 views

### Why I cannot use CrossEntropyLoss for an input size (7,7,2048) and a target size (7)? [closed]

I have an output tensor with size (7,7,2048) and a target tensor with size (7). When I am trying to calculate cross-entropy loss with this code: loss = nn.CrossEntropyLoss(output, targets), this error ...
11 views

### Comparison of model performances in Pytorch and Tensorflow implementations of Differential privacy [closed]

I'm exploring differential privacy implementations of tensorflow-privacy and pytorch Opacus on Titanic dataset and other linear regression datasets. Tensorflow implementations of differential privacy ...
20 views

### Computing d1 and d2 for weighted cross entropy in U-Net paper [closed]

I am trying to implement the original U-Net paper. They are using weighted cross entropy and i get it but i couldn't find how can i calculate the d2 value. I can use scipy's binary_transform_edt ...
1 vote
75 views

### How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?

To generate synthetic dataset using a trained VAE, there is confusion between two approaches: Use learned latent space: z = mu + (eps * log_var) to generate (...
1 vote
8 views

### DDPG agent with convolutional layers for feature extraction [closed]

I'm trying to come up with a definition of the critic for a DDPG agent in PyTorch using a CNN as a feature extractor. It is pretty straight forward for the actor model. However, for the critic model I ...
1 vote
40 views

### What should I think about when designing a custom loss function?

I'm trying to get my toy network to learn a sine wave. I output (via tanh) a number between -1 and 1, and I want the network to minimise the following loss, where ...
23 views

### How to create a loss function that penalizes duplicate indices in the output tensor?

We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
1 vote
21 views

### How can I produce crossbred images out of two datasets?

I'm very new to AI and deep learning. So my question is going to be very basic. I'm trying to understand which approach I would need to use to cross-breed set of images. Let's say I'm having dataset ...
92 views

### When exactly does the split into different heads in Multi-Head-Attention occur?

I am confused by the Multi-Head part of the Multi-Head-Attention used in Transformers. My question concerns the implementations in Pytorch of nn.MultiheadAttention and its forward method ...
15 views

### Using an RNN for predicting columns of characters

I'm making an RNN using pytorch to learn from columns of tiles (each tile represented by a text character) and predict the next column of tiles. The training sequences are from maps of level data ...
68 views

### How can I reduce the loss? Why do I have the high loss and why do I have the gradient?

I want to classify some images (there are about 200.000 images) with a CNN. But I get a very high loss, see figures: Loss over the hole training run Loss for each epoch It's confused me, that there ...
12 views

### How is state size obtained/calculated?

I am looking at the Generator portion of the Pytorch tutorial for Generative Adversarial Networks and I am confused as to how the last 2 dimensions are obtained for what is ...
53 views

### 1D Sequence Classification with self-supervised learning

I am working on a multi-class classification task on long one-dimensional sequences. The sequence length may vary in the range $[512, 30720]$, and there is one feature associated each time-step in the ...
40 views

### Training a u-net for multi-landmark heatmap regression producing the same heatmap for each channel

I’m training a U-Net (model below) to predict 4 heatmaps (gaussian centered around a keypoint, one in each channel). Each channel is for some reason outputting the same result, an example is given of ...
17 views

### Matching a reconstructed 3D Face Model and a 2D Image

I'm working on a project where I should replace the face in the original video with a reconstructed face. I read a lot of articles about keypoints matching and deepfake but there are no pretrained ...
99 views

### Does a colour consistency loss in neural networks (cycleGAN) make sense?

My neural network takes an image as an input and outputs another image. It's the generator of a cycleGAN. I would like to add (to the discriminator loss, the ...
12 views

### Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
1 vote
25 views

### How to define a custom layer in Pytorch [closed]

I am new to PyTorch and seeking your help regarding a problem I have. I need to add a costume layer to a NN in training phase. Please see the figure which shows a simple DNN with the custom layer. NN ...
52 views

### GANs inputs normalized and generator only outputs in [-1; 1]

I'm currently coding a GAN on the dataset MNIST. I'm using the following code to transform my data: ...
18 views

### Why doesn't torch use a nonlinearity in its RNN implementation?

The RNN example implementation and the RNN tutorial from pytorch doesn't use a nonlinarity in the hidden layer. Shouldn't the network have at least one nonlinear activation to be able to learn ...
77 views

### Which approach can I use to generate forged signatures from real ones?

I am in internship period and I'm working on a signature verification problem. This process needs real and forged signatures. All I have are the real signatures (like 30 signatures per person), and I ...
19 views

### How to interpret Transformer output

In this (https://towardsdatascience.com/a-detailed-guide-to-pytorchs-nn-transformer-module-c80afbc9ffb1) article the author says, that the output of the ...
126 views

### Why is my Advantage Actor Critic agent learning and then forgetting (i.e. the loss drops to zero)?

I am writing my own version of the A2C algorithm in PyTorch, and for the most part my algorithm is learning the simple environments (like CartPole-v1). Unfortunately during the training process, the ...
238 views

### How can the input order of pairs into a neural network not matter (i.e. symmetry)?

Let me explain, suppose we are building a neural network that predicts if two items are similar or not. This is a classification task with hard labels (0, 1) of examples of similar and dissimilar ...
1 vote
67 views

### Why doesn't the high precision of neural network weights improve accuracy?

Consider the following paragraph from the subsubsection 3.5.2: A dtype for every occasion chapter named It starts with a tensor from the textbook titled Deep Learning with PyTorch by Eli Stevens et al....
35 views

### What does it mean by "lazy mean" here?

Consider the following paragraph, taken from 3.4: Named Tensors of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the calculation of the mean for RGB channels of an RGB ...
1 vote
29 views

### How to compute the loss for a sequence labeling task without the Softmax distribution?

For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (...
21 views

### How does Stack-Augmented Recurrent Nets in work?

I am new to RNN/LSTM and I am working on a project about language modeling. I just got familiarized with simple RNN and LSTM. However, these simple models did not achieve the performance I want. Since ...
1 vote
54 views

### How to make NN distinguish between two types of functions (data)?

I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict ...
168 views

### Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

Below are the two tensors ...
1 vote
198 views

### What exactly is embedding layer used in RNN encoders?

I am reading about RNN encoders. I came across the following line from this code. And I am facing difficulty in understanding the theoretical details regarding it. ...
134 views

For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
24 views

### Proper loss function for regression with uniform target distribution

I'm doing some simulations and I would like to estimate a real number that is uniformly distributed between minValue and maxValue...
31 views

### What loss function should be used for negative log likelihood labels

I am trying to build a ranking CNN model for document - query pairs using MS Marco dataset and python pytorch. My supervisor suggested to use the same CNN to extract feature vector for document and ...
97 views

### Why do we multipy context_size with embedding_dim? (PyTorch)

I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-...
46 views

### How are partial derivatives calculated in a computational graph?

I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at ...
178 views

### What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?

In PyTorch, transformer (BERT) models have an intermediate dense layer in between attention and output layers whereas the BERT and Transformer papers just mention the attention connected directly to ...
1 vote
136 views

### Multi label classification on non binary labels with pytorch

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ). Would be the best approach to hard ...
1 vote
37 views

### Is there any recommended way to perform pooling in this context?

Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
68 views

### Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images). I am doing transfer learning (pre-trained ...
275 views

### What exactly happens in gradient clipping by norm?

Consider the following description regarding gradient clipping in PyTorch ...
65 views

### How to pass multiple vectors and numeric features as input to the neural network?

I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
46 views

### How to increase accuracy of image orientation classification (Left, Right, Center)?

I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this: The images are "Left", "...
1 vote
72 views

### Joined vs Separate optimizer for Actor-Critic

Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
1 vote
182 views

### Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]

I have trained a classification network with PyTorch lightning where my training step looks like below: ...
32 views

### Is it normal that we get different AUC results after running with various seeds?

We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch. We saw that choosing ...
1 vote
39 views

### Are there any benefits of adding attention to linear layers?

Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
86 views

### How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...