Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
138
questions
0
votes
0answers
11 views
Positional Encoding in Transformer on multi-variate time series data hurts performance
I set up a transformer model that embeds positional encodings in the encoder. The data is multi-variate time series-based data.
As I just experiment with the positional encoding portion of the code I ...
0
votes
0answers
12 views
Feeding the output back to input in 3D CNN model
I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below
I have two thoughts in training the model for above situation.
Feed output $...
1
vote
0answers
29 views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
1
vote
1answer
18 views
Are there any stats available on the usage of libraries by deep learning researchers?
I know three Python libraries that are popular in deep learning research community: Keras, PyTorch, Tensorflow. I don't know much about Theano.
This question is not about the efficiency, flexibility ...
1
vote
0answers
11 views
What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch
PyTorch provides max pooling and adaptive max pooling.
Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this ...
1
vote
0answers
12 views
Is there any closed form analytical expression to represent fractional max pooling?
There are Nineteen pooling layers in PyTorch.
Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae is not provided for the fractional max pooling ...
1
vote
0answers
29 views
Is there any animation that illustrates the “fold” and “unfold” operations of convolutional layers?
There are fourteen convolution layers in PyTorch. Among them six are related to convolution, another six are related to transposed convolution. The remaining two are fold and unfold operations.
The ...
1
vote
0answers
10 views
Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?
The basic layers for performing convolution operation in PyTorch are
...
1
vote
1answer
23 views
What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?
PyTorch documentation provided the following descriptions to the Convolution layers
...
1
vote
0answers
30 views
What does 'channel' mean in the case of an 1D convolution?
While reading about 1D-convolution in PyTorch, I encountered the concept of channels
...
1
vote
1answer
42 views
What exactly is an XPU?
I know about CPU, GPU and TPU. But, it is the first time for me to read about XPU from PyTorch documentation about MODULE.
...
0
votes
0answers
31 views
What are the input and output gradients in PyTorch?
Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$.
Let the number of ...
0
votes
1answer
32 views
Is precision of weights unimportant in neural networks?
While reading about Module in PyTorch, I came across a new data type called half datatype.
half() method when calls on a Module ...
0
votes
0answers
16 views
Pytorch-Lightning.Trainer maxes out ram and causes crash when completing epoch [closed]
I have been working on a collab project which trains a model to generate news articles from given headlines.
I am using the Pytorch-lightning.Trainer to train my model for a single epoch. The model is ...
0
votes
0answers
35 views
What are the numbers thar are useful (need be stored) other than parameters of a model?
Consider the following method related to buffers in PyTorch
...
1
vote
1answer
39 views
What does it mean by “zeros the networks parameters gradients” in the context of training a neural network?
Consider the following PyTorch code
...
3
votes
2answers
77 views
Is there any difference between affine transformation and linear transformation?
Consider the following statements from A Simple Custom Module of PyTorch's documentation
To get started, let’s look at a simpler, custom version of PyTorch’s
Linear module. This module applies an ...
0
votes
1answer
34 views
0
votes
1answer
41 views
How does CURL extract labels from logits?
While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
0
votes
0answers
11 views
A2C value function outputs keep increasing
I was implementing the A2C algorithm with as close to baseline setup as possible, and this is the code I came up with. The problem is that even after multiple rechecks, the program isn't showing ...
0
votes
0answers
18 views
How to obtain STD from Neural Network with 2 continuous action output
In my Environment, I have two continuous action space
self.action_space = spaces.Box(low=np.array([0.,0.]), high=np.array([4.,0.02]), shape=(2,), dtype=np.float32)
...
0
votes
0answers
29 views
GAN performs worse after 50 epochs than after 2
I am training GAN on SVHN dataset (house numbers in Google Street View images, dimensions: 3x32x32 - 3 color channels).
The problem is that it performs worse after some training (e.g. after 50 epochs) ...
0
votes
0answers
6 views
How can I get the images with the highest activation for a given unit?
I am new to machine learning. I am working on the pretrained AlexNet on Pytorch and i would like to visualize the receptive fields of a given unit U. To do that I am trying to give like 200K images as ...
1
vote
0answers
44 views
How can I add a Sequential CNN layer on top of BERT model?
Information
I'm working on a binary classification task and used BERT model from transformers library to do it using the custom model below:
...
0
votes
1answer
24 views
Why do I have better RMSE when I don't scale the target? [closed]
I use PyTorch for training a simple neural net for a regression task on a dataset with 12 numerical features + target (target is the 13th column) + 2 categorical features
Before training, I execute
<...
0
votes
1answer
19 views
Changing a CNN-LSTM image captioning architecture to use BiLSTMs
Currently I'm dealing with an assignment that made us implement the network mentioned in this paper. The network has an architecture similar to this:
As you can see it uses a Unidirectional RNN (in ...
0
votes
0answers
16 views
Neural Networks different architectures but similar training curves
I have a base neural network architecture for (3D) image sequences classification, made of conv layers followed by a LSTM and dense layers.
I have 3 similar architectures :
3 Conv -> 1 LSTM -> ...
0
votes
0answers
11 views
Visualizing encoder-attention after ResNet in terms of ResNet input
I have a transform-encoder only architecture, which has the following structure:
...
0
votes
0answers
22 views
Why does my model only predict 1s or 0s in multiclass segmentation?
I am currently trying to train a UNet model on the dronedeploy segmentation dataset. My problem is that I only get outputs like this from my model:
This means that in the output maps I only get 0s ...
7
votes
1answer
102 views
When should you not use the bias in a layer?
I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
0
votes
0answers
12 views
Things to consider while adding custom function to generator output in GAN
I am training a GAN model (DCGAN) to generate 128x128 images. Now, I wish to add a function which will take the generator output, perform some pre-defined operations on it, and return the modified ...
0
votes
0answers
17 views
Adversarial Attacks and interpolation methods
I am attacking a model. The model is a simple CNN and PGD is used. The model runs on 112x112 ImageNet dataset. So I first load images as 224x224 and use interpolation function to downsample it to ...
0
votes
0answers
16 views
Which is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?
Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch.
...
0
votes
0answers
17 views
Can anybody just confirm whether or not my understanding of depthwise separable convolutions is correct?
I just need a simple Yes/No confirmation or to debunk my understanding of the difference between the normal convolutions and depthwise seperable convs. I have read quite a few articles and watched a ...
1
vote
2answers
65 views
CNN based model poor result [closed]
My goal is to train and evaluate The German Traffic Sign Recognition Benchmark (GTSRB) dataset using Pytorch.
I downloaded the datasets from the official site GTSRB_Final_Training_Images.zip and ...
0
votes
0answers
30 views
Meaning of grad_outputs in torch.autograd.grad for complex input and output
Let's say we have a mathematical expression,
$$
\mathbf{y} = \mathbf{Ax},
$$
where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix.
Let's say the vector $\mathbf{y}$ is used ...
0
votes
0answers
52 views
PyTorch: LSTM error while trying to update the hidden state
I am trying to train an LSTM while keeping its hidden state (LSTM stateful) until the moment when I am going to start a new epoch(episode). But here it's come an interesting situation because I am ...
0
votes
0answers
65 views
PyTorch: How to deal with hidden states of an LSTM?
I have a time series in which each date is correlated with the preview one, and base on that I am trying to predict action 1 and action 2. But the problem is that I am not sure how to deal with the <...
-3
votes
1answer
84 views
0
votes
2answers
117 views
Does anybody know what would happen if I changed input shape of pytorch models?
In this https://pytorch.org/vision/stable/models.html tutorial it clearly states:
All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of ...
0
votes
0answers
19 views
Why (not) using pre-processing before using Transformer models?
Regarding the use of pre-processing techniques before using Transformers models, I read this post that apparently says that these measures are not so necessary nor interfere so much in the final ...
0
votes
0answers
15 views
Mixed precision training - why we're fine with doing point wise operations in FP32
I'm starting to learn more about mixed-precision training, and I'm in particular confused about point-wise operations. In the original article (link), the authors mention, citing:
Point-wise ...
1
vote
3answers
65 views
How can I model any structure for a neural network?
Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
0
votes
0answers
18 views
Is there an effective way of obtaining the topic distribution for a given document from a VAE-LDA?
Is there an effective way of obtaining the topic distribution for a given document from a Variational AutoEncoder Latent Dirichlet Allocation (VAE-LDA)?
Most existing public VAE-LDA codebases seem to ...
0
votes
0answers
24 views
How is a tensor implemented? How does it adjust for covariant/contravariant?
This stackoverflow question answers how tensors represent data, but not how they adjust for covariance/contravariance.
I'm interested in understanding how the coupling of tensors with each other is ...
1
vote
0answers
47 views
What's the best way to take a list of lists as DQN input?
I have my own environment for the DQN algorithm. In my environment, the state space is represented by a list of lists, where each sublist can be of different lengths. In my case, the length of the ...
0
votes
0answers
53 views
Are there any good tutorials on using continuous normalizing flows (with PyTorch)?
I just have a very general question. Are there any good tutorials on using continuous normalizing flows? I'd say I have a decent understanding of normalizing flows, but not their continuous variant. I'...
1
vote
0answers
100 views
Why does the VAE using a KL-divergence with a non-standard mean does not produce good images?
I know I can make a VAE do generation with a mean of 0 and std-dev of 1.
I tested it with the following loss function:
...
3
votes
1answer
74 views
How to incorporate a symmetry constraint in the loss function to train a CNN?
I have a task of extremely sparse binary segmentation, i.e. the segmentation mask contains either 0 or 1, and there are ~95% zeros and only ~5% ones. I use the focal loss to address the sparseness (...
0
votes
0answers
18 views
How to deal with dynamically changing input tensor in neural networks without padding?
I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where:
person==...
