Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
141
questions
1
vote
1answer
18 views
How can the input order of pairs into a neural network not matter (i.e. symmetry)?
Let me explain, suppose we are building a neural network that predicts if two items are similar or not. This is a classification task with hard labels (0, 1) of examples of similar and dissimilar ...
-1
votes
0answers
30 views
What the specific dimensions in torch.Conv2D mean? [closed]
x = torch.randn(3,64,161,161)\
model = nn.Conv2d(64, 1, kernel_size=1) result = model(x)\
print(result.shape)
output : 3, 1, 161, 161
Output has the first two ...
1
vote
1answer
49 views
Why doesn't the high precision of neural network weights improve accuracy?
Consider the following paragraph from the subsubsection 3.5.2: A dtype for every occasion chapter named It starts with a tensor from the textbook titled Deep Learning with PyTorch by Eli Stevens et al....
0
votes
1answer
33 views
What does it mean by "lazy mean" here?
Consider the following paragraph, taken from 3.4: Named Tensors of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the calculation of the mean for RGB channels of an RGB ...
1
vote
0answers
20 views
How to compute the loss for a sequence labeling task without the Softmax distribution?
For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (...
0
votes
0answers
13 views
How does Stack-Augmented Recurrent Nets in work?
I am new to RNN/LSTM and I am working on a project about language modeling. I just got familiarized with simple RNN and LSTM. However, these simple models did not achieve the performance I want. Since ...
1
vote
2answers
52 views
How to make NN distinguish between two types of functions (data)?
I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict ...
0
votes
1answer
48 views
Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression
Below are the two tensors
...
1
vote
1answer
62 views
What exactly is embedding layer used in RNN encoders?
I am reading about RNN encoders. I came across the following line from this code. And I am facing difficulty in understanding the theoretical details regarding it.
...
0
votes
0answers
39 views
ignoring instances or masking by zero in multitask learning model
For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
0
votes
0answers
14 views
Proper loss function for regression with uniform target distribution
I'm doing some simulations and I would like to estimate a real number that is uniformly distributed between minValue and maxValue...
0
votes
0answers
22 views
What loss function should be used for negative log likelihood labels
I am trying to build a ranking CNN model for document - query pairs using MS Marco dataset and python pytorch. My supervisor suggested to use the same CNN to extract feature vector for document and ...
3
votes
1answer
79 views
Why do we multipy context_size with embedding_dim? (PyTorch)
I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-...
3
votes
0answers
35 views
How are partial derivatives calculated in a computational graph?
I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at ...
1
vote
1answer
53 views
What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?
In PyTorch, transformer (BERT) models have an intermediate dense layer in between attention and output layers whereas the BERT and Transformer papers just mention the attention connected directly to ...
0
votes
1answer
46 views
Multi label classification on non binary labels with pytorch
I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).
Would be the best approach to hard ...
1
vote
1answer
36 views
Is there any recommended way to perform pooling in this context?
Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
0
votes
0answers
24 views
Without using data augmentation gives results better than using data augmentation
I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images).
I am doing transfer learning (pre-trained ...
1
vote
1answer
180 views
What exactly happens in gradient clipping by norm?
Consider the following description regarding gradient clipping in PyTorch
...
0
votes
0answers
28 views
specifying a hybrid Bayesian network in pyro
I am trying to learn about Bayesian networks and am really having a hard time to figure out how to setup some simple models.
Say, I have a model as:
...
0
votes
1answer
40 views
How to pass multiple vectors and numeric features as input to the neural network?
I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
0
votes
0answers
37 views
How to increase accuracy of image orientation classification (Left, Right, Center)?
I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this:
The images are "Left", "...
0
votes
0answers
26 views
Joined vs Separate optimizer for Actor-Critic
Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
0
votes
1answer
58 views
Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]
I have trained a classification network with PyTorch lightning where my training step looks like below:
...
0
votes
0answers
27 views
Is it normal that we get different AUC results after running with various seeds?
We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch.
We saw that choosing ...
1
vote
0answers
31 views
Are there any benefits of adding attention to linear layers?
Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
0
votes
1answer
57 views
How do CNNs handle inputs of different sizes and shapes?
I am new to deep learning so feel free to correct me where I am wrong.
Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
-1
votes
1answer
41 views
How to re-training an AI model to have smaller input image size
I need a PyTorch Model which can do road segmentation on OAK-D camera.
The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
0
votes
1answer
130 views
Deep Q-Learning with multiple discrete actions
I am working on a DQN project with Pytorch, where I should choose multiple discrete actions, each in a range, say, (0, 15). I am wondering how I can model it, such ...
1
vote
1answer
41 views
Is this calculation of the vector-Jacobian product in the PyTorch documention wrong?
In the official PyTorch documentation there is the following calculation (here):
$$
J^{T} \cdot \vec{v}=\left(\begin{array}{ccc}
\frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial ...
0
votes
1answer
70 views
Is "kernel" different from "filter" in convolutional neural networks?
Recently I asked a question on how a convolution 2d layer changes an RGB image into a grayscale image. Assume that our task is to convert an RGB image into a grayscale image. I use to believe that ...
2
votes
0answers
28 views
What is an "additional channel dimension" contain in batch normalization?
Consider the following explanations regarding batch normalization layers in PyTorch
#1: one dimensional batch normalization
class torch.nn.BatchNorm1d(.........)
Applies Batch Normalization over a 2D ...
2
votes
0answers
57 views
Positional Encoding in Transformer on multi-variate time series data hurts performance
I set up a transformer model that embeds positional encodings in the encoder. The data is multi-variate time series-based data.
As I just experiment with the positional encoding portion of the code I ...
0
votes
0answers
21 views
Feeding the output back to input in 3D CNN model
I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below
I have two thoughts in training the model for above situation.
Feed output $...
1
vote
0answers
47 views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
1
vote
1answer
32 views
Are there any stats available on the usage of libraries by deep learning researchers?
I know three Python libraries that are popular in deep learning research community: Keras, PyTorch, Tensorflow. I don't know much about Theano.
This question is not about the efficiency, flexibility ...
1
vote
0answers
192 views
What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch
PyTorch provides max pooling and adaptive max pooling.
Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this ...
2
votes
0answers
25 views
Is there any closed form analytical expression to represent fractional max pooling?
There are Nineteen types of pooling layers in PyTorch.
Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
1
vote
0answers
41 views
Is there any animation that illustrates the "fold" and "unfold" operations of convolutional layers?
There are fourteen convolution layers in PyTorch. Among them six are related to convolution, another six are related to transposed convolution. The remaining two are fold and unfold operations.
The ...
1
vote
0answers
107 views
Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?
The basic layers for performing convolution operations 1,2,3 in PyTorch are
...
1
vote
1answer
60 views
What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?
PyTorch documentation provided the following descriptions to the Convolution layers
...
2
votes
1answer
135 views
What exactly is an XPU?
I know about CPU, GPU and TPU. But, it is the first time for me to read about XPU from PyTorch documentation about MODULE.
...
0
votes
0answers
79 views
What are the input and output gradients in PyTorch?
Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$.
Let the number of ...
0
votes
1answer
61 views
What are the applications in which the precision of the neural network's weights is unimportant?
While reading about Module in PyTorch, I came across a new data type called half datatype.
half() method when calls on a Module ...
0
votes
0answers
54 views
What are the numbers that are useful (may need to be stored) other than parameters of a model?
Consider the following method related to buffers in PyTorch
...
1
vote
1answer
43 views
What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?
Consider the following PyTorch code
...
3
votes
2answers
120 views
Is there any difference between affine transformation and linear transformation?
Consider the following statements from A Simple Custom Module of PyTorch's documentation
To get started, let’s look at a simpler, custom version of PyTorch’s
Linear module. This module applies an ...
0
votes
1answer
50 views
How does CURL extract labels from logits? [closed]
While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
0
votes
0answers
20 views
A2C value function outputs keep increasing
I was implementing the A2C algorithm with as close to baseline setup as possible, and this is the code I came up with. The problem is that even after multiple rechecks, the program isn't showing ...
0
votes
0answers
24 views
How to obtain STD from Neural Network with 2 continuous action output
In my Environment, I have two continuous action space
self.action_space = spaces.Box(low=np.array([0.,0.]), high=np.array([4.,0.02]), shape=(2,), dtype=np.float32)
...
