Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
148
questions
0
votes
0answers
11 views
How to install IBM Analog Hardware Acceleration Kit
As part of a project I'm working on for my studies, we want to use the "Analog Hardware Acceleration Kit" you developed. We're having trouble installing the kit properly, despite following ...
1
vote
1answer
55 views
PyTorch `torch.no_grad` vs `torch.inference_mode` [closed]
PyTorch has new functionality torch.inference_mode as of v1.9 which is "analogous to torch.no_grad... Code run under this ...
1
vote
1answer
32 views
Is there any recommended way to perform pooling in this context?
Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
0
votes
0answers
17 views
Without using data augmentation gives results better than using data augmentation
I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images).
I am doing transfer learning (pre-trained ...
1
vote
1answer
102 views
What exactly happens in gradient clipping by norm?
Consider the following description regarding gradient clipping in PyTorch
...
0
votes
0answers
15 views
specifying a hybrid Bayesian network in pyro
I am trying to learn about Bayesian networks and am really having a hard time to figure out how to setup some simple models.
Say, I have a model as:
...
0
votes
1answer
29 views
How to pass multiple vectors and numeric features as input to the neural network?
I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
0
votes
0answers
33 views
How to increase accuracy of image orientation classification (Left, Right, Center)?
I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this:
The images are "Left", "...
0
votes
0answers
18 views
Joined vs Separate optimizer for Actor-Critic
Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
0
votes
1answer
30 views
Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]
I have trained a classification network with PyTorch lightning where my training step looks like below:
...
0
votes
0answers
25 views
Is it normal that we get different AUC results after running with various seeds?
We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch.
We saw that choosing ...
1
vote
0answers
26 views
Are there any benefits of adding attention to linear layers?
Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
0
votes
1answer
31 views
My weights for binary classification are not getting updated [closed]
I am very new to this pytorch and neural networks.I am stuck in training one model since last 1 week. My model paramters are not getting updated after each epoch. Also,...
0
votes
1answer
44 views
How do CNNs handle inputs of different sizes and shapes?
I am new to deep learning so feel free to correct me where I am wrong.
Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
0
votes
1answer
33 views
How to re-training an AI model to have smaller input image size
I need a PyTorch Model which can do road segmentation on OAK-D camera.
The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
0
votes
1answer
71 views
Deep Q-Learning with multiple discrete actions
I am working on a DQN project with Pytorch, where I should choose multiple discrete actions, each in a range, say, (0, 15). I am wondering how I can model it, such ...
1
vote
0answers
16 views
Non-determinism with mixed precision?
Currently, we're trying to improve failure analysis capability when using neural nets. One thing we want to resolve is output variation between batched runs and non-batched runs.
For example, we wrote ...
1
vote
1answer
34 views
Is this calculation of the vector-Jacobian product in the PyTorch documention wrong?
In the official PyTorch documentation there is the following calculation (here):
$$
J^{T} \cdot \vec{v}=\left(\begin{array}{ccc}
\frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial ...
0
votes
1answer
57 views
Is "kernel" different from "filter" in convolutional neural networks?
Recently I asked a question on how a convolution 2d layer changes an RGB image into a grayscale image. Assume that our task is to convert an RGB image into a grayscale image. I use to believe that ...
1
vote
0answers
17 views
What is an "additional channel dimension" contain in batch normalization?
Consider the following explanations regarding a batch normalization layers in PyTorch
#1: one dimensional batch normalization
class torch.nn.BatchNorm1d(.........)
Applies Batch Normalization over a ...
0
votes
0answers
21 views
Positional Encoding in Transformer on multi-variate time series data hurts performance
I set up a transformer model that embeds positional encodings in the encoder. The data is multi-variate time series-based data.
As I just experiment with the positional encoding portion of the code I ...
0
votes
0answers
13 views
Feeding the output back to input in 3D CNN model
I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below
I have two thoughts in training the model for above situation.
Feed output $...
1
vote
0answers
46 views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
1
vote
1answer
28 views
Are there any stats available on the usage of libraries by deep learning researchers?
I know three Python libraries that are popular in deep learning research community: Keras, PyTorch, Tensorflow. I don't know much about Theano.
This question is not about the efficiency, flexibility ...
1
vote
0answers
65 views
What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch
PyTorch provides max pooling and adaptive max pooling.
Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this ...
2
votes
0answers
23 views
Is there any closed form analytical expression to represent fractional max pooling?
There are Nineteen types of pooling layers in PyTorch.
Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
1
vote
0answers
29 views
Is there any animation that illustrates the "fold" and "unfold" operations of convolutional layers?
There are fourteen convolution layers in PyTorch. Among them six are related to convolution, another six are related to transposed convolution. The remaining two are fold and unfold operations.
The ...
1
vote
0answers
40 views
Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?
The basic layers for performing convolution operation in PyTorch are
...
1
vote
1answer
34 views
What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?
PyTorch documentation provided the following descriptions to the Convolution layers
...
1
vote
1answer
62 views
What exactly is an XPU?
I know about CPU, GPU and TPU. But, it is the first time for me to read about XPU from PyTorch documentation about MODULE.
...
0
votes
0answers
35 views
What are the input and output gradients in PyTorch?
Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$.
Let the number of ...
0
votes
1answer
40 views
Is precision of weights unimportant in neural networks?
While reading about Module in PyTorch, I came across a new data type called half datatype.
half() method when calls on a Module ...
0
votes
0answers
46 views
What are the numbers thar are useful (may need to be stored) other than parameters of a model?
Consider the following method related to buffers in PyTorch
...
1
vote
1answer
41 views
What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?
Consider the following PyTorch code
...
3
votes
2answers
100 views
Is there any difference between affine transformation and linear transformation?
Consider the following statements from A Simple Custom Module of PyTorch's documentation
To get started, let’s look at a simpler, custom version of PyTorch’s
Linear module. This module applies an ...
0
votes
1answer
45 views
0
votes
1answer
42 views
How does CURL extract labels from logits?
While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
0
votes
0answers
14 views
A2C value function outputs keep increasing
I was implementing the A2C algorithm with as close to baseline setup as possible, and this is the code I came up with. The problem is that even after multiple rechecks, the program isn't showing ...
0
votes
0answers
19 views
How to obtain STD from Neural Network with 2 continuous action output
In my Environment, I have two continuous action space
self.action_space = spaces.Box(low=np.array([0.,0.]), high=np.array([4.,0.02]), shape=(2,), dtype=np.float32)
...
0
votes
0answers
30 views
GAN performs worse after 50 epochs than after 2
I am training GAN on SVHN dataset (house numbers in Google Street View images, dimensions: 3x32x32 - 3 color channels).
The problem is that it performs worse after some training (e.g. after 50 epochs) ...
1
vote
0answers
7 views
How can I get the images with the highest activation for a given unit?
I am new to machine learning. I am working on the pretrained AlexNet on Pytorch and i would like to visualize the receptive fields of a given unit U. To do that I am trying to give like 200K images as ...
1
vote
0answers
51 views
How can I add a Sequential CNN layer on top of BERT model?
Information
I'm working on a binary classification task and used BERT model from transformers library to do it using the custom model below:
...
0
votes
1answer
26 views
Why do I have better RMSE when I don't scale the target? [closed]
I use PyTorch for training a simple neural net for a regression task on a dataset with 12 numerical features + target (target is the 13th column) + 2 categorical features
Before training, I execute
<...
0
votes
1answer
22 views
Changing a CNN-LSTM image captioning architecture to use BiLSTMs
Currently I'm dealing with an assignment that made us implement the network mentioned in this paper. The network has an architecture similar to this:
As you can see it uses a Unidirectional RNN (in ...
0
votes
0answers
17 views
Neural Networks different architectures but similar training curves
I have a base neural network architecture for (3D) image sequences classification, made of conv layers followed by a LSTM and dense layers.
I have 3 similar architectures :
3 Conv -> 1 LSTM -> ...
0
votes
0answers
12 views
Visualizing encoder-attention after ResNet in terms of ResNet input
I have a transform-encoder only architecture, which has the following structure:
...
0
votes
0answers
22 views
Why does my model only predict 1s or 0s in multiclass segmentation?
I am currently trying to train a UNet model on the dronedeploy segmentation dataset. My problem is that I only get outputs like this from my model:
This means that in the output maps I only get 0s ...
7
votes
1answer
154 views
When should you not use the bias in a layer?
I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
0
votes
0answers
13 views
Things to consider while adding custom function to generator output in GAN
I am training a GAN model (DCGAN) to generate 128x128 images. Now, I wish to add a function which will take the generator output, perform some pre-defined operations on it, and return the modified ...
0
votes
0answers
18 views
Adversarial Attacks and interpolation methods
I am attacking a model. The model is a simple CNN and PGD is used. The model runs on 112x112 ImageNet dataset. So I first load images as 224x224 and use interpolation function to downsample it to ...
