Questions tagged [pytorch]

For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.

Filter by
Sorted by
Tagged with
0
votes
1answer
79 views

Transformer Language Model generating meaningless text

I currently learning on Transformers, so check my understanding I tried implementing a small transformer-based language model and compare it to RNN based language model. Here's the code for ...
0
votes
0answers
13 views

How to install IBM Analog Hardware Acceleration Kit [closed]

As part of a project I'm working on for my studies, we want to use the "Analog Hardware Acceleration Kit" you developed. We're having trouble installing the kit properly, despite following ...
0
votes
0answers
18 views

Joined vs Separate optimizer for Actor-Critic

Say that I have a simple Actor-Critic architecture, (I am not familiar with Tensorflow, but) in Pytorch we need to specify the parameters when defining an optimizer (SGD, Adam, etc) and therefore we ...
2
votes
1answer
92 views

What is the difference between FC and MLP in as used in PointNet?

I am trying to understand the PointNet network for dealing with point clouds and struggling with understanding the difference between FC and MLP: "FC is fully connected layer operating on each ...
1
vote
1answer
56 views

PyTorch `torch.no_grad` vs `torch.inference_mode` [closed]

PyTorch has new functionality torch.inference_mode as of v1.9 which is "analogous to torch.no_grad... Code run under this ...
0
votes
0answers
18 views

Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images). I am doing transfer learning (pre-trained ...
1
vote
1answer
32 views

Is there any recommended way to perform pooling in this context?

Suppose I have three batches of feature maps, each of size $180 \times 100 \times 100$. I want to concatenate all these feature maps channel-wise, and then resize them into a single feature map. The ...
0
votes
1answer
71 views

Deep Q-Learning with multiple discrete actions

I am working on a DQN project with Pytorch, where I should choose multiple discrete actions, each in a range, say, (0, 15). I am wondering how I can model it, such ...
1
vote
1answer
102 views

What exactly happens in gradient clipping by norm?

Consider the following description regarding gradient clipping in PyTorch ...
0
votes
1answer
45 views

Why would the reward of A3C with LSTM suddenly drop off after many episodes?

I am training an A3C with stacked LSTM. During initial training, my model was giving descent +ve reward. However, after many episodes, its reward just goes to zero and is continuing for a long time. ...
0
votes
0answers
15 views

specifying a hybrid Bayesian network in pyro

I am trying to learn about Bayesian networks and am really having a hard time to figure out how to setup some simple models. Say, I have a model as: ...
1
vote
1answer
143 views

RealNVP gives wrong probabilities

I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long)...
0
votes
1answer
29 views

How to pass multiple vectors and numeric features as input to the neural network?

I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
0
votes
1answer
58 views

Is "kernel" different from "filter" in convolutional neural networks?

Recently I asked a question on how a convolution 2d layer changes an RGB image into a grayscale image. Assume that our task is to convert an RGB image into a grayscale image. I use to believe that ...
2
votes
0answers
23 views

Is there any closed form analytical expression to represent fractional max pooling?

There are Nineteen types of pooling layers in PyTorch. Almost all of the layers are provided with corresponding analytical formulae. But analytical formulae are not provided for the fractional max-...
1
vote
0answers
67 views

What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch

PyTorch provides max pooling and adaptive max pooling. Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this ...
0
votes
1answer
234 views

Feeding YOLOv4 image data into LSTM layer?

How would one extract the feature vector from a given input image using YOLOv4 and pass that data into an LSTM to generate captions for the image? I am trying to make an image captioning software in ...
7
votes
1answer
3k views

Can neural networks be evolved using NEAT in TensorFlow? [closed]

I am making a machine learning program for time series data analysis, and using NEAT could help the work. I started to learn TensorFlow not long ago, but it seems that the computational graphs in ...
0
votes
1answer
33 views

How to re-training an AI model to have smaller input image size

I need a PyTorch Model which can do road segmentation on OAK-D camera. The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
0
votes
0answers
33 views

How to increase accuracy of image orientation classification (Left, Right, Center)?

I am working on classifying images in "Left", "Right", "Center", "Back". Training and Validation images look like this: The images are "Left", "...
0
votes
1answer
31 views

Is it normal that the values of the LogSoftmax function are very large negative numbers? [closed]

I have trained a classification network with PyTorch lightning where my training step looks like below: ...
2
votes
1answer
101 views

Policy Gradient on Tic-Tac-Toe not working

I wanted to implement the Policy Gradient on Tic-Tac-Toe. I tried to use the code that worked for any environment like CartPole-v0 to my Tic-Tac-To game. But it is not learning. There are no errors. ...
0
votes
0answers
25 views

Is it normal that we get different AUC results after running with various seeds?

We are working on optimizing a CNN made for binary image classification (by that I mean to classify each image to group A or group B). It is based on InceptionV3, using PyTorch. We saw that choosing ...
0
votes
1answer
31 views

My weights for binary classification are not getting updated [closed]

I am very new to this pytorch and neural networks.I am stuck in training one model since last 1 week. My model paramters are not getting updated after each epoch. Also,...
1
vote
0answers
26 views

Are there any benefits of adding attention to linear layers?

Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
0
votes
1answer
152 views

Can you train Transformers sequentially?

I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code ...
0
votes
1answer
44 views

How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
1
vote
1answer
28 views

Are there any stats available on the usage of libraries by deep learning researchers?

I know three Python libraries that are popular in deep learning research community: Keras, PyTorch, Tensorflow. I don't know much about Theano. This question is not about the efficiency, flexibility ...
0
votes
1answer
40 views

Is precision of weights unimportant in neural networks?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
1
vote
0answers
16 views

Non-determinism with mixed precision?

Currently, we're trying to improve failure analysis capability when using neural nets. One thing we want to resolve is output variation between batched runs and non-batched runs. For example, we wrote ...
1
vote
1answer
34 views

Is this calculation of the vector-Jacobian product in the PyTorch documention wrong?

In the official PyTorch documentation there is the following calculation (here): $$ J^{T} \cdot \vec{v}=\left(\begin{array}{ccc} \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial ...
1
vote
0answers
17 views

What is an "additional channel dimension" contain in batch normalization?

Consider the following explanations regarding a batch normalization layers in PyTorch #1: one dimensional batch normalization class torch.nn.BatchNorm1d(.........) Applies Batch Normalization over a ...
0
votes
0answers
46 views

What are the numbers thar are useful (may need to be stored) other than parameters of a model?

Consider the following method related to buffers in PyTorch ...
1
vote
0answers
46 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
1
vote
1answer
63 views

What exactly is an XPU?

I know about CPU, GPU and TPU. But, it is the first time for me to read about XPU from PyTorch documentation about MODULE. ...
0
votes
0answers
22 views

Positional Encoding in Transformer on multi-variate time series data hurts performance

I set up a transformer model that embeds positional encodings in the encoder. The data is multi-variate time series-based data. As I just experiment with the positional encoding portion of the code I ...
0
votes
0answers
13 views

Feeding the output back to input in 3D CNN model

I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below I have two thoughts in training the model for above situation. Feed output $...
1
vote
0answers
29 views

Is there any animation that illustrates the "fold" and "unfold" operations of convolutional layers?

There are fourteen convolution layers in PyTorch. Among them six are related to convolution, another six are related to transposed convolution. The remaining two are fold and unfold operations. The ...
1
vote
1answer
34 views

What does 'input planes' mean in the phrase 'input signal/image composed of several input planes'?

PyTorch documentation provided the following descriptions to the Convolution layers ...
1
vote
0answers
40 views

Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?

The basic layers for performing convolution operation in PyTorch are ...
0
votes
0answers
35 views

What are the input and output gradients in PyTorch?

Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$. Let the number of ...
1
vote
1answer
106 views

How to properly optimize shared network between actor and critic?

I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment. When I share the encoder with the actor ...
3
votes
2answers
100 views

Is there any difference between affine transformation and linear transformation?

Consider the following statements from A Simple Custom Module of PyTorch's documentation To get started, let’s look at a simpler, custom version of PyTorch’s Linear module. This module applies an ...
0
votes
1answer
46 views

What are the labels of EMNIST?

I'm using pytorch and downloading the EMNIST dataset using ...
1
vote
1answer
66 views

time-series prediction : loss going down, then stagnates with very high variance

I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and ...
1
vote
3answers
66 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
2
votes
0answers
56 views

How will the filter size affect the transpose convolution operation?

After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling. How will the ...
0
votes
1answer
42 views

How does CURL extract labels from logits?

While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
0
votes
0answers
14 views

A2C value function outputs keep increasing

I was implementing the A2C algorithm with as close to baseline setup as possible, and this is the code I came up with. The problem is that even after multiple rechecks, the program isn't showing ...