Questions tagged [pytorch]
For conceptual questions that somehow involve the PyTorch library, but note that programming questions are off-topic here.
111
questions
6
votes
1answer
43 views
When should you not use the bias in a layer?
I'm not really that experienced with deep learning, and I've been looking at research code (mostly PyTorch) for deep neural networks, specifically GANs, and, in many cases, I see the authors setting <...
0
votes
0answers
12 views
Things to consider while adding custom function to generator output in GAN
I am training a GAN model (DCGAN) to generate 128x128 images. Now, I wish to add a function which will take the generator output, perform some pre-defined operations on it, and return the modified ...
-1
votes
0answers
15 views
Pytorch YOLOv4 - Getting low mAP and IoU results [closed]
I am new to computer vision and object detection, and I am using YOLOv4 in Pytorch for the object detection. My end goal is to get my mAP to 60-70% and my IoU to 80-90%. I am trying to detect roadside ...
0
votes
0answers
16 views
Adversarial Attacks and interpolation methods
I am attacking a model. The model is a simple CNN and PGD is used. The model runs on 112x112 ImageNet dataset. So I first load images as 224x224 and use interpolation function to downsample it to ...
0
votes
0answers
16 views
Which is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?
Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch.
...
0
votes
0answers
16 views
Can anybody just confirm whether or not my understanding of depthwise separable convolutions is correct?
I just need a simple Yes/No confirmation or to debunk my understanding of the difference between the normal convolutions and depthwise seperable convs. I have read quite a few articles and watched a ...
1
vote
2answers
50 views
CNN based model poor result
My goal is to train and evaluate The German Traffic Sign Recognition Benchmark (GTSRB) dataset using Pytorch.
I downloaded the datasets from the official site GTSRB_Final_Training_Images.zip and ...
0
votes
0answers
13 views
Meaning of grad_outputs in torch.autograd.grad for complex input and output
Let's say we have a mathematical expression,
$$
\mathbf{y} = \mathbf{Ax},
$$
where $\mathbf{y}$ and $\mathbf{x}$ are a vector, and $\mathbf{A}$ is a matrix.
Let's say the vector $\mathbf{y}$ is used ...
0
votes
0answers
17 views
PyTorch: LSTM error while trying to update the hidden state
I am trying to train an LSTM while keeping its hidden state (LSTM stateful) until the moment when I am going to start a new epoch(episode). But here it's come an interesting situation because I am ...
0
votes
0answers
26 views
PyTorch: How to deal with hidden states of an LSTM?
I have a time series in which each date is correlated with the preview one, and base on that I am trying to predict action 1 and action 2. But the problem is that I am not sure how to deal with the <...
-3
votes
1answer
77 views
0
votes
2answers
34 views
Does anybody know what would happen if I changed input shape of pytorch models?
In this https://pytorch.org/vision/stable/models.html tutorial it clearly states:
All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of ...
0
votes
0answers
17 views
Why (not) using pre-processing before using Transformer models?
Regarding the use of pre-processing techniques before using Transformers models, I read this post that apparently says that these measures are not so necessary nor interfere so much in the final ...
0
votes
0answers
15 views
Mixed precision training - why we're fine with doing point wise operations in FP32
I'm starting to learn more about mixed-precision training, and I'm in particular confused about point-wise operations. In the original article (link), the authors mention, citing:
Point-wise ...
1
vote
2answers
46 views
How can I model any structure for a neural network?
Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
0
votes
0answers
15 views
Is there an effective way of obtaining the topic distribution for a given document from a VAE-LDA?
Is there an effective way of obtaining the topic distribution for a given document from a Variational AutoEncoder Latent Dirichlet Allocation (VAE-LDA)?
Most existing public VAE-LDA codebases seem to ...
0
votes
0answers
22 views
How is a tensor implemented? How does it adjust for covariant/contravariant?
This stackoverflow question answers how tensors represent data, but not how they adjust for covariance/contravariance.
I'm interested in understanding how the coupling of tensors with each other is ...
1
vote
0answers
45 views
What's the best way to take a list of lists as DQN input?
I have my own environment for the DQN algorithm. In my environment, the state space is represented by a list of lists, where each sublist can be of different lengths. In my case, the length of the ...
0
votes
0answers
32 views
Are there any good tutorials on using continuous normalizing flows (with PyTorch)?
I just have a very general question. Are there any good tutorials on using continuous normalizing flows? I'd say I have a decent understanding of normalizing flows, but not their continuous variant. I'...
1
vote
0answers
84 views
Why does the VAE using a KL-divergence with a non-standard mean does not produce good images?
I know I can make a VAE do generation with a mean of 0 and std-dev of 1.
I tested it with the following loss function:
...
3
votes
1answer
70 views
How to incorporate a symmetry constraint in the loss function to train a CNN?
I have a task of extremely sparse binary segmentation, i.e. the segmentation mask contains either 0 or 1, and there are ~95% zeros and only ~5% ones. I use the focal loss to address the sparseness (...
0
votes
0answers
18 views
How to deal with dynamically changing input tensor in neural networks without padding?
I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where:
person==...
0
votes
0answers
40 views
Conversion of convolution filter weights between NCHW and NHWC formats
Popular deep learning frameworks have different default data formats (PyTorch with NCHW, TensorFlow with NHWC), which lead to published pretrained models with weights that may have an incompatible ...
0
votes
0answers
15 views
feeding a NN with tensors with varying spatial dimensions
I have a huge dataset where I have a tensor with 535 channels but varying spatial dimension (but always a square) it can vary from ...
2
votes
1answer
46 views
Is it possible to train one part of the network with a particular learning rate and the other part with a different one?
I have a combined network consisting of two parts: one is for images and the other is for numerical data. Each sample is matched with a numerical case by an ID. For this combined network, a ...
1
vote
0answers
28 views
variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas
My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data.
By "fails" I mean there are at least two apparent problems:
Very poor reconstruction, for ...
2
votes
1answer
62 views
Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?
Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories. Can existing open source frameworks ...
0
votes
1answer
190 views
Transformers: How to use the target mask properly?
I try to apply Transformers to an unusual use case - predict the next user session based on the previous one. A user session is described by a list of events per second, e.g. whether the user watches ...
2
votes
0answers
24 views
How will the filter size affect the transpose convolution operation?
After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling.
How will the ...
0
votes
1answer
70 views
How to use a conv2d layer after a flatten?
I am not familiar with Deep learning and Pytorch. And I want to know how to deal, in general with such a situation. So, I was wondering if I used a pretrained model (EfficientNet for example) if I ...
0
votes
0answers
31 views
Training Object Detection model on just 10 images
I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
0
votes
0answers
23 views
Transformer Language model produces only <pad> tokens when generating new sentences
I am training a word-level language model using the transformer module available in Pytorch. I am getting a really good training loss and the model is able to reproduce the sentences in the training ...
0
votes
0answers
33 views
Pytorch and keras ddqn seem identical, only keras learns
I followed a tutorial for ddqn to beat pong, it beats it with a perfect score in keras, but trying to translate it to pytorch it doesn't learn at all. What am I missing? I pasted all the code for each ...
0
votes
1answer
50 views
How to use validation dataset in my logistic regression model?
I am new to machine learning and recently I joined a course where I was given a logistic regression assignment in which I had to split 20% of the training dataset for the validation dataset and then ...
0
votes
0answers
64 views
Multi-label dataloading bottleneck Pytorch
I am trying to write custom dataset and dataloader for pascal-voc-2007. It is a multi-label classification problem. There is csv file to hold the name of the images and their corresponding labels. I ...
0
votes
0answers
61 views
Transformer Language Model generating meaningless text
I currently learning on Transformers, so check my understanding I tried implementing a small transformer-based language model and compare it to RNN based language model. Here's the code for ...
0
votes
1answer
60 views
How to implement or avoid masking for transformer?
When it comes to using Transformers for image captioning is there any reason to use masking?
I currently have a resnet101 encoder and am trying to use the features as the input for a transformer model ...
2
votes
3answers
69 views
DQN not learning and step not stepping towards target
I am trying to create a simple Deep Q-Network with 2d convolutional layers.
I can't figure out what I am doing wrong, and the only thing I can see that doesn't seem right is when I get the model ...
0
votes
1answer
35 views
Why would the reward of A3C with LSTM suddenly drop off after many episodes?
I am training an A3C with stacked LSTM.
During initial training, my model was giving descent +ve reward. However, after many episodes, its reward just goes to zero and is continuing for a long time. ...
0
votes
0answers
16 views
How to use an image tensor for caption generation with Transformer-XL or BERT?
I am fairly new to transformers and deep learning in general so please be kind,
I am currently working on a project that will caption images using either Transformer-XL or BERT, however, I am not sure ...
1
vote
0answers
15 views
is it ok to take random actions while training a3c as in below code
i am trying to train an A3C algorithm but I am getting same output in the multinomial function.
can I train the A3C with random actions as in below code.
can someone expert comment.
...
0
votes
1answer
153 views
Feeding YOLOv4 image data into LSTM layer?
How would one extract the feature vector from a given input image using YOLOv4 and pass that data into an LSTM to generate captions for the image?
I am trying to make an image captioning software in ...
0
votes
0answers
56 views
Class activation maps for 3D Convolutional neural network?
I have implemented a 3D convolutional neural network and I was not able to find resources for interpretation for my model. I have found some techniques such as GradCam and GradCam++ but these generate ...
0
votes
0answers
47 views
Why does my loss value of autoencoder in PyTorch is negative?
I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension.
The idea is kind of simple, SDNE uses the ...
0
votes
1answer
28 views
Get Neural Network to predict a tag/class on a certain word using the surrounding words as context [PyTorch]?
I am somewhat a novice at the topic of Neural Netoworks and PyTorch.
I am trying to create a model that takes a word (that I have modified very slightly) and a 'window' of context around it and ...
0
votes
0answers
23 views
Automated Scoring (non-english language) Using BERT
i'm a student and i'm new to NLP.
I want to build an Automated Scoring system which is in Indonesian Language using BERT. The system is expected to be able to measure the similarity of an answer(e.g: ...
0
votes
0answers
23 views
Descent Training episodes for LSTM + TD3
I am building an AI with TD3 and lstm in both actor and critic.
By LSTM size is 5,5 with 3 layers and hidden layers with 400 and 300 neurons respectively.
I have states dimension of 5 with each value ...
0
votes
0answers
26 views
TD3+lstm predicting the same output for varying states
I have a model with TD3 + lstm in both actor and critic.
I am trying to make it learn to predict some specific actions based on the environment conditions.
However i see that the AI predicts very ...
0
votes
1answer
82 views
Is it possible to have a fixed trajectory size in the vanilla policy gradient algorithm?
In the concept of the vanilla policy gradient algorithm, is it possible for our trajectory size to be fixed?
For example, my environment is the space of embedded images (using a pre-trained encoder to ...
1
vote
0answers
69 views
Is it good practice to save NLP Transformer based pre-trained models into file system in production environment
I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers.
I have saved the pretrained model into the file directory in dev environment. ...
