Questions tagged [deep-learning]
For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".
1,979
questions
1
vote
1
answer
30
views
Which one to choose from Pruning, Quantization , Distillation and others when optimizing speed
I want to understand how to optimize models for inference speed and am seeking some advice and best practices for the same.
I am a little bit aware of the concepts of pruning, quantization, and ...
- 577
1
vote
1
answer
33
views
Is Softmax Necessary as the Activation Function for Self-Attention Mechanisms?
I’m curious about the mathematical reasoning behind the use of the softmax function as the activation function in self-attention mechanisms within neural networks. Specifically, I’m interested in ...
- 303
0
votes
0
answers
12
views
Neural network dynamic input shape vs fixed input shape in input layer for handling NULL values
I have binary classification problem, we know that the output layer will be scalar or dense a.k.a. 1 unit neuron with sigmoid as function activation. 1 means is the subject will die, while 0 means ...
1
vote
0
answers
34
views
Do diffusion models take a long time to train?
I am trying to train a diffusion model (from scratch in pytorch). UNet used is not anything too fancy, takes in images and time step as input for about 512 time steps. I am using learnable embeddings ...
- 11
0
votes
0
answers
13
views
Classical architecture of autoencoder neural networks
I want to try Autoencoders(AE) (not variational) with a PINNs. The AE will be to encode the points of the domain. Lets say the variable domain (time-space + parameter space) dimension is n where n is ...
- 1
-1
votes
0
answers
26
views
cause and effect [closed]
I watched a tutorial video on AI. The author stated these:
1: "Any device that can perceive its environment and take action accordingly has AI."
2: "AI understands the cause and ...
- 97
0
votes
0
answers
20
views
How to let the AI trained on MNIST recognize my own hand-written number?
I have already trained a simple model without using any frameworks to recognize the hand-written numbers from the MNIST dataset. Actually it's just a copy of the example on the book Grokking ...
1
vote
1
answer
449
views
Which epoch is the best for me to choose?
I have trained my deep learning model. I also saved the validation loss to a file and plotted on a graph
I have $2$ questions for this:
Does the validation loss look normal? Is there any issue with ...
user78437
0
votes
0
answers
13
views
i need dataset for human activity and diseases from smartphone accelerometer sensor? [closed]
project aims to develop a health monitoring system utilizing smartphone accelerometer sensors and machine learning techniques. The primary focus is on detecting specific health conditions through the ...
0
votes
0
answers
24
views
Prefix tuning in LLM uses learnable vectors to fine tune the model
I would like to implement a new architecture for Transformer.
Below description is my thought.
Prefix tuning in LLM uses learnable vectors to fine tune the model.
Is there a way to use the output ...
- 9
1
vote
1
answer
66
views
Do GANs have constant running time?
After the model is trained, you just need to input random noise and the generator will output an image, does this mean GANs have constant running time ? I'm asking about both naïve GAN and variants of ...
user78202
1
vote
0
answers
123
views
Question about the Conditioning Augmentation technique?
In the paper StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, the goal is to convert text descriptions into images. The text encoder encodes the ...
user78202
0
votes
1
answer
64
views
How to detect negative (absence of) an object?
I want to detect the people that are NOT wearing PPE vests using a pre-trained object detection model like YOLO or Grounding Dino.
The models are able to detect people and vests separately, but I am ...
0
votes
2
answers
45
views
how to determine the number of units for dense layer for transfer learning?
I'm using MobileNetV2 for classification, and I want to add dense layers(i remove the last layer of the MobileNetV2 model). How do I choose the number of units for the dense layer after obtaining the ...
- 1
0
votes
0
answers
19
views
Can't get a correct accuracy on tabular data using deep learning
This is my first message here, and I would like to seek some assistance !
I have a technical test for a job that I really want, and I have 10 days to complete it. I've attempted to work on it, but I'm ...
1
vote
2
answers
159
views
The training process of a conditional GAN
For example, consider a dataset like MNIST. I give the conditional vector to produce only the number $7$ for both the generator and discriminator. In the following scenarios, what will the ...
- 15
-1
votes
0
answers
39
views
Diffusion model for image to image translation
I am interested in using the diffusion model for image-to-image translation (pair images). I used this repository for semantic image synthesis via diffusion models Code. How can I use it for image-to-...
- 1
2
votes
2
answers
176
views
Can mini-batches for stochastic gradient be balanced but not representative of the training data?
When we construct mini-batches for stochastic gradient, it is important to ensure that the different mini-batches are balanced (for example, in case of classification they contain the same ratio for ...
- 493
0
votes
3
answers
59
views
My model is only improving when learning rate is 1. Should I be worried?
As the title says my GNN with three layers of GAT (Graph attention layers) is only moving the metrics when the learning rate is 1. As generally the learning rate is (0,1) should I be worried?
Also ...
- 1
-2
votes
1
answer
53
views
the best choice to reduce a features vector
i have 1200 features highly correlated , and i want to reduce those number of features
so the best choice is use feature selection or dimensionality reduction? and which method is the best in this ...
0
votes
1
answer
54
views
Are foundation models something fundamentally new? Is there a proper definition?
Currently, one can hear more and more about "foundation models" but details of this are not always clear. Also, I even have the impression, that sometimes people don't talk about exactly ...
- 156
0
votes
1
answer
47
views
What is the potential issue of nested neural networks
everyone. I am working on a nested neural network architecture. For the sake of better understanding my question, simply assume the loss is
$L = G(k’) - H(k'')$
where $G$ and $H$ are two functions we ...
0
votes
0
answers
15
views
Keeping the grad of the tensor when using inverse_transform
To train a network, I scaled both the input and outputs of my data like the following:
...
- 1
0
votes
0
answers
11
views
Why my deep learning model (FCNN/ 1DCNN) fails to learn when training on medical dataset?
I am working on a project to predict the severity of the disease, Hemophilia using a deep learning model(FCNN or 1DCNN). I am working based on the information provided in this article: https://www....
- 1
0
votes
1
answer
117
views
In the conditional GAN (cGAN) architecture, why does the discriminator need conditional variable?
I'm reading about conditional GAN (cGAN) architecture, what I know is that the generator creates images combining both noise vector and conditional variable, the noise vector brings in random elements ...
- 15
3
votes
1
answer
51
views
Regression loss conditioned by the ground-truth values
I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image).
Sometimes, the ...
- 43
0
votes
0
answers
17
views
Book suggestion about deploying real deep learning models in real world
Can you suggest books about deploying machine learning algorithms on robots, especially on real time stream? I don't know how to deal with latency and other challenges that real time inference/stream ...
- 862
0
votes
0
answers
23
views
Encoding categorical data with "many" unique values in neural network
I am new to machine learning, in fact, I am implementing my first deep neural network from scratch without any framework.
The dataset has 3500 rows, and 4 categorial columns of which two have about ...
- 113
0
votes
0
answers
23
views
how to know to define asuitable posterior for bayesian deep learning
I work with model its purpose for classification task in medical image, i want also to estimate uncertainty , so i work with Bayesian technique , i use a pre-trained model for feature extraction and ...
- 1
0
votes
0
answers
9
views
I'm trying to build image search like Google Photo-Image with face is given to model & it'll get all the images in database in which he/she is present
When a user upload a selfie, the model search same person in dataset of images of multiple persons and get back all the images in which that person is present.
Step 1: From dataset of images I detect ...
- 1
0
votes
0
answers
10
views
Classes definition for detecting impervious surfaces on aerial photographies
My project is to use deep learning, essentially a UNET segmentation model, to detect impervious surfaces on high resolution aerial photographies.
I wonder if it's better to train the model with many ...
- 101
1
vote
1
answer
75
views
Inquiry on Combining Two Neural Networks for unsupervised training: Has This Been Researched?
Hello AI Stack Exchange Community,
I am exploring an idea related to neural networks, and I'm curious to know if this method has been previously researched or if there is a specific term for it.
I am ...
0
votes
0
answers
28
views
Has There Been Research on Using a Neural Network as a Loss Function for Another Neural Network?
I'm intrigued by the idea of employing a separate neural network (which I'll refer to as the "loss network") to compute the loss for a primary network based on its inputs and outputs. The ...
0
votes
0
answers
31
views
Watermark removal without inpainting
Suppose I have a bunch of images that have been watermarked with a transparent logo with some unknown blending function g as ...
- 159
1
vote
0
answers
39
views
Are there neural networks that compute weights dynamically based on geometric attributes of neurons?
I am interested in exploring neural network architectures where the weights are not stored but are computed dynamically based on certain attributes or "dimensions" of the connected neurons.
...
0
votes
0
answers
7
views
I have n-dimensional latent representational data, with a y logit label: How do I find peaks in the data using the label?
I essentially need a "find peaks" algorithm for when the input data is n-dimensional.
Specifically, in the latent space of my neural network I have have collected all the training data ...
- 101
0
votes
0
answers
23
views
What algorithms could I use if I want to increase the accuracy of matched keypoints in an image pair?
Let's say that I used a keypoint detector like SIFT or SuperPoint to detect keypoints in image 1 and 2. Afterwards, I used a keypoint matcher to match corresponding keypoints in this image pair. The ...
- 101
1
vote
0
answers
63
views
In the figure of Stable Diffusion, when does the switch part change?
In the illustration of Stable Diffusion, there is a concatenation part through Cross-Attention. Why is there a switch in the concatenation part?
0
votes
1
answer
24
views
Multiple Loss Functions For Proper Parameter Updates [closed]
I am working on a model on PyTorch where it has two loss functions each fed from two separate input datasets. I want to update the model parameters based on these loss functions, ic_loss, res_loss, ...
0
votes
0
answers
21
views
Building an algorithm that detects an area delimited by a quadrat
I want to build an algorithm that takes an image and outputs the same image, but cropped so that it focuses only on a particular area delimited by a quadrat, like so:
This is not my image as I am not ...
- 1
0
votes
1
answer
28
views
Data out of scale
Imagine that you want to train a deep learning model for forecasting/transforming data and have a training dataset with endpoints 0 and 1000. Now, Imagine receiving a new dataset with endpoints 0 and ...
0
votes
0
answers
18
views
Exploring the Similarity of Sibling’s Voices Using Automatic Speaker Recognition
I want to start project on Exploring the Similarity of Sibling’s Voices Using Automatic Speaker Recognition
Everyone has a unique voice, because of the different structure of their articulatory ...
0
votes
0
answers
12
views
Possible Reasons for the Discrepancy in Trainable Parameters of the Extended DeepConvLSTM Model
I have implemented DeepConvLSTM baseline Model input are 60×d frames each representing 60 samples with d features. Frames are fed into four consecutive convolution layers with standard rectified ...
- 41
1
vote
1
answer
105
views
Reinforcement Learning vs Supervised Learning [duplicate]
I have never tried reinforcement learning in my life.
I'm planning to apply it in robotics.
I have some experiences using supervised learning mainly deep learning. So, that's mean I will use neural ...
1
vote
0
answers
24
views
Choosing a Deep Learning model to analyse microscope images
I have lots of simulated training data of microscope images and I want to train a network to count the number of points in the image. These points are distributed in concentric oblate shells. The ...
- 111
1
vote
0
answers
26
views
Is it possible to have a good validation accuracy with random labels?
I'm currently trying to train a siamese network to determine if two inputs are similar or not. The inputs are power consumption traces and I'm basically using the siamese network as some sort of ...
1
vote
0
answers
23
views
Siamese network, cosine similarity unexpected result?
I was reading more about siamese network and it's use for similarity problems and I've stumbled upon this https://keras.io/examples/vision/siamese_network/
I was surprised to see both similarities in ...
- 11
0
votes
1
answer
72
views
How to force Transformer to give more weight to certain tokens
I'm developing an encoder-decoder based transformer model and I would like to ask if there are ways to incentivize or penalize certain tokens during training.
I'm working on a translation task where ...
- 101
0
votes
0
answers
32
views
How does a zero-order hold kernel in a Convolutional Neural Network look like?
Several papers co-authored by Hitoshi Kiya propose to use a fixed convolutional layer with a zero-order hold kernel to avoid checkerboard artifacts in CNNs. [1, 2, 3]
While there is plenty of ...
- 101
1
vote
1
answer
24
views
LSTM with multiple data streams
I am working on the following problem:
I have ~10 weather stations in somewhat approximate areas, at some points during the day (different for each station), I get readings of various data points (...
- 11