Questions tagged [deep-learning]
For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".
1,859
questions
3
votes
0
answers
14
views
Why does training converges when the norm of gradient increases?
This is from deep learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville.
When training converges well, I thought the gradient should be at local minima. But the book says it often does ...
- 143
1
vote
0
answers
13
views
Best calculus books for Deep Learning
Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
- 11
0
votes
1
answer
16
views
Implementing deep learingn paper from scratch
I am fairly new to deep learning. I have understood theory part of the subject.
My knowledge of python is decent.
As a project I want to implement a deep learning paper from scratch (does not have to ...
1
vote
1
answer
29
views
Does MSE loss function work in NN training for predicting values between 0-1?
In a NN regression problem, considering that MSE is squaring the error and the error is between 0 and 1 would it be pointless to use MSE as our loss function during model training?
For example:
...
0
votes
1
answer
18
views
Does changing deep learning model loss function affect the occurence of plateau?
If we use two loss functions that are in the same "units" (say MAE and RMSE). Should training the same model with these two different loss functions result in plateau happening at a ...
-1
votes
1
answer
38
views
I'd like some suggestions on how to effectively master machine learning. I'm new to this and I'm kinda lost [closed]
Thing is, I got a lot of resources to learn from and I have no idea which ones will be the most effective, and where to start. I've come across 2 kinds of books, the ones which talk about the ...
- 1
0
votes
0
answers
11
views
LRP (Layer wise relevance propagation ) backward pass for two layer LSTM Networks
I am trying to calculate relevance scores for each time step of the input of my network which is a time series of shape (batch_size = 7000,sequence_length = 20,number of features = 1).
my network has ...
- 1
0
votes
1
answer
20
views
How to convert my test data in the same dimensionality as my train data
I have trained a VAE with jpg images. My latent space dimension has 768 features and when plotting the latent space it looks like this:
However, when I use the scikit learn tool LDA (Linear ...
- 3
-1
votes
0
answers
24
views
What are the key challenges and limitations of using AI models like GPT-4 for social media management? [closed]
I am currently running an experiment to explore the potential of AI models, specifically ChatGPT based on GPT-4 architecture, in managing social media content. As part of the experiment, ChatGPT ...
4
votes
2
answers
84
views
What makes the approximation capabilities of neural networks different than something like, say, Fourier series?
People often cite the universal approximation theorem as a reason for why neutral networks are so effective at capturing patterns or features of various training data. However, this seems unremarkable ...
- 143
0
votes
1
answer
20
views
Difference between sequence length and hidden size in LSTM
It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs.
For example, in the next ...
- 1
0
votes
0
answers
6
views
Using MSE or RMSE instead of CrossEntropy in Question Answering NLP problems. What are the problems if we used?
When you predict Start Index, end Index in Question Answering NLP task (SQUAD Data), you use CrossEntropy as a loss function. ...
- 253
0
votes
0
answers
37
views
Problem about Xavier Initialization [closed]
I'm learning about Xavier initialization and reading the paper "Understanding the difficulty of training deep feedforward neural networks": https://www.researchgate.net/publication/...
- 1
0
votes
0
answers
16
views
Missing Value Imputation for Time Series
I am working on a Stock Price Forecasting project, the data of which consists of 5306 instance & 12 columns. Of these, 2 columns contain about 500 instances of missing values (the starting 500 ...
0
votes
0
answers
16
views
Fast Fourier Transform in computer vision
Can someone explain me how does FFT works in computer vision, please. I know something about FFT as an algorithm of competitive programming but I can't understand how it perform an image in computer ...
- 101
0
votes
0
answers
14
views
Does splitting the classes in my dataset into sub classes improve classification accuracy?
My problem is basically classifying ok / not ok. But I do have additional information on the error cases for the "not ok" class. Should I just train on the classes that I need for my output, ...
- 73
0
votes
1
answer
52
views
How vision models based on CNNs learn the relative positions of each pixel in the image?
A CNN model is based on a series of filters applied to an image. However, these filters can only "see" a small portion of the image and they have no information of the relative position of ...
- 63
0
votes
0
answers
30
views
Why can't traditional neural networks learn to perform the same tasks that attention layers do?
If your task is to predict $t_{n+1}$ given tokens $(t_1,...,t_n)$, you could do two things:
Straight NN - feed $t=(t_1,...,t_n)$ into a neural network as an n-dimensional input and train it on ...
0
votes
0
answers
11
views
Hard time trying to fix overfit
I'm trying to make a binary classification model using keras, but it seems to overfit every time. I have tried differents architectures and its seems that a larger model performs better than a smaller ...
3
votes
1
answer
61
views
What is meant by "Deep Learning models are not understood"?
It is a sentence that I hear a lot and I guess I don't get what it means.
It seems that the weight optimization procedure is very well understood and there is, to some extent, theoretical/empirical ...
- 33
0
votes
1
answer
44
views
Difficulty in coding research papers [closed]
I understood how Yolov1 works but finding difficulty in coding it. How should I proceed?
- 13
1
vote
0
answers
13
views
Creating border around specific areas of images
I have some 1000+ image, containing data like this
Red area: Symbols
Grey area: Text describing the symbol
Note: I draw these red/grey boxes just for visualization only.
Each symbol is unique in this ...
- 111
0
votes
0
answers
41
views
Looking for a reinforcement learning algorithm that deals well with a model-based, curiosity-driven approach for chess AI
I am a software engineer that meddled with machine learning (classifiers) during my thesis.
After being out of it for a while I decided I want to try and do a neural network project to learn from, ...
- 119
0
votes
0
answers
18
views
Transformer parallelization during training
What does it mean that the decoder can be parallelized during training?
Let's assume a transformer (with both encoder and decoder) is employed for a time-series prediction. I.e. from the input ...
- 111
0
votes
0
answers
34
views
CNN without actuators
After training CNNs without actuators, I have an idea to compare their weights with each other using image mirroring. I am looking for ideas about reality perception of CNNs in this way.
What might ...
- 1
0
votes
2
answers
20
views
Higher accuracy in the test set than in the training set
Hi I'm trying to train an ANN model to classify images containing these characters: 0,1,2,3,4,T,X,S
eg.
etc...
so something like the classification of records of the MNIST dataset but using my ...
- 101
12
votes
5
answers
2k
views
Why is a bias parameter needed in neural networks?
I have read several resources, including previously asked questions such as this. I have also read arguments related to intercepts needed to separate linearly separable data. If my neural network can ...
- 326
1
vote
0
answers
15
views
Can I use zero-padded input and output layers in a 1D convnet to predict an element of interest from a variable-length input sequence?
I have developed a small encoding algorithm that accepts a time series of n = 750 samples and m = 1 feature from a scientific ...
- 111
0
votes
0
answers
37
views
Can an RNN be trained using its continuous predicted output as input at each timestep?
I am trying to train a sequence model with an RNN component that iterates through each timestep of a training sequence and decides with some probability p whether to use the previous model prediction ...
- 1
0
votes
1
answer
75
views
Keypoint generation in 3D point clouds with Deep Learning
I have a huge dataset of 3D point clouds (each point consists of X,Y,Z coordinates) and another dataset with keypoints (also X,Y,Z) which lie on quite recognizable structures in the point cloud. As a ...
- 1
0
votes
0
answers
34
views
How can a neural network learn to predict shapes or sets using only sampled points?
For $t=0,1,\dots$, consider a parameter $x_t \in ${$1, \dots, n$}, where $n \in \mathbb{N}$, and a shape $S(x_t)$ on an $m \times m$ square grid $G$. Let's denote the status of a cell $(i,j) \in G$ at ...
- 1
-1
votes
1
answer
40
views
Why use Deep Learning instead of algorithms for decision making in self-driving cars?
Let's say I want to build a self-driving car that uses cameras and ai to detect objects and driving lanes.
So if the camera detects e.g. a car in front, it calculates the distance (using a sensor) and ...
- 99
0
votes
1
answer
33
views
lyric search algorithm :prompt to real lyric
I want to make a lyric search tool.
In other words,it is means that give some prompts and get some lyrics which exist in the real world.
For example:
Input: some lyrics from coldplay ,give me power
...
- 1
0
votes
0
answers
20
views
How does "noises" in computing convolution affect the model precision and the training speed?
Consider the discrete convolution written in a matrix form, if a small amount $s$ of the zero entries (represented as white blocks) are deviated from zero, can the model precision or the training ...
0
votes
0
answers
12
views
Wondering why UNet architecture doesn't predict well. I have more information within the body of the question
I have 45K images of training set (3, 256, 256) same size and it's corresponding output is a 3D tensor with 26 masks (26, 256, 256). I have been training many times without understanding why it doesn'...
1
vote
0
answers
75
views
Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset
I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they ...
- 127
1
vote
2
answers
48
views
How do you name your deep learning training outputs?
After some time starting the deep learning project, training output files (model weights,training configuration files) will be piled up. Naming all outputs and training files can become complicated if ...
- 121
1
vote
2
answers
58
views
Does layer freezing offer other benefits other than to reduce computational time in gradient descent?
In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent?
Assuming I train a neural network on task A to derive ...
- 11
1
vote
1
answer
49
views
Does object detection for single-class images have any advantages over classification?
I recently joined a new project, and saw that they are using object detection instead of image classification for one of the business cases. The images can only belong to one class (example, the image ...
- 13
0
votes
0
answers
13
views
Which deep learning models are suitable for network_parameters-to-images mapping?
I face the problem of learning a mapping (or a translation) $f: (x,\theta) \to x^\prime$, where $x, x^\prime$ are images, $\theta$ is the parameters of a neural network.
I know the models for ...
- 111
0
votes
1
answer
45
views
Resources for NLP [closed]
I am an undergraduate student in mathematics. I have a fair bit of experience with deep learning in computer vision research and am willing to dabble into NLP. I hope that things won't be very ...
0
votes
0
answers
8
views
How many pretraining image is enough for Swin Transformer?
Here is the spec of experiment setup:
We have 3D micro CT image of the rats, and we want to perform pretraining on such data. The image is masked, so only the portion around the backbone is visible.
...
1
vote
0
answers
22
views
How can I solve the blurring problem in GAN generated images?
In my project I work in dresses dataset. I can solve the problem of black pixels but blurring still existed. I tried many computer vision filters like median filter, Biliteral filter, Sharpen methods ...
1
vote
2
answers
102
views
Why do LLMs need massive distributed training across nodes -- if the models fit in one GPU while batch decreases the variance of gradients?
Why do large language models (LLMs) need massive distributed training across nodes -- if the models fit in one GPU and larger batch only decreases the variance of gradients?
tldr: assuming for models ...
- 151
0
votes
0
answers
22
views
How to use Deep Learning to join two images of scraps of paper?
I'm trying to find a way to use a Convolutional Neural Network to join/stitch two pieces of scraps of paper that belong to the same letter. Suppose I have these two images:
I'd like to find a way to ...
1
vote
0
answers
53
views
What does Deep Q-Learning (DQL) do?
Hello :) I'm required to write a document where I describe what DQL does in short.
This is what I wrote:
DQL: instead of a Q-table, a DNN is used to approximate the Q-values for each action based on a ...
- 166
1
vote
1
answer
90
views
Formally, what are the layers in an Artificial Neural Network?
You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN).
According to the Deep ...
- 11
0
votes
0
answers
28
views
Plotting gradient over weights ratio, mean or std dev to synthetize tensors?
I was studying the lecture of Andrej Karpathy about "Activations, Gradients and BatchNorm" that he uploaded on youtube: link here.
At chapter "viz #4: update:data ratio over time" ...
- 1
1
vote
1
answer
38
views
In the Dropout paper, why would increasing the dropout increase the error rate if the capacity is constant?
In the original paper on dropout, in section 7.3.2, we see that while keeping $pn$ constant, we get a (test) error increase by decreasing retainment below 0.6. Why would that happen? If $pn$ is ...
- 11
0
votes
1
answer
32
views
L2 regularization in BN layers, how to set gamma?
I have read tensorflow's documents about batch normzalization , but still don't get what is the gamma regulizer?
the link to document:
https://www.tensorflow.org/api_docs/python/tf/keras/layers/...
