Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

KD-loss between intermediate feature maps of different channels

Assume a teacher model and a student model. Teacher is bigger than student in terms of depth and/or width, however, comes from the same "family". In addition to the loss that involves the ...
Natan ZB's user avatar
1 vote
1 answer
23 views

When should one not use Normalization Techniques in a deep learning models?

When should one not use normalization between layers like Batch Norm, Layer Norm, Instance Norm, and Group Norm in deep learning while training a DL model?
Jivitesh's user avatar
0 votes
0 answers
13 views

Does reducing the image dimensions before sending them to Language Models (LMs) significantly affect the processing time?

I am currently working on utilizing Language Models (LMs) to describe images in my project (Search Engine for AI images). However, I am wondering about the efficiency of the process, particularly in ...
Oleksandr G's user avatar
0 votes
0 answers
10 views

How to train an AI model to identify GUI elements

Goal: I would like to make an app that can detect UI elements, extract them, and let the user manipulate them in some way. Existing Solutions: There are 2 tools that I used that try to do this, but ...
OGOG's user avatar
  • 11
-1 votes
0 answers
21 views

EEG signal prediction of stress and non stress dataset. using MNE PYTHON AND ICA FOR better accuracy [closed]

...
abdul's user avatar
  • 1
0 votes
0 answers
18 views

Advanced Techniques for Feature Detection in 3D Models with Overlapping Features Question:

I'm working on feature detection within 3D models (given only points and lines forming areas), specifically focusing on extracting detailed information (e.g., corner coordinates) of complex features. ...
Sheyteo's user avatar
0 votes
0 answers
21 views

CNN accuracy is too low using VGG16 [closed]

Hello i'm working on a simple problem, my model is simple it consists of VGG16 as a base_model and a fully connected layer that has 16 units and relu as activation and finally the output layer which ...
Gabovix's user avatar
0 votes
0 answers
18 views

Object detection with deep learning

I am getting familiar with video detection/tracking using neural networks. Specifically, I am referring to algorithms that take in input an image or a sequence of images and, for each frame, it ...
user1135371's user avatar
2 votes
1 answer
47 views

Why do adversarial attack transfer well?

I have read (*) that a common technique to attack a black box AI system based on a neural network is to use it to train a surrogate model to make the same classifications as the black box one. Once ...
Weier's user avatar
  • 131
1 vote
2 answers
58 views

Books that contains exclusively math problems/assignments in Deep Learning & Neural Networks

I am doing a Deep Learning Course.Suggest some books that contains exclusively math problems/assignments in Deep Learning & Neural Networks. I can understand that majority of the replies suggest &...
GKK's user avatar
  • 13
0 votes
0 answers
17 views

What is downsampling in Resnet Paper [closed]

I am trying to implement Resnet paper from scratch (i.e., without using resnet library). However I couldn't understand what is downsampling and its implementation and need.
RAJASEKHAR REDDY's user avatar
0 votes
1 answer
19 views

How can I train AI to learn how to recognize highly impaired speech from a dystonic person? [closed]

Speech recognition AI generated technology for impaired speech recognition.
Nelson's user avatar
  • 1
0 votes
0 answers
24 views

Much higher LLM training loss when gradients are accumulated in FP32

I'm training a large language model in BF16 precision. The model architecture is the same as llama v1. I'm using Megatron-LM (along with my own integration of DeepSpeed) as the main training software. ...
user416983's user avatar
0 votes
0 answers
52 views

What are the differences between Inception Score and Fréchet Inception Distance?

From the articles I've read about image generation using GANs, the Inception Score measures two things simultaneously: the variety of images (diversity) and the distinct quality of each image. Does ...
abcd's user avatar
  • 35
1 vote
1 answer
41 views

Image classification of more than 60,000 classes

I am working on a problem that requires the classification of more than 60k classes. I have around 1k to 1.5k images per class. I am using synthetic data for training and want to evaluate it on real ...
pks's user avatar
  • 11
0 votes
0 answers
8 views

How do encode selecting certain cards or combinations of cards into my output layer?

I'm currently trying to teach an AI to play the card/dice game DragonWood. I have generated a python library to handle the game rules and I have a simple deterministic algorithm to make decisions. I ...
Domronic's user avatar
0 votes
0 answers
22 views

Is using LLMs a good solution for classification problems? [closed]

Consider a classification problem. In such a way that we have a large number of samples and in the learning process we classify them into several specific classes. In the test phase, the desired model ...
user16385455's user avatar
1 vote
1 answer
35 views

Is "The Dimpled Manifold Hypothesis" correct to say this about autoencoders?

This quite famous paper states page 3 that: The (well-known) fact which underlies the new conceptual framework is that all the natural images are located on or near some low-dimensional manifold (as ...
Quersi's user avatar
  • 13
0 votes
0 answers
47 views

When training an autoencoder model, the loss is too large. Am I doing something wrong? [closed]

I trained an image recommendation AI model. The model's source code was not written by me, but was written by the model developer. And this model's loss function was slightly different from the loss ...
yeongha kim's user avatar
5 votes
1 answer
618 views

How can the discriminator determine the sample is fake or real?

Based on the articles I've read, the discriminator can identify whether a sample is fake or real. However, the articles don't clarify the conditions used to determine if a sample is fake or real. I ...
David's user avatar
  • 75
2 votes
1 answer
59 views

What are meaning of parameters $\theta$ in this context?

I'm reading the article about generative model from Open AI, here is the section where they explain them: Our network is a function with parameters $\theta$, and tweaking these parameters will tweak ...
David's user avatar
  • 75
0 votes
0 answers
12 views

Explanation for the expression of positional encoding in NeRFs

I was reading the NeRF paper recently (https://arxiv.org/pdf/2003.08934.pdf), and under the positional encoding section, I see that the authors propose usage of the following function for transforming ...
user185887's user avatar
1 vote
0 answers
18 views

What is the right way to make a neural network learn a period function with known period

I want a neural network to learn the representation of a periodic function whose period is known to be $T$. What is the correct way to achieve that? From my reading, I could infer two things: This ...
user185887's user avatar
2 votes
1 answer
41 views

What does Figure 3 in the BERT paper represent?

The BERT paper has the following diagram (Figure 3): It's captioned "Differences in pre-training model architectures". However, I thought the BERT architecture was just a stack of attention ...
statusfailed's user avatar
0 votes
2 answers
70 views

Is it legitimate to train a model on a benchmark dataset and use this model only for labeling another datasets

To assess our deep learning models (CNN) we have labeled a big benchmark dataset (it was labeled by specialists so it is kind of Ideal). I of course know that we do not want to train new models using ...
Igor's user avatar
  • 181
1 vote
1 answer
30 views

How come all the multi-headed self-attention layers don't end up learning the same aspect of a natural language?

How come all the multi-headed self-attention layers don't end up learning the same aspect of a natural language? Since we don't dictate ahead of time what the self-attention layers focus on, how do we ...
Tfovid's user avatar
  • 187
3 votes
1 answer
69 views

Are the Dot Product and Tensor Product the same thing in Machine Learning?

I'm currently reading "Deep Learning with Python, Second Edition" by François Chollet, and I need help understanding one thing. Below paragraph was copied from the page 41 2.2.3 Tensor ...
Kamil Bęben's user avatar
0 votes
0 answers
7 views

How to calibrate IMU for large scale deployments possibly using deep neural network

We were testing our visual SLAM algorithm on robots. We were getting poor performance. Then we calculated wite noise and random walk parameters (using kalibr) for the IMU and used it in our algorithm ...
Mahesha999's user avatar
0 votes
0 answers
10 views

How to train ViT on smaller datasets?

I know ViTs aren't made for small datasets and low resolution. But have you ever reached traditional CNN accuracy using ViT on CIFAR10/100. I have been playing around with ViT on CIFAR10 and 100. But ...
v1998199904's user avatar
0 votes
0 answers
23 views

Calculating class-specific permutation feature importances for multilabel classification problem

I would like to apply the permutation feature importance technique to rank the features of a siamese network model that I trained. I am currently using this siamese network to perform some kind of ...
ashenflower's user avatar
1 vote
1 answer
32 views

Impact of scaling in loss terms when loss function is a composition of multiple functions

I am training a deep learning model, the loss function of which is of the form $$ \cal{L} = \cal{L_1} + \cal{L_2} $$ where $\cal{L_1}$ and $\cal{L_2}$ are of very different orders. WLOG, let's assume ...
user185887's user avatar
0 votes
0 answers
46 views

How do I deal with dynamic, parameterized, action spaces?

I want to design an AI Learning Algorithm for a Student made, round based Game. Let me first explain the Game/Environment You have a round based HTTP Game, in which multiple Players can participate. ...
Andre's user avatar
  • 1
0 votes
0 answers
36 views

Understanding the concepts of embedding in Roberta architecture?

I'm reading the implementation file of Roberta architecture, specifically in the RobertaEmbedding class, this class has the comment: ...
David's user avatar
  • 75
0 votes
0 answers
12 views

Looking for an audio classification approach

I am working on a deepfake audio classification project with a dataset consisting of only 3000 samples. I have made several attempts to address this challenge. Firstly, I extracted melspectrograms and ...
user21456801's user avatar
-1 votes
1 answer
88 views

Difference between FLAX and pytorch

I am going through a code written in FLAX instead of pytorch .Can someone please explain what is the difference between these two deep learning frameworks?
akhil's user avatar
  • 9
0 votes
1 answer
65 views

Are LLM parameters synonymous with dimensions?

For example, would a Large Language Model (LLM) with parameter size 140 Billion have 140 Billion dimensions as defined in deep learning as the number of nodes in the input layer? Another way to ask ...
geominded's user avatar
  • 101
1 vote
1 answer
106 views

Do I understand the technology of AI upscaling of films and cartoons correctly?

With the help of artificial intelligence, it is possible to increase the resolution of images that are initially low resolution, bringing it to ultra-high resolution. Also, initially static images are ...
dtn's user avatar
  • 239
0 votes
1 answer
57 views

Main subjects to learn Artificial Intelligence in CS [closed]

In my PhD, I will work with ML models. However, I will only use ready-made models as a tool, but I want to delve deeper into Artificial Intelligence not just to use ready-made models, but to ...
Everson Gomes's user avatar
0 votes
0 answers
16 views

What are efficient methods for classifying tremor intensity, given challenges with 3D CNN and memory issues with a pretrained VideoMAE model?

I have a training dataset of 80 videos (without augmentation). They are 15 seconds video recorded at 30fps and 240*480 dimensions. What options do I have to train a classification model? The problem ...
Ravideep Singh's user avatar
0 votes
1 answer
42 views

How to prevent update a pretrained model if a model is optimized with backpropagation? [closed]

These are components in my model: A generator An encoder: a pretrained, and should not updated. A loss function. Input is passed to the encoder to generate X, X is then passed to generator to ...
Jesse's user avatar
  • 9
1 vote
0 answers
15 views

Enhancing Generalization in DRL Agents in Static Data Environments

Context: I'm working with a deep reinforcement learning (DRL) agent in a market-like environment where its actions do not affect the environment. The environment uses historical data up to a certain ...
ElonMuskofBadIdeas's user avatar
-1 votes
1 answer
51 views

PINN - Why Differential equations only? [closed]

Im recently studying Physics informed neural networks. I found that it is designed for solving Differential equations. Is there any reason why it is used for differential equations only?
COTHE's user avatar
  • 3
1 vote
0 answers
15 views

Neural Machine Translation with multi-language input to a single-language output?

I'm looking for NMT paradigms where the input to the model is the same text in N languages (e.g., L1, L2, L3) and the output is the translation in a different target language (e.g., L4). However, I ...
yigitcankaya's user avatar
1 vote
1 answer
37 views

Comparing the performances of GPTs with deep learning in the field of binary files and their related reports

Regarding the case study of a dataset including binary files (containing assembly code) and reports related to each file (the content of the static analysis of the file as well as the analysis of the ...
user16385455's user avatar
0 votes
0 answers
9 views

What do RNN neural networks lack in nowcasting time series?

I want to write a master thesis on nowcasting GDP? Has this been used and if so I don't fully understand how the neural networks should be built if I forecast quarterly GDP and link that to ...
J_Bake's user avatar
  • 11
1 vote
0 answers
26 views

Why doesn't CLIP use a pretrained large language model as the text encoder?

CLIP: https://openai.com/research/clip They use a small text encoder and suggest that the simpler the model the better. Is there any reason why that is? Has anyone tried using a pretrained LLM as the ...
JobHunter69's user avatar
2 votes
1 answer
50 views

In neural style transfer, why do we focus on a single style image?

In all the descriptions of neural style transfer that I've seen so far, there is a single style image and a single content image, and the task is to produce a new image with the style of one image and ...
A Kubiesa's user avatar
  • 123
1 vote
1 answer
32 views

How to choose the metric value when evaluating the performance of a deep learning model?

When evaluating the performance of a deep learning model (for the purpose of publishing a research work), should we choose the optimal value from the metric curve (such as the accuracy curve) or the ...
Liuji's user avatar
  • 19
0 votes
0 answers
26 views

How to find distance between class representation's decision boundaries for a neural network?

I have a 5 layer DNN with data containing 10 classes. To study how the model works, one thing among many I am looking at is the class wise representations. I can extract the representations of each ...
v1998199904's user avatar
0 votes
1 answer
70 views

Is global pooling necessary in image classification models? [closed]

In many image classification models, the global pooling operation is performed before the classification layer (i.e. fully connected layer) to reduce model complexity. Is the global pooling layer a ...
Liuji's user avatar
  • 19

1
2 3 4 5
41