Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0 votes
1 answer
62 views

Main subjects to learn Artificial Intelligence in CS [closed]

In my PhD, I will work with ML models. However, I will only use ready-made models as a tool, but I want to delve deeper into Artificial Intelligence not just to use ready-made models, but to ...
Everson Gomes's user avatar
0 votes
0 answers
16 views

What are efficient methods for classifying tremor intensity, given challenges with 3D CNN and memory issues with a pretrained VideoMAE model?

I have a training dataset of 80 videos (without augmentation). They are 15 seconds video recorded at 30fps and 240*480 dimensions. What options do I have to train a classification model? The problem ...
Ravideep Singh's user avatar
0 votes
1 answer
42 views

How to prevent update a pretrained model if a model is optimized with backpropagation? [closed]

These are components in my model: A generator An encoder: a pretrained, and should not updated. A loss function. Input is passed to the encoder to generate X, X is then passed to generator to ...
Jesse's user avatar
  • 9
1 vote
0 answers
15 views

Enhancing Generalization in DRL Agents in Static Data Environments

Context: I'm working with a deep reinforcement learning (DRL) agent in a market-like environment where its actions do not affect the environment. The environment uses historical data up to a certain ...
ElonMuskofBadIdeas's user avatar
-1 votes
1 answer
62 views

PINN - Why Differential equations only? [closed]

Im recently studying Physics informed neural networks. I found that it is designed for solving Differential equations. Is there any reason why it is used for differential equations only?
COTHE's user avatar
  • 3
1 vote
0 answers
17 views

Neural Machine Translation with multi-language input to a single-language output?

I'm looking for NMT paradigms where the input to the model is the same text in N languages (e.g., L1, L2, L3) and the output is the translation in a different target language (e.g., L4). However, I ...
yigitcankaya's user avatar
1 vote
1 answer
37 views

Comparing the performances of GPTs with deep learning in the field of binary files and their related reports

Regarding the case study of a dataset including binary files (containing assembly code) and reports related to each file (the content of the static analysis of the file as well as the analysis of the ...
user16385455's user avatar
0 votes
0 answers
10 views

What do RNN neural networks lack in nowcasting time series?

I want to write a master thesis on nowcasting GDP? Has this been used and if so I don't fully understand how the neural networks should be built if I forecast quarterly GDP and link that to ...
J_Bake's user avatar
  • 11
1 vote
0 answers
30 views

Why doesn't CLIP use a pretrained large language model as the text encoder?

CLIP: https://openai.com/research/clip They use a small text encoder and suggest that the simpler the model the better. Is there any reason why that is? Has anyone tried using a pretrained LLM as the ...
JobHunter69's user avatar
2 votes
1 answer
55 views

In neural style transfer, why do we focus on a single style image?

In all the descriptions of neural style transfer that I've seen so far, there is a single style image and a single content image, and the task is to produce a new image with the style of one image and ...
A Kubiesa's user avatar
  • 123
1 vote
1 answer
32 views

How to choose the metric value when evaluating the performance of a deep learning model?

When evaluating the performance of a deep learning model (for the purpose of publishing a research work), should we choose the optimal value from the metric curve (such as the accuracy curve) or the ...
Liuji's user avatar
  • 19
0 votes
0 answers
28 views

How to find distance between class representation's decision boundaries for a neural network?

I have a 5 layer DNN with data containing 10 classes. To study how the model works, one thing among many I am looking at is the class wise representations. I can extract the representations of each ...
v1998199904's user avatar
0 votes
1 answer
70 views

Is global pooling necessary in image classification models? [closed]

In many image classification models, the global pooling operation is performed before the classification layer (i.e. fully connected layer) to reduce model complexity. Is the global pooling layer a ...
Liuji's user avatar
  • 19
-1 votes
1 answer
67 views

How is back propagation applied in case the activation function is not differentiable?

Back propagation is based on partial derivatives including the activation functions. How is back propagation applied when the activation function is not differentiable?
DSPinfinity's user avatar
0 votes
2 answers
66 views

Why we use Convolutional Neural Network for image data and not the Feedforward Neural Network Draw and explain the architecture of Convolutional Netwo [closed]

Why do we use Convolutional Neural Network (CNN) for image data and not the Feedforward Neural Network (FNN)? Draw and explain the architecture of Convolutional Network
Himanshu Sinha's user avatar
2 votes
1 answer
74 views

Do different camera angles affect the performance of the deep learning model?

I'm working on a project to build a face recognition system and I have a question: do different camera angles affect the performance of the deep learning model? For example, in CCTV, training data and ...
Naay's user avatar
  • 23
0 votes
1 answer
321 views

How does Chat GPT encode a question?

Chat GPT is based on a decoder-only Transformer so it does not have an encoder. Given that, how is a user's question passed as input to Chat GPT's decoder? In a regular encoder-decoder architecture, ...
joan's user avatar
  • 1
0 votes
1 answer
54 views

Using naive bayesian vs. transformer-based architecture model for human-annotated data?

I have a reddit dataset with thousands of online posts over the economy and inflation. We have used human-annotation on 60% of posts to determine whether users blame the following entities over the ...
maldini1990's user avatar
0 votes
0 answers
27 views

High Fluctuations in Validation Curve

Below I attach an image of accuracy curves. I got a lot of suggestions regarding some improvement in below curves. Following are my experiments in order to make the curve stable--> I used lr = 4....
Sarvagya Porwal's user avatar
0 votes
1 answer
82 views

Advanced / Complex Neural Network (System) Design

In addition to basic forward networks, many books cover other basic network designs like CNN's and RNN's. However, they don't really go any further than that, explaining things like common approaches ...
Thorsten Schmitz's user avatar
2 votes
0 answers
88 views

What is the meaning of "dimensionality of the embeddings and hidden states"?

I was reading the GPT-2 and LSTM documents and noticed that they use the terms "dimension of embedding and hidden state". For GPT-2, the size is $768$, and for LSTM, the size is $256$. What ...
user avatar
1 vote
1 answer
116 views

When to use Pruning, Quantization , Distillation and others when optimizing speed

I want to understand how to optimize models for inference speed and am seeking some advice and best practices for the same. I am a little bit aware of the concepts of pruning, quantization, and ...
Hiren Namera's user avatar
1 vote
1 answer
347 views

Is Softmax Necessary as the Activation Function for Self-Attention Mechanisms?

I’m curious about the mathematical reasoning behind the use of the softmax function as the activation function in self-attention mechanisms within neural networks. Specifically, I’m interested in ...
Kasia's user avatar
  • 303
0 votes
0 answers
37 views

Neural network dynamic input shape vs fixed input shape in input layer for handling NULL values

I have binary classification problem, we know that the output layer will be scalar or dense a.k.a. 1 unit neuron with sigmoid as function activation. 1 means is the subject will die, while 0 means ...
Muhammad Ikhwan Perwira's user avatar
1 vote
2 answers
674 views

Do diffusion models take a long time to train?

I am trying to train a diffusion model (from scratch in pytorch). UNet used is not anything too fancy, takes in images and time step as input for about 512 time steps. I am using learnable embeddings ...
Aditya 's user avatar
0 votes
0 answers
29 views

How to let the AI trained on MNIST recognize my own hand-written number?

I have already trained a simple model without using any frameworks to recognize the hand-written numbers from the MNIST dataset. Actually it's just a copy of the example on the book Grokking ...
Jason Jia's user avatar
1 vote
1 answer
492 views

Which epoch is the best for me to choose?

I have trained my deep learning model. I also saved the validation loss to a file and plotted on a graph I have $2$ questions for this: Does the validation loss look normal? Is there any issue with ...
user avatar
0 votes
0 answers
56 views

Prefix tuning in LLM uses learnable vectors to fine tune the model

I would like to implement a new architecture for Transformer. Below description is my thought. Prefix tuning in LLM uses learnable vectors to fine tune the model. Is there a way to use the output ...
jackson's user avatar
1 vote
1 answer
79 views

Do GANs have constant running time?

After the model is trained, you just need to input random noise and the generator will output an image, does this mean GANs have constant running time ? I'm asking about both naïve GAN and variants of ...
user avatar
1 vote
0 answers
147 views

Question about the Conditioning Augmentation technique?

In the paper StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, the goal is to convert text descriptions into images. The text encoder encodes the ...
user avatar
0 votes
2 answers
112 views

How to detect negative (absence of) an object?

I want to detect the people that are NOT wearing PPE vests using a pre-trained object detection model like YOLO or Grounding Dino. The models are able to detect people and vests separately, but I am ...
Krithik Roshan's user avatar
0 votes
2 answers
95 views

how to determine the number of units for dense layer for transfer learning?

I'm using MobileNetV2 for classification, and I want to add dense layers(i remove the last layer of the MobileNetV2 model). How do I choose the number of units for the dense layer after obtaining the ...
Cy Rine's user avatar
0 votes
0 answers
25 views

Can't get a correct accuracy on tabular data using deep learning

This is my first message here, and I would like to seek some assistance ! I have a technical test for a job that I really want, and I have 10 days to complete it. I've attempted to work on it, but I'm ...
Enzo Durand's user avatar
2 votes
2 answers
202 views

The training process of a conditional GAN

For example, consider a dataset like MNIST. I give the conditional vector to produce only the number $7$ for both the generator and discriminator. In the following scenarios, what will the ...
abcd's user avatar
  • 153
2 votes
2 answers
193 views

Can mini-batches for stochastic gradient be balanced but not representative of the training data?

When we construct mini-batches for stochastic gradient, it is important to ensure that the different mini-batches are balanced (for example, in case of classification they contain the same ratio for ...
DSPinfinity's user avatar
0 votes
3 answers
101 views

My model is only improving when learning rate is 1. Should I be worried?

As the title says my GNN with three layers of GAT (Graph attention layers) is only moving the metrics when the learning rate is 1. As generally the learning rate is (0,1) should I be worried? Also ...
DataDoge's user avatar
-2 votes
1 answer
60 views

the best choice to reduce a features vector

i have 1200 features highly correlated , and i want to reduce those number of features so the best choice is use feature selection or dimensionality reduction? and which method is the best in this ...
myriamkach's user avatar
1 vote
1 answer
77 views

Are foundation models something fundamentally new? Is there a proper definition?

Currently, one can hear more and more about "foundation models" but details of this are not always clear. Also, I even have the impression, that sometimes people don't talk about exactly ...
BanDoP's user avatar
  • 168
0 votes
1 answer
58 views

What is the potential issue of nested neural networks

everyone. I am working on a nested neural network architecture. For the sake of better understanding my question, simply assume the loss is $L = G(k’) - H(k'')$ where $G$ and $H$ are two functions we ...
Zuba Tupaki's user avatar
0 votes
0 answers
15 views

Why my deep learning model (FCNN/ 1DCNN) fails to learn when training on medical dataset?

I am working on a project to predict the severity of the disease, Hemophilia using a deep learning model(FCNN or 1DCNN). I am working based on the information provided in this article: https://www....
54rnd's user avatar
  • 1
1 vote
1 answer
198 views

In the conditional GAN (cGAN) architecture, why does the discriminator need conditional variable?

I'm reading about conditional GAN (cGAN) architecture, what I know is that the generator creates images combining both noise vector and conditional variable, the noise vector brings in random elements ...
abcd's user avatar
  • 153
3 votes
1 answer
67 views

Regression loss conditioned by the ground-truth values

I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image). Sometimes, the ...
Cezoz08's user avatar
  • 43
0 votes
0 answers
21 views

Book suggestion about deploying real deep learning models in real world

Can you suggest books about deploying machine learning algorithms on robots, especially on real time stream? I don't know how to deal with latency and other challenges that real time inference/stream ...
dato nefaridze's user avatar
0 votes
0 answers
26 views

Encoding categorical data with "many" unique values in neural network

I am new to machine learning, in fact, I am implementing my first deep neural network from scratch without any framework. The dataset has 3500 rows, and 4 categorial columns of which two have about ...
M a m a D's user avatar
  • 113
0 votes
0 answers
25 views

how to know to define asuitable posterior for bayesian deep learning

I work with model its purpose for classification task in medical image, i want also to estimate uncertainty , so i work with Bayesian technique , i use a pre-trained model for feature extraction and ...
a-eng's user avatar
  • 1
0 votes
0 answers
21 views

I'm trying to build image search like Google Photo-Image with face is given to model & it'll get all the images in database in which he/she is present

When a user upload a selfie, the model search same person in dataset of images of multiple persons and get back all the images in which that person is present. Step 1: From dataset of images I detect ...
BKP's user avatar
  • 1
0 votes
0 answers
11 views

Classes definition for detecting impervious surfaces on aerial photographies

My project is to use deep learning, essentially a UNET segmentation model, to detect impervious surfaces on high resolution aerial photographies. I wonder if it's better to train the model with many ...
Below the Radar's user avatar
1 vote
1 answer
124 views

Inquiry on Combining Two Neural Networks for unsupervised training: Has This Been Researched?

Hello AI Stack Exchange Community, I am exploring an idea related to neural networks, and I'm curious to know if this method has been previously researched or if there is a specific term for it. I am ...
Deadbeef Development's user avatar
0 votes
0 answers
29 views

Has There Been Research on Using a Neural Network as a Loss Function for Another Neural Network?

I'm intrigued by the idea of employing a separate neural network (which I'll refer to as the "loss network") to compute the loss for a primary network based on its inputs and outputs. The ...
Deadbeef Development's user avatar
0 votes
0 answers
98 views

Watermark removal without inpainting

Suppose I have a bunch of images that have been watermarked with a transparent logo with some unknown blending function g as ...
Michael's user avatar
  • 159

1
2
3 4 5
41