Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Why is the embedding of a task using Task2Vec not depend on the model?

I saw this in the Task2Vec paper: TASK2VEC depends solely on the task, and ignores interactions with the model which may however play an important role. To address this, we learn a joint task and ...
1 vote
1 answer
36 views

Similarities between 2d-vectors. (to flatten or to not)

I have this scenario where I need to measure the similarity between a 2d tensor t1: (100,8) and 61 tensors of the same shape(100,8). 100 represent time-steps and 8 is the no. of options. I first tried ...
0 votes
0 answers
19 views

Can Graph Neural network leverage only the topological structure?

Graph Neural Networks (GNNs) are a powerful tool that allow learning on graphs by leveraging both the topological structure and the feature information for each node. For the particular problem I am ...
1 vote
1 answer
25 views

What loss function will be correlated with classification metrics?

Recently I developed a custom training algorithm for deep learning models, based on evolutionary algorithms. Details are not important, except that it also uses decreasing regular cross entropy loss ...
  • 1,195
0 votes
2 answers
37 views

What is the difference between Restricted Boltzmann Machine and Artificial Neural Network?

In the deep learning course I took at the university, the professor touched upon the subject of the Restricted Boltzmann Machine. What I understand from this subject is that this system works ...
0 votes
0 answers
27 views

Ancient Artificial Intelligence

I wanted to ask a question about the potential existence of an older (non modern) Artificial Intelligence. If artificial intelligence is a collection of data, algorithms/responses to questions and ...
0 votes
1 answer
23 views

Is data augmentation beneficial even if the dataset is large/diverse enough?

I'm training a deep learning model to map binary images to grayscale values of the same shape. For the dataset, I can genearate one as large and diverse as I want it to be. My question is: let's say ...
0 votes
0 answers
14 views

How does backpropagation work with multi-branch models?

How does backpropagation work when the input layer feeds into two or more separate branches of layers before merging back to produce a single output, such as can be implemented in the Keras Functional ...
0 votes
0 answers
11 views

Any learning method to connect an edgeless graph?

Given N nodes with no edges connecting them at all. Each node has certain n features. Is there a way to connect these nodes and form a connected graph. The idea is to then feed the outputted graph ...
  • 101
0 votes
0 answers
26 views

Deep reinforcement learning:DQN can't converge

i am work on a project.i use the dqn to maximize return. this picture are some env states. i found that dqn did learn a bit, but after a while it stopped improving and even started to decline. this my ...
0 votes
1 answer
31 views

Non-Convex loss-surface although quadratic loss function

there is one problem which bugs me quite a long time, it is the non-convex loss shape (multiple minima, e.g. shown here) of neural networks which use a quadratic loss function. Question: Why is a “...
  • 3
0 votes
0 answers
29 views

Deep Learning model to predict ELO chess rating from games?

I was wondering if anyone had already tried building a regression model that could predict the ELO rating of a chess player based on his last N games? Inputs could be anything, from chess notation of ...
2 votes
0 answers
58 views

Deep reinforcement learning the board game "Battle Sheep" - too large action space?

I was recently introduced to this simple board game called "Battle Sheep". In this game, two to four players try to acquire as many hex tiles from a hex grid as possible. You can find the ...
0 votes
1 answer
35 views

How to incorporate domain knowledge into a semantic segmentation network?

I'm working on a semantic segmentation project, and want to add some domain knowledge to the system. I want to ensure that for segmentation, there can only be one group of pixels that are predicted as ...
0 votes
0 answers
11 views

How to get sentence from embedding vector with Universal Sentence Encoder?

Given a sentence embedding vector from a Sentence Encoder (like Sentence-BERT), I want to train a model to generate the original sentence (list of word embedding). Are there any architectures to ...
0 votes
1 answer
31 views

How is a filter actually applied to all input channels in a ConvLayer2D

I was studying Convolutional Layers and some of their variations and I came across this post which says: 'For rgb vs greyscale, think about channels as feature maps for input layer and a filter gets ...
0 votes
0 answers
19 views

Why remove stop words, numbers in a conversational chatbot?

I have been working on a conversational chatbot recently using movie dialogues corpus dataset, since i am very new to this i started to see if there's already code available for chatbots. I came ...
0 votes
1 answer
24 views

Coherence is classifying time series data

I have been told that models that are continually running to detect a temporal pattern, say wake up words in Alexa or Echo, the model establishes this coherence or another word I heard, statistical ...
  • 111
0 votes
0 answers
11 views

What kind of network would you use to predict event times, when there can be many events per sample?

The each sample I have is an m x n input of time series data, with m dimensions, sampled at n times. I am trying to predict events which could happen at any of the n timespoints. For each m x n sample,...
0 votes
1 answer
27 views

How do I train a model to classify if it's a Full Human Body in the picture?

recently I started a personal project that uses some Machine Learning techniques in the process, so I'm currently collecting human images with a web scraper. I know that I can use some pre-trained ...
-1 votes
1 answer
21 views

What are "feature planes" in neural networks? (current context is deep reinforcement learning)

The input contains 14 feature planes, each of shape 11x11 What does this mean?
  • 9
0 votes
0 answers
23 views

When use positive or negative rewards in reinforcement learning? Is there anything in literature?

Let's say I can design a reward as function of a distance $d>0$ from the target in 2 ways: $r=\frac{1}{1+d}$ or $r=-d$. The first is defined in $(0,1]$ the second in $(-\infty,0]$. I would expect ...
  • 41
1 vote
0 answers
19 views

Pseudo Label Generation for Generative Cooperative Learning

I am trying to implement this paper for unsupervised video anomaly detection. The gist of the paper seems to be: Create a dataset for an unsupervised setting, by mixing up the train and anomalous ...
  • 131
0 votes
1 answer
20 views

Reach optimal values by not decreasing gradient

Is it possible to reach the optimal values ​​for the parameters by not applying the decreasing gradient in some layers?
  • 1
0 votes
0 answers
8 views

How to make inference in energy based models

How do you make inferences in energy-based models? If there is an infinite set of continuous inference points, how do you choose them?
0 votes
0 answers
19 views

saliency vs. sensitivity: proper definition and distinction

My understanding (to be critisized potentially) In general, what I understand from saliency and sensitivity in a classification problem is: Sensitivity means how sensitive is the predicted class to ...
  • 395
0 votes
0 answers
9 views

How feasible is it to partition a DNN model into functional pieces?

Just read Auto-Split: A General Framework of Collaborative Edge-Cloud AI by a group of Huawei researchers (https://arxiv.org/pdf/2108.13041.pdf). How feasible is it to break up the models and serve ...
0 votes
0 answers
17 views

How to use strong labels in image classification?

I have a dataset where I have the labels cancer & non-cancer, and I also have localized pixel-level annotation masks of important regions/features in the images. In a binary classification task, ...
  • 1
0 votes
0 answers
17 views

How does a CNN work in detecting absence of features?

I'm trying to understand how a CNN operates internally. Let's say I'm doing binary classification with 1 output neuron and a sigmoid to classify dog vs no dog. No dog meaning the image does not ...
  • 1
-1 votes
1 answer
36 views

Choosing proper graphic card for deep learning AND gaming [closed]

Though the combination between gaming and deep learning might not sound "serious" enough still this is the case - on the one hand, I need a great GPU for my son to play games, and it will be ...
  • 137
0 votes
0 answers
14 views

Exact definition of WRN-d-k (Wide ResNet)

I am a little confused about the WRN-d-k notation from Wide Residual Networks. To quote the paper, In the rest of the paper we use the following notation: WRN-n-k denotes a residual network that has ...
  • 161
0 votes
1 answer
58 views

Is large language model and foundation model the same thing?

I read a lot about foundation model and large language model. However, I dont find a clear definition what exactly is a foundation model. Is large language model and foundation model the same thing?
0 votes
0 answers
35 views

How exactly Keras calculate the validation accuracy and training accuracy?

After each epoch, Keras provides you some evaluations; train_accuracy,train_loss, validation_loss, and validation_accuracy. Keras evaluates the whole model using validation set to get how good the ...
  • 101
0 votes
0 answers
7 views

Shuffle data inside learning sample in order independet transformer model

Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
0 votes
1 answer
19 views

Does the Weights of Discriminator get updated when traning Generators in GANs?

When we train the GAN we usually train the discriminator first then the generator, first we stop the generator from updating its weight by removing it from the computation graph, using fake_image....
0 votes
1 answer
30 views

Meaning of Large Dataset for machine learning

Some online answers about parameters in machine learning mention that it is dependent on the size of dataset we have (if it is a large dataset or not). Is this size related to the number of samples we ...
1 vote
0 answers
26 views

Categorical loss function for variable number of labels

I have a model for binary classification. The target variable has the different number of labels (instances) in each sample. For example, a batch of size 2 with 2 and 3 instances and correspondingly ...
0 votes
1 answer
102 views

Last linear layer of the decoder of a transformer

I am learning the transformers architecture from these two sources: https://arxiv.org/pdf/1706.03762.pdf https://jalammar.github.io/illustrated-transformer/ I just wanted to ask about the final step ...
  • 101
1 vote
1 answer
31 views

When do you know that your neural network is learning something when metrics are garbage?

When training a neural network for binary classification on a highly imbalanced set its training loss decreases, however validation loss increase even though accuracy is very high (due to highly ...
  • 113
0 votes
0 answers
8 views

Is my Deep Feature Extractor pertinent?

I have multiple DNN that can extract features vector from images. Those can be used for two main goals: Use them for transfer learning ang faster trainings Use them as feature extractors, and train ...
0 votes
0 answers
12 views

Machine/Deep learning model for object labeling in Check Images?

I am currently facing an issue with identifying sections within a check images, something like object identification. Initially it seemed I could use YOLOv5, because it is good with object detection. ...
0 votes
0 answers
14 views

How to embed relational information in a Transformer for NMT task?

I have AMR graph like the following: I am using Transformer model for machine translation. However, my input data has relational information as shown above. This information has semantic information ...
0 votes
0 answers
29 views

Why "Good Model" that performs great on holdout validation data fails on production data

I have this binary regression model that has ~500 futures with an unbalanced dataset with the following results. ...
  • 13
1 vote
1 answer
38 views

How to handle out-of-bound values in Production data?

So I have this model but the data may vary. And it is virtually impossible to always have the values in bounds. If I do I`d have to use larger period leading to concept shift which is worse. The ...
  • 13
1 vote
0 answers
10 views

Bayesian optimization with confidence bound not working

I have a simple MLP for which I want to optimize some hyperparameters. I have fixed the number of hidden layers (for unrelated reasons) to be 3. So the hyperparameters being optimized through Bayesian ...
  • 111
4 votes
1 answer
68 views

Why is multilayer perceptron a nonparametric model?

E. Alpaydin, Introduction to ML, 4-th edition, page 46: Over time, it has been realized in the neural network community that most neural network learning algorithms have their basis in statistics—for ...
0 votes
0 answers
15 views

uniform gap between training and validation metrics

I am training a neural network (Deep and cross network) for a multi-label classification task (~700 labels). I have around 2.5 million samples, splitted 8/1/1 for train/test/validation. I am seeing a ...
  • 121
0 votes
1 answer
29 views

What is the training accuracy of this model?

I’m trying to classifiy ECG signals using LSTM and MATLAB, the above plot shows that the training accuracy of the system is 100% but when I apply this code to calculate and get the accuracy I get only ...
1 vote
0 answers
164 views

Whys and Why-nots using Rust for AI

The title says it all. I would like to know more about what attributes and design choices of Rust that make it a good (or bad) language for the entire ecosystem of AI (both research and production) ...
0 votes
0 answers
7 views

Difference in mask in the end-to-end transformer model>

In the book Deep Learning with Python, 2nd edition François Chollet writes (section 11.5.3. listing 11.36, page 361): ...

1
2 3 4 5
36