Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Using clustering to improve speed of classification

Hi I have a neural network which is very resource intensive and is used to classify audio clips. The classification is done in batches, where I record for a set period of time and then go through and ...
user avatar
  • 101
0 votes
0 answers
10 views

Can GNNs be used to predict the performance of a Neural Network?

Is possible to use a GNN to learn the hyperparameters and structure of a given DNN program (Tensorflow or PyTorch) and predict some metric about the program (accuracy, etc). Apparently, all PyTorch ...
user avatar
0 votes
0 answers
10 views

AI for text extraction and text recognition

Starting from text I'd like to be able to identify specific informations. Example : Input texts : "The invoice number is 18", "Inv : 75", "Inv N. : 84" Identified invoice ...
user avatar
  • 101
0 votes
0 answers
10 views

Is there a model for image pair duplicate detection?

Is there a deep learning model for duplicate image pair detection? Looks like I have to use a Siamese network for this. I have a dataset with image pairs with labelling that they are duplicates or not:...
user avatar
-2 votes
0 answers
20 views

Identifying country name with input as country national anthem audio sound wave file [closed]

Identifying country name with input given as country national anthem audio sound wave file. Identify the country name. This could be a good quiz question in General Knowledge (GK). What are the ...
user avatar
3 votes
1 answer
40 views

What is the total number of actions and rewards count

Reading this two articles about Reinforcement Learning: Deep Reinforcement Learning with Double Q-learning by Hado van Hasselt et al. Human-level control through deep reinforcement learning by ...
user avatar
  • 133
0 votes
0 answers
15 views

Build the architecture of an ensemble model for tabular classification

I have data that looks like this: ...
user avatar
  • 111
0 votes
0 answers
19 views

Why are separate, bigger Encoder-Decoder architectures used instead of Bidirectional RNNs/Transformers for Seq2Seq tasks?

Whether with RNNs or Transformers, Encoder-Decoder networks are used for Sequence to Sequence (Seq2Seq) tasks, like Machine Translation. Why are separate, bigger Encoder-Decoder networks used for this ...
user avatar
0 votes
1 answer
18 views

What is the right way to find the alphas in this equation?

In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function): $$ Y^c = \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\...
user avatar
  • 131
0 votes
1 answer
35 views

Is the Machine Learning community going against Occam's razor?

I have been using ML models, for a couple of years, but I am actually in the neuroscience field. In it, mathematical models try to assume the smaller number of things and make hypothesis as simple as ...
user avatar
  • 11
0 votes
0 answers
14 views

How to inform the autograd, my network structure includes RNN or LSTM? [closed]

Among all the Neural Network structures that are introduced, RNN has received noticeable attention because of the state art included in its gradient computation with backpropagation. On the other hand,...
user avatar
  • 1
1 vote
1 answer
35 views

Deep Features of Images - Better to use an unsupervised approach, or train a classifier with many classes?

I'd like to build a deep feature extractor of images (using a Bi-linear CNN). I was wondering what would lead to the best results: An unsupervised approach (such as https://iopscience.iop.org/article/...
user avatar
0 votes
0 answers
33 views

How to explain near zero gradients on first epochs?

As I understand the gradient should reflect how near the weights are to the optimal values. In this way i will expect that on the first epochs the gradients far from zero or at least not mostly zero ...
user avatar
0 votes
0 answers
39 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
user avatar
  • 131
4 votes
1 answer
58 views

Why is it important/beneficial for an activation function to be zero-meaned?

Conventionally, (although there are plenty of better options) it is being said that as the choice of activation function for hidden layers, tanh should be prefered ...
user avatar
0 votes
0 answers
9 views

Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?

I am looking at this lecture, which states (link to exact time): What the triplet loss allows us in contrast to the contrastive loss is that we can learn a ranking. So it's not only about similarity, ...
user avatar
  • 669
0 votes
1 answer
40 views

How to understand the GCN equation?

I understand GCN does message passing with its neighbours to learn the node embedding. But I don't understand the following equation. What "tilda" is referring to equation ...
user avatar
1 vote
0 answers
75 views

How are 4D cost volumes constructed for DL based stereo matching?

I read a paper on Stereo Matching using Pyramid Cost Volumes (paper link: Semantic Stereo Matching with Pyramid Cost Volumes). At some point, in the proposed architecture, after: Feature extraction ...
user avatar
0 votes
0 answers
19 views

How to generate data lying in the union of different hyperplanes using a VAE

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
user avatar
0 votes
0 answers
13 views

How to use a Siamese network at test time?

I am trying to understand Siamese networks, and understand how to train them. Once I have a trained network, I want to know if a new image is close or far to other images in the train set, and fail to ...
user avatar
  • 669
1 vote
0 answers
39 views

Learning values in open ball: which final layers to employ?

I'm fairly new to deep learning and looking for some reference literature... Specifically, I want to train a neural network to predict vectors $v \in \mathbb{R}^3$ under the constraint $||v||\leq 1$. ...
user avatar
  • 111
0 votes
0 answers
11 views

Vector to sequence RNNs: do they take a random initial "prompt"?

I am going through the Deep Learning book by Ian Goodfellow (here) and came by the architecture for a vector to sequence RNN (Figure 10.9). I am not sure I understand how this architecture works and ...
user avatar
0 votes
0 answers
23 views

Finer control over the distribution of output from a GAN

I have cast some data into an image (25 x 56) to work better with existing tools, and then used a CNN to train a GAN using the Wasserstein loss with gradient penalty to generate new samples of the ...
user avatar
  • 1
0 votes
0 answers
21 views

What is SE(3) transformer and what does RoseTTAFold use it for?

As is mentioned in its paper, the SE(3) transformer is a kind of self-attention-based structure that guarantees SE(3)-equivariance. So what is the reason that RoseTTAFold uses it and what for?
user avatar
  • 101
0 votes
0 answers
17 views

What are the differences between BLEU and METEOR?

I am trying to understand the concept of evaluating the machine translation evaluation scores. I understand how what BLEU score is trying to achieve. It looks into different n-grams like BLEU-1,BLEU-2,...
user avatar
0 votes
0 answers
22 views

how to define or calculate the similarity betweeen two curves as the loss funtion to optimize in the generative model?

I want to train a neural network as the curve productor that can generate the specific type of curves (e.g. exponential decay curves). I take the encoder-decoder structure, the curves in a dataset is ...
user avatar
0 votes
0 answers
7 views

Quick Lightweight Image Segmentation Model For Training on Custom COCO-Format Dataset

I'm trying to build a model for image segmentation on a Raspberry Pi. I have a dataset with annotations in the COCO-format that took a long time to build, so I'd prefer not to have to build another ...
user avatar
0 votes
1 answer
13 views

Object Classification: How to decide which detected region is a RoI for classification?

I am working on a project where I am working on the Flickr-47 dataset to do logo detection and classification. My approach is to first finetune a YOLO v5 model with high recall to detect as many "...
user avatar
0 votes
1 answer
34 views

1D Sequence Classification with self-supervised learning

I am working on a multi-class classification task on long one-dimensional sequences. The sequence length may vary in the range $[512, 30720]$, and there is one feature associated each time-step in the ...
user avatar
  • 101
1 vote
0 answers
21 views

Backprop to calculate mean and standard deviation in batch normalization?

On page 310 of the Deep Learning book by Ian Goodfellow (Page 310 can be viewed here for better context: https://www.deeplearningbook.org/contents/optimization.html ), it is mentioned that one crucial ...
user avatar
0 votes
0 answers
24 views

Determine Gridworld values

I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning. I am slightly confused in scenarios ...
user avatar
  • 123
1 vote
1 answer
29 views

How does SGD training error decrease in subsequent epochs when statistically, it requires that samples in subsequent epochs be i.i.d and they are not?

I have been reading the Deep Learning book by Ian Goodfellow and on pg. 277, they mention: It is also crucial that the minibatches be selected randomly. Computing an unbiased estimate of the expected ...
user avatar
0 votes
1 answer
25 views

Does make sense to add an additional Attention layer while Fine-Tuning Bert?

I'm fine tuning a Bert/Roberta model for a classification task. As I need to improve my results I'm thinking about to add an additional Attention layer after Bert Model and before Dense and Dropout ...
user avatar
0 votes
0 answers
20 views

Why does the feature space of an autoencoder typically contain more info than a teacher-student model?

This is a question our Prof gave us as exam preparation, but I don't know why the Autoencoder should contain more info than the Teacher Student model. Teacher Student Models are a class of models in ...
user avatar
  • 348
0 votes
0 answers
20 views

Is there any sort of 'best practice' for giving the general public access to deep learning models to accompany academic papers?

Is there any sort of 'best practice' for giving the general public access to deep learning models to accompany academic papers?
user avatar
1 vote
1 answer
118 views

What is the reason we loop over epochs when training a neural network?

After reading through this thread and some other resources online, I still do not understand the role of epochs in training a neural network. I understand that one epoch is one iteration through the ...
user avatar
0 votes
0 answers
16 views

Appropriate machine learning approach for object detection of connector

I am suppose to detect a female connector which happens to be an automotive part. I need to draw a bounding box around the connector when it appear. Here's the closest resemblance part I could find to ...
user avatar
0 votes
0 answers
12 views

How do I compute the convolution of two kernels of the same size in practice?

Suppose I have a 256-by-256 input matrix called $X$ and two 3-by-3 kernels called $K_1$ and $K_2$. By the associativity of convolution \begin{equation} (X \star K_1) \star K_2 = X \star (K_1 \star K_2)...
user avatar
  • 131
0 votes
0 answers
10 views

Does having more edges on a GNN helps learning?

I am doing a machine translation task using a Graph2Seq graph neutral network. I am using GAT as my encoder. Graph stats: I have around 400 nodes in the graph per data point. In the current graph, on ...
user avatar
0 votes
0 answers
11 views

Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
user avatar
0 votes
0 answers
28 views

Why is the vanishing gradient problem more discussed than the exploding gradient problem in the context of deep CNNs?

Both vanishing and exploding gradient problems may occur if we increase the number of layers in CNN. It is due to the product operation in the chain rule. Whenever I read recent research papers, i.e. ...
user avatar
  • 2,977
0 votes
1 answer
36 views

Do the values over 0.5 mean my model classified the data as a "1" label and vice versa?

I am doing binary classification using an LSTM and my output layer is 1 neuron with a sigmoid function. My labels are either 0 or 1. ...
user avatar
0 votes
1 answer
45 views

What is the best GNN for a NMT task?

I am doing a machine translation task using a Graph2Seq graph neutral network. There are many different variants of GNN: GCN GAT GraphSage GGNN Which one would be the most effective for a machine ...
user avatar
0 votes
0 answers
17 views

How to use copy mechanism and attention together?

Is the copy mechanism and attention related for a Neural Machine Translation task when source and target vocabulary are the same? Copy mechanism means unknown words would be copied from source to the ...
user avatar
0 votes
0 answers
12 views

Deep learning to fill sequence elements missing at random

I have the following problem setup: There is a list of floats (between -1 and 1) that is about 768*2 in length. The values of the floats are features that depend on two documents, the first 768 ...
user avatar
1 vote
1 answer
61 views

How to use structural information in a Transformer?

I am performing a Neural Machine Translation (NMT) task. In my case, input data has relational information. I know I can use a Graph Neural Network (GNN) and use a Graph2Seq model. But I can't find a ...
user avatar
0 votes
1 answer
22 views

Performance of augmented dataset with or without original images

I am training on yolo and I had a small dataset. I decided to increase it by augmenting it with rotation, shearing, etc to increase the size and increase accuracy. Now I have seen augmented datasets ...
user avatar
0 votes
1 answer
16 views

A technique of aggregating many input images to a single representation of the relevant features within

I have a few thousand images and I would like to generate a representation of the foreground patterns within - a composition of all of its features, so to speak. In simple terms: take 10000 images of ...
user avatar
  • 101
0 votes
0 answers
21 views

Neural network have difficulty on capturing weak characteristics

I want use neural network to approximate a non-linear function. The function is, $$ F(X1,X2,X3) = A \times X1^{K1} \times exp((X1-X2) \times K2) \times exp(X3 \times K3) $$ where X1/X2/X3 are input ...
user avatar
2 votes
1 answer
27 views

Why does backprop algorithm store the inputs to the non-linearity of the hidden layers?

I have been reading the Deep Learning book by Ian Goodfellow and it mentions in Section 6.5.7 that The main memory cost of the algorithm is that we need to store the input to the nonlinearity of the ...
user avatar

1
2 3 4 5
34