Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0
votes
1answer
27 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
0
votes
0answers
25 views

How to predict the possible next moves of cars from given first moves?

I want to find the next moves of cars from the previous moves, but I could not figure out what should I use as an algorithm. Can you help me to find a way to solve this problem? I have a lot of car ...
4
votes
2answers
72 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
0
votes
1answer
72 views

Decision boundary figure in Least square GAN paper

I currently reading Least Square GAN paper. But, I cannot interpret one of its figures. . Explanation of the figure goes like this: Figure 1: Illustration of different behaviors of two loss functions....
1
vote
0answers
30 views

Reason why chess neural network might not be training

I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked. Input: ...
1
vote
1answer
130 views

How to normalize for perceptual loss when training neural net from scratch?

Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) ...
2
votes
1answer
50 views

What is the stride information of an image referring here?

In convolutional neural networks, the convolution and pooling operations have a parameter known as stride, which decides the amount of jump the kernel needs to do on the input image. You can get more ...
2
votes
1answer
807 views

Is my understanding of how the convolution with stride 2 works in this example correct?

I'm currently reading this explanation of convolutional neural networks and there's a part around strides that I don't quite understand. I'm just starting with this, so I apologize if this is a really ...
0
votes
1answer
68 views

How to retrain a Facenet model with the triplet loss function?

I want to calculate the similarity or distance of two faces. I'm using Python. I have read and done what this tutorial says. However, the result is not good (the similarity of same faces and ...
0
votes
1answer
41 views

When can we call a feature "hierarchical"?

Features in machine learning are the attributes of the elements of a data set. They are considered as random variables in probability. Consider the following excerpt from 1.1: The deep learning ...
1
vote
0answers
7 views

Output representation for a neural network to learn grid-based game with multiple units

I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
0
votes
1answer
39 views

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
0
votes
1answer
12 views

What does a value of -1.000 mean in MS COCO Metrics for Object Detection

I am training some Object-Detection-Models from the TensorFlow Object Detection API and got from the evaluation with MS COCO metrics the following results for Average Precision: IoU = 0.5;0.9 maxDets =...
3
votes
2answers
648 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
1
vote
0answers
14 views

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours ...
0
votes
0answers
17 views

What is the motivation behind NAS Bench 201 research?

I recently read the "paper NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search", which can be found here. I can say that I understood most of the paper but I am not ...
1
vote
2answers
23 views

How can equivariance to translation be a benefit of a CNN?

I just learnt about the properties of equivariance and invariance to translation and other transformations. Being invariant to translation is clearly an advantage, as even if the input gets shifted, ...
21
votes
3answers
4k views

Why do most deep learning papers not include an implementation?

I'm a novice researcher, and as I started to read papers in the area of deep learning I noticed that the implementation is normally not added and is needed to be searched elsewhere, and my question is ...
0
votes
1answer
60 views

Can the attention mechanism improve the performance in the case of short sequences?

I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise. ...
0
votes
1answer
40 views

Equations for computing true positives and false positives when using object detection algorithms?

I am running some evaluation metrics using the YOLOv5 object detection algorithm, and wish to calculate my true positives and false positives. For instance, the evaluation metric outputs are as ...
1
vote
1answer
49 views

Does a better discriminator in GANs mean better sample generation by the generator?

Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, ...
1
vote
2answers
349 views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
1
vote
1answer
71 views

In the update rule of RMSprop, do we divide by a matrix?

I've been trying to understand RMSprop for a long time, but there's something that keeps eluding me. Here is a screenshot from this video by Andrew Ng. From the element-wise comment, from what I ...
1
vote
1answer
64 views

Are there fundamental learning theories for developing an AI that imitates human behavior?

Most, if not all, AI systems do not imitate humans. Some of them out-perform humans. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. Those ...
0
votes
1answer
156 views

What is uncentered variance and how it becomes equal to mean square in Adam?

I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean In this statement, the author is talking about ...
1
vote
1answer
39 views

Does reaching the global optima guarantee good performance in a task?

It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. ...
2
votes
1answer
297 views

Finding patterns in binary files using deep learning

I am a newbie in deep learning and wanted to know if the problem I have at hand is a suitable fit for deep learning algorithms. I have thousands of fragments each of about 1000 bytes size (i.e. ...
0
votes
1answer
50 views

Attention mechanism: Why apply multiple different transformations to obtain query, key, value

I have two questions about the structure of attention modules: Since I work with imagery I will be talking about using convolutions on feature maps in order to obtain attention maps. If we have a set ...
0
votes
1answer
26 views

How can I adapt a trained neural network model to learn from newer data containing additional features?

We shall assume that we have a trained neural network model for some task $A$. The dataset used to train the model contained some $n$ features per sample. Using this dataset, we were able to train a ...
2
votes
1answer
1k views

Why can the learning rate make the loss increase in stochastic gradient descent?

In Deep Learning by Goodfellow et al., I came across the following line on the chapter on Stochastic Gradient Descent (pg. 287): The main question is how to set $\epsilon_0$. If it is too large, the ...
3
votes
1answer
134 views

How can I build an AI with NLP that read stories [closed]

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet. The idea is as follows: Let's say we have a story (in the text) that has 10 ...
0
votes
0answers
30 views

Limit of momentum update equation

I am self-studying on optimization algorithm on https://d2l.ai/chapter_optimization/momentum.html and couldn't get my head around some derivation: Instead of the standard gradient descent update ...
3
votes
2answers
78 views

What is the difference between batch and mini-batch gradient decent?

I am learning deep learning from Andrew Ng's tutorial Mini-batch Gradient Descent. Can anyone explain the similarities and dissimilarities between batch GD and mini-batch GD?
0
votes
0answers
34 views

Deep learning and machine learning [duplicate]

If I was Given a set of large training examples (xi,yi), how can training a neural network (NN) via stochastic gradient descent differs from using regular gradient descent in terms of the mathematical ...
0
votes
1answer
59 views

How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
1
vote
1answer
46 views

Training and Evaluating BERT and XLNET

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice. In the project, I have been given a data set with (...
2
votes
2answers
152 views

What is the difference between game theory and machine learning?

What is the difference between game theory and machine learning? I had gone through the papers Deep Learning for Predicting Human Strategic Behavior, by Jason Hartford et al., and When Machine ...
0
votes
1answer
29 views

Does "number of actions" refer to the number of actions taken or size of the action space?

In the original DDQN article (https://arxiv.org/pdf/1509.06461.pdf,) the phrase "number of actions" is used twice; First, in the following context: Secondly in Theorem 1. I have a hard ...
4
votes
0answers
1k views

Which other loss functions for hierarchical multi-label classification could I use?

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron (MLP) branch ...
3
votes
2answers
563 views

Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)

Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision ...
1
vote
1answer
58 views

How to calculate the precision and recall given the predictions and targets in this case?

I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a ...
4
votes
2answers
303 views

Is AlphaFold just making a good estimate of the protein structure?

In the news, DeepMind's AlphaFold is said to have solved the protein folding problem using neural networks, but isn't this a problem only optimised quantum computers can solve? To my limited ...
0
votes
1answer
27 views

What is the difference between Mean Teacher and Knowledge Distillation?

I recently read two papers: BYOL Bootstrap your own latent: A new approach to self-supervised Learning DINO Emerging Properties in Self-Supervised Vision Transformers. I am confused about the terms ...
21
votes
2answers
22k views

What are "bottlenecks" in neural networks?

What are "bottlenecks" in the context of neural networks? This term is mentioned, for example, in this TensorFlow article, which also uses the term "bottleneck values". How does ...
1
vote
2answers
48 views

How to make NN distinguish between two types of functions (data)?

I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict ...
1
vote
1answer
571 views

Machine learning to predict 8*8 matrix values using three independent matrices

Problem Statement I have 4 main input features. This is a small snippet of the data for clearer understanding. Gate name -> for example AND Gate index_1 -> ...
0
votes
0answers
39 views

Is it effective to use deep learning method to produce a 1D signal as output from a 2D image as input?

I have a 1D signal that will produce a 2D image after some image processing algorithm. Would it be possible and effective to use deep learning method to reproduce the 1D signal if I have the 2D image ...
4
votes
1answer
167 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
0
votes
0answers
29 views

How to use EfficientDet for semantic segmentation?

In the EfficientDet paper, section 5.2. 5.2. EfficientDet for Semantic Segmentation, the authors say we modify our EfficientDet model to keep feature level $\{P2, P3, ..., P7\}$ in BiFPN, but only ...
4
votes
1answer
162 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable-length input padded with 0. However, the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...

1
2 3 4 5
33