Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
1 vote
0 answers
6 views

How to tackle the Human Error made in labeling datasets for Classifcation Tasks like Facial Expression Recognition?

I am working on the Facial Expression Recognition Task. One of the most challenging tasks that I faced was Human Error in labeling the datasets (ex: let's say FER2013). Are there anyways to Handle ...
0 votes
1 answer
32 views

What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
12 votes
3 answers
505 views

Can some one help me understand this paragraph from Nvidia's progressive GAN paper?

In the paper Progressive growing of gans for improved quality, stability, and variation (ICLR, 2018) by Nvidia researchers, the authors write Furthermore, we observe that mode collapses traditionally ...
0 votes
0 answers
6 views

Deep learning feature extraction using image processing batch script?

I am planning a CNN deep learning project using photos of handwritten notes, and try to label them. I am still at the early stages, but I expect that accuracy of this neural net would improve, and ...
0 votes
0 answers
4 views

Is it possible to embed a neural network layer into decision tree/random forest?

I want to do a classification task. I designed a customed layer for it. I also want to try decision tree/random forest, but as far as I know there is no way to embed my layer into a decsion tree/...
0 votes
1 answer
38 views

What techniques exist to increase the learning importance of difficult-to-learn labels over easy ones?

I am training a model to place labels in image data. Some labels are learnt very quickly by the model while others take a long time to perfect. I cannot simply add more labeled data with only the ...
1 vote
2 answers
19 views

How to properly name given type of classification problem?

What is the proper technical name of the classification problem where each data sample can be classified according to two different criteria and each of them can have two or more classes? For example ...
0 votes
1 answer
15 views

Using GraphSAGE model for multigraph datasets

I checked out applications of GraphSAGE and it seems like its primarily used for single graph datasets. For example - Cora dataset - Its one big graph with 2708 nodes and 5429 edges. The model can ...
0 votes
0 answers
6 views

Change fully connected layer for YOLO v4 network , analogous to GoogLeNet (matlab)

I have a challenging task here. I would like to train a YOLOv4 - network, but with the following adjustment: In my previous implementation of CNNs, a pre-trained network was loaded, and the last layer ...
1 vote
0 answers
17 views

How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?

Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
0 votes
1 answer
56 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
0 votes
0 answers
11 views

What are "volatile" learning curves indicative of?

I have a dataset set with ~40 features onto which I'm applying a multi-layer perceptron for regression purposes. The train, validation, and test sets are made up of 3M, 800K, and 800K examples each, ...
0 votes
2 answers
117 views

Is there any relationship between the batch size and the number of epochs?

I am currently running a program with a batch size of 17 instead of batch size 32. The benchmark results are obtained at a batch size of 32 with the number of epochs 700. Now I am running with batch ...
0 votes
1 answer
69 views

How do I create an AI controller for Pacman? [closed]

How do I create an AI controller, which can play pacman - by taking in pixel values (or some other data by represents the state) which perhaps runs on a separate thread, which can control the game? It ...
1 vote
0 answers
21 views

Why can't I reproduce my results in keras using random seed? [closed]

I was doing a task using RNN to predict a time series movement. I want to make my results reproducible. So I strictly followed this post: https://stackoverflow.com/questions/32419510/how-to-get-...
2 votes
1 answer
92 views

Is batch learning with gradient descent equivalent to "rehearsal" in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
3 votes
0 answers
141 views

Train, Validation and Test Split for Reporting Accuracy of Neural Model and BOW

I need to report the accuracies of my neural model in a conference paper as compared to various baselines. What are the accepted standards for reporting accuracies in a fair manner? Neural Model: To ...
3 votes
1 answer
47 views

Why are embeddings added, not concatenated?

Let's consider the following example from BERT I cannot understand why "the input embeddings are the sum of the token embeddings, the segmentation embeddings, and the position embeddings". ...
1 vote
1 answer
27 views

How does YOLO detect the object when the object is in multiple grid cells?

I have been reading various articles and watching videos on YouTube, but I can't seem to understand one thing. How does YOLO make a bounding box for an object if it is in multiple grid cells? For ...
0 votes
1 answer
48 views

How to compare memory requirements for tabular Q-learning vs deep neural network?

I want to compare the space complexity/memory requirement of tabular Q-learning v.s. deep neural Q-network (DQN). I think DQN would be faster and Q-table has a disadvantage at large table sizes but ...
1 vote
1 answer
27 views

Is the described Q-table considered large?

I never saw any rule of thumb as to what size is said as large for a q-table but I have a Q-table with like 2500 entries. Is it considered large for a tabular approach? Anyone from experience can ...
0 votes
0 answers
26 views

How to get into robotic simulation for RL purposes? [closed]

For my master's thesis, I've joined a robotics team that tries to build a flying robot based on the mechanism of flies (i.e - two wings that perform reciprocating motion to generate lift). My part in ...
0 votes
0 answers
20 views

Why is it difficult to train large RL networks?

First of all I know that: 'it makes training less stable' & 'RL is already inherently unstable'. I'm asking why those things are true? Intuitively it seems very strange & to be perhaps a ...
0 votes
0 answers
20 views

Is 3D pose prediction directly from point clouds commonplace?

I've spent a couple of days experimenting and trying to find papers on learning 3D pose directly from unordered point clouds with no color information. This paper from 2020 claims to be the first that ...
0 votes
0 answers
9 views

Is there a state-of-the-art deep learning paper that uses center point regression instead of bounding box regression, for object tracking?

Almost all deep learning based object tracking methods perform bounding box regression. Siamese-based networks which are very popular for object tracking also perform bounding box regression most of ...
1 vote
2 answers
126 views

Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?

I am using the following architechture: 3*(fully connected -> batch normalization -> relu -> dropout) -> fully connected Should I add the ...
1 vote
1 answer
65 views

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours ...
0 votes
2 answers
376 views

What is meant by an axis of a tensor?

Tensor is an ordered collection of elements. The elements are generally real numbers. Tensors are used in deep learning for storing data. There is a wide usage of the word "axis" related to ...
1 vote
0 answers
19 views

Model Agnostic Meta Learning on custom data

I have been trying to train the model agnostic meta learning model on a custom dataset with 100 classes and 5 examples in each class from here The structure of my data is: ...
1 vote
1 answer
35 views

What is the difference between fine tuning and variants of few shot learning? [duplicate]

I am trying to understand the concept of fine-tuning and few-shot learning. I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, ...
1 vote
1 answer
22 views

How to instruct Mask RCNN to identify objects too close to each other?

I have been trying to train a Mask RCNN model to identify individual poker chips in a stack. No matter what property I change, the end results look like the following image. I was guessing the issue ...
4 votes
2 answers
80 views

What makes a transformer a transformer?

Transformers are modified heavily in recent research. But what exactly makes a transformer a transformer? What is the core part of a transformer? Is it the self-attention, the parallelism, or ...
0 votes
1 answer
302 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
4 votes
1 answer
191 views

Will there be some promising techniques that can make AI greener and affordable in the future?

The recent advances in machine learning were mostly achieved by the hardware, and the hardware is said to continue driving the development of AI, but I was still shocked by this thread which reads ...
0 votes
1 answer
23 views

How to justify the chosen neural architecture?

I had a task to implement a neural network that would carry out multiclass classification of traffic by several parameters. On the advice of colleagues, I chose the "Multilayer Perceptron" ...
3 votes
1 answer
184 views

When to use RMSE as opposed to MSE and vice versa?

I understand that RMSE is just the square root of MSE. Generally, as far as I have seen, people seem to use MSE as a loss function and RMSE for evaluation purposes, since it exactly gives you the ...
0 votes
1 answer
177 views

How "Patch Merging" works in SWIN-Transformers?

In the SOTA paper: SWIN-Transformers, the authors have tried their best to explain everything clearly. I have got an idea of how it works except the Patch Merging ...
0 votes
1 answer
57 views

Is the Machine Learning community going against Occam's razor?

I have been using ML models, for a couple of years, but I am actually in the neuroscience field. In it, mathematical models try to assume the smaller number of things and make hypothesis as simple as ...
2 votes
2 answers
353 views

Finding patterns in binary files using deep learning

I am a newbie in deep learning and wanted to know if the problem I have at hand is a suitable fit for deep learning algorithms. I have thousands of fragments each of about 1000 bytes size (i.e. ...
1 vote
1 answer
53 views

Does reaching the global optima guarantee good performance in a task?

It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. ...
0 votes
1 answer
63 views

How can I adapt a trained neural network model to learn from newer data containing additional features?

We shall assume that we have a trained neural network model for some task $A$. The dataset used to train the model contained some $n$ features per sample. Using this dataset, we were able to train a ...
0 votes
1 answer
590 views

Deep Q-Learning with multiple discrete actions

I am working on a DQN project with Pytorch, where I should choose multiple discrete actions, each in a range, say, (0, 15). I am wondering how I can model it, such ...
1 vote
1 answer
55 views

How can I generalize a machine learning model to multiple curves?

I have a family of convergence curves as you can see in the image below: I would like to train a model that fits reasonably well to all the curves at the same time in my dataset. Is it possible? Do ...
-1 votes
1 answer
66 views

How to use a trained neural network to find optimal function inputs? [closed]

I have a deep neural network with 4 input nodes, 4 hidden layers with 4 nodes each and 1 output layer with one node (TRUE, FALSE). I have already trained the NN using backpropagation because I have ...
0 votes
1 answer
86 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
0 votes
1 answer
47 views

How to explain near zero gradients on first epochs?

As I understand the gradient should reflect how near the weights are to the optimal values. In this way i will expect that on the first epochs the gradients far from zero or at least not mostly zero ...
0 votes
1 answer
120 views

What is the difference between Mean Teacher and Knowledge Distillation?

I recently read two papers: BYOL Bootstrap your own latent: A new approach to self-supervised Learning DINO Emerging Properties in Self-Supervised Vision Transformers. I am confused about the terms ...
4 votes
3 answers
1k views

Which neural network to use for optical mark recognition?

I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons each, each layer with a ReLU layer ...
5 votes
2 answers
4k views

Are neural networks statistical models?

By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models. In contrast Machine Learning is not just glorified Statistics. I am looking ...
4 votes
1 answer
304 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...

1
2 3 4 5
35