Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
2
votes
1answer
192 views

What do equations 1 and 3 describe in the "Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels" paper?

This paper uses image augmentation to improve RL algorithms. It contains the following paragraph - "Our approach, DrQ, is the union of the three separate regularization mechanisms introduced ...
0
votes
0answers
21 views

Feeding the output back to input in 3D CNN model

I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below I have two thoughts in training the model for above situation. Feed output $...
0
votes
0answers
34 views

How Long Short Term Memory (LSTM) work for time series classification?

I first got the concept of LSTM on how it works word to word prediction etc. However, I want to know how it work with the time-series classification. For example I have the follwing data (see image ...
2
votes
2answers
144 views

Not able to find a good fit for a simple function with neural networks

I have been trying to adjust a neural network to a simple function: the mass of an sphere. I have tried with different architectures, for example, a single hidden layer and two hidden layers, always ...
1
vote
0answers
47 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
0
votes
0answers
18 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
1
vote
1answer
554 views

Isn't attention mask for BERT model useless?

I have just dived into deep learning for NLP, and now I'm learning how the BERT model works. What I found odd is why the BERT model needs to have an attention mask. As clearly shown in this tutorial ...
1
vote
1answer
32 views

Are there any stats available on the usage of libraries by deep learning researchers?

I know three Python libraries that are popular in deep learning research community: Keras, PyTorch, Tensorflow. I don't know much about Theano. This question is not about the efficiency, flexibility ...
0
votes
1answer
54 views

Is it possible to use deep learning to generate a 2D image from a few numerical values?

Is it possible to train a DL model that will generate a full resolution 2D image based on few numbers describing this image and what type of model or architecture would that be? What I want to ...
1
vote
0answers
42 views

Computing the mean attention distance for ViT

Recently I came across the paper that introduces the Vision Transformer (ViT) "AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE". The thing I don't really ...
0
votes
0answers
11 views

Does it makes sense to use directed graphs for node classification tasks in graph neural network?

I was wondering if it does makes sense or not to use directed graphs for node classification tasks in graph neural network. Most of the successful methods for node classification tasks are the ones ...
0
votes
1answer
27 views

Train a deep learning model with input as a vector and predicts as a vector? [closed]

I am trying to build a Deep Learning model that takes a numeric vector $X$ of dimension $1 \times 50$ and predicts a numeric vector $y$ of dimension $1 \times 50$. It's a linear regression problem. I ...
0
votes
0answers
14 views

Predicting yes/no questions with input query

If I have information about a person e.g. their lifestyle, how big is their house, where they live, their income history, I can build a model to predict their future income. But what if my task is not ...
0
votes
0answers
6 views

The ratio between number of units in multi-input model

I have the model that accepts two inputs: Image from camera Speed of the car I can create some CNN layers to process the image input and some MLP layers to process other type of data (for example ...
1
vote
0answers
24 views

Would the reward normalization be wrong in early episodes?

It's confusing me that how can we normalize the reward without actually knowing the true mean and variance of the reward distribution, specifically, at the early steps and episodes. This may cause ...
0
votes
0answers
14 views

How to deal with unbalanced data in multilabel classification problem

I have 3 possible solutions, but I am not sure if they are good. I think they are a bit clunky (especially 1st and 2nd). Use multiple small models. So instead of having the model that can tell you ...
0
votes
0answers
12 views

How to approach receipt interpretation with neural networks

I have never worked with neural networks before, but I want to learn and I have a specific use case that I would want to solve (or at least try to). I have developed software for multiple years, in a ...
2
votes
1answer
50 views

What does "differentiable architecture" mean?

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", ...
0
votes
0answers
39 views

Why don't I get the same results of Q-Learning as in Aurélion Géron's Hands-on Machine Learning book?

I noticed something rather intriguing while testing the Deep Q-Network implementation from Aurélion Géron's book Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition; I copy-...
1
vote
1answer
129 views

What is the proper way to process continuous sequence data, such as time-series, using the Transformer?

What is the right way to input continuous, temporal (time-series) data into the Transformer? Assume we're using the basic TransformerBlock here. Since data is continuous with no tokens, Token ...
1
vote
1answer
54 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
0
votes
0answers
128 views

Computational complexity of a CNN network

In the following network, the convolution operations of convolutional blocks are performed by three 1-D kernels with the sizes 8, 5, and 3 respectively along with stride equal to 1. The final network ...
0
votes
0answers
37 views

tanh activation function output is not between -1 and 1 for continuous action PPO

I am using RLlib's (Ray = 1.4.0) PPO policy, and my first layer after the input (Conv layer) is producing a strange output keeping in mind that the activation for the output is a tanh, which I do ...
0
votes
0answers
37 views

Conditional input deep neural network

I need to input data conditionally to my deep network. In order to explain cases, I'd like to give an example. Assume that I have a 50-attribute dataset. For some attributes, a specific part of hidden ...
2
votes
2answers
216 views

How to determine the embedding size?

When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden ...
0
votes
0answers
29 views

What is the difference between model setup, model configuration, and model customization?

In the context of research papers related to deep learning models, the authors usually mention these terms in the experiment section when they are talking about the model: configuration, setup. For ...
1
vote
1answer
71 views

How much money is spent training neural networks each year by companies such as Google and Facebook?

I am wondering what order of magnitude estimates for the following are for companies Google and Facebook, as well as total globally. What is the rough amount of money spent to train neural networks? ...
0
votes
0answers
15 views

Why doesn't a neuron activation depend on number of input (presynaptic) neurons?

In an artificial neural network, we usually use the same activation function for all neurons, independently of the number of input (presynaptic) neurons. However, usually, the number of input neurons ...
0
votes
0answers
13 views

How to scale Computer Vision? How to implement Emotion detection from live video feed of N different video simultaneously?

I have a pipeline based on Scaled Yolov4 detection algorithm for faces which extract faces of users and then uses a CNN to ...
0
votes
0answers
25 views

Machine learning to find fewest number of "puzzle pieces" that fullfill a certain requirement

Would machine learning be suitable for the following problem, and if so, what kind of learning? I have numerous puzzle pieces, all having a value for identical properties. Example of one puzzle piece: ...
0
votes
0answers
13 views

What are some of the ideas to solve Learning-to-choose problem?

Suppose I want to predict cats and dogs, but with a twist: the model can choose the image to predict. For example: Given a list of 10 images (with both dogs and cats), the model need to choose one ...
0
votes
2answers
102 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
0
votes
0answers
17 views

Naming convention for deep learning layer sequences ("FC7", "Conv-1-3")

I was looking at the deep learning paper A Target-agnostic Attack on Deep Models and saw this figure (figure 3 on paper) demonstrating the performance of a transfer-learning-based adversarial attack ...
0
votes
0answers
17 views

How to manually optimize Neural Networks the most systematical way?

Do you have any ideas or guidance on how to do manual neural network optimization in the most systematic way? Especially when models train longer and the effects of hyperparameter fitting are very ...
0
votes
0answers
6 views

Learning a utility/cost function from user prefernces between sequences

I want to estimate the cost of certain sequences and I "know" a number of useful things about the cost. However the source of truth for this cost is actually a preference as stated by a user ...
0
votes
0answers
16 views

Looking for advice on how to train an AI object detection algorithm to recognize smaller objects than what it has been trained on

For some academic work, I am training an AI object detection algorithm (TensorFlow models) to look for specific objects (plants, in my case). I am taking photos with a hand-held camera, and am having ...
1
vote
1answer
94 views

What math should I learn before and while using and applying deep learning?

I want to learn deep learning. After researching a little, I came to the conclusion that I need a lot of math. I've started a linear algebra course, and it takes a long time (2-3 weeks). I want to ...
0
votes
0answers
7 views

Can I use transfer learning with Bert and none text sequential data?

I'm working on a multiclass classification problem, each row on my dataset have 5 time windows with 23 values on each time window, I would like to use transfer learning using the Bert transformer to ...
0
votes
0answers
14 views

Loss function to Push response value towards extremes

I have a feature map whose values are in the range of [0,1]. I want to push these values either towards extreme 0 or 1 using some loss function. Since I don't have any target value so it had to be in ...
0
votes
0answers
36 views

AI model to predict/generate person's image

I want to make a model that predicts person's shape depending on his son's image. My plan is to create a dataset and each data point in it consists of two images; One for the father or mother and one ...
2
votes
0answers
92 views

How would we get a good estimation of the asymptotic performance of machine learning algorithms?

The following question is from the webbook Neural Networks and Deep Learning by Michael Nielson: How do our machine learning algorithms perform in the limit of very large data sets? For any given ...
2
votes
1answer
399 views

How to add negative samples for object detection?

My question is: how to add certain negative samples to the training dataset to suppress those samples that are recognized as the object. For example, if I want to train a car detector. All my training ...
0
votes
0answers
29 views

How can a CNN be used in machine translation?

How can a convolutional neural net (CNN) be used in machine translation? Convolution is a mathematical operation, so how are natural languages translated into matrices? e.g., DeepL_Translator#...
0
votes
0answers
15 views

In CNN, how the conversion of convolution layer to fully connected layer decides the no. of kernel

I am trying to understand the shape of the activation map after every operation. Here is the model summary . All is clear, but from the point labeled 1, how 7x7x512 turns out to be 4096 specifically ...
0
votes
0answers
19 views

How is the size of the class activation maps related to the size of the input images?

According to the images of the block diagram of class activation maps on the Internet, it seems that these images are a weighted sum of feature maps of the last convolutional layer as shown in the ...
0
votes
1answer
46 views

How does CURL extract labels from logits? [closed]

While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
1
vote
1answer
24 views

What to do when the ROIs are smaller than $227 \times 227$ in R-CNN?

As English is not my native language, I have some hard time understanding the following sentence: Regardless of the size or aspect ratio of the candidate region, we warp all pixels in a tight ...
0
votes
0answers
48 views

GAN performs worse after 50 epochs than after 2

I am training GAN on SVHN dataset (house numbers in Google Street View images, dimensions: 3x32x32 - 3 color channels). The problem is that it performs worse after some training (e.g. after 50 epochs) ...
0
votes
0answers
11 views

What are the typical things in data that I would have to look for, when implementing survival models using machine learning?

Problem Scenario I am working on an industry-specific problem focussed on predicting the failure of a seal/gasket in the given time interval(T) in a high-pressure-compression environment. Whenever ...
0
votes
0answers
9 views

Evaluating a convolutional neural network on an imbalanced (academic) dataset

I have trained a posture analysis network to classify in a video of humans recorded in public places if there is a) shake-hand between two humans, b) Standing close together that their hands touch ...

1
3 4
5
6 7
33