Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Picking Order with expert knowledge: Modeling issues

I want to guess the optimal picking order. The orders have 10-50 positions and the optimal picking order not available. (The picking order depends on several things: (not every box is stackable; some ...
0
votes
0answers
14 views

Encoding Image Priors into CNN

There's a core problem with all of ML which I haven't really seen made explicit: the issue is every model needs to have an assumption on the structure of the data you learn and this assumption needs ...
0
votes
0answers
12 views

What are some applications of virtual try-on other than in the fashion industry?

I've been considering doing research in virtual try-on technology. There are various computer vision techniques that go into this, but I was wondering if there is any potential application of virtual ...
0
votes
0answers
45 views

Why is val accuracy 100% within 2 epochs and incorrectly predicting new images? (1,000 images per class when training)

My CNN tensorflow model reports 100% validation accuracy within 2 epochs. But it incorrectly predicts on single new images. (It is multiclass problem. I have 3 classes). How to resolve this? Can you ...
0
votes
0answers
18 views

How to predict using softmax having separate inputs and outputs?

I am new to Deep Learning. Having completed the coursera courses and read something from Deep Learning with Python, I am trying to implement one idea using DL. There is a number of user equipment (UE) ...
0
votes
0answers
9 views

Why batch normalization before upsampling is giving worse results?

I am training a model to generate images. I am applying the batch normalization layers just before upsampling and I am not getting the results that are at least comparable to the results by the model ...
2
votes
1answer
24 views

What is the stride information of an image referring here?

In convolutional neural networks, the convolution and pooling operations have a parameter known as stride, which decides the jump the kernel needs to do on the input image. You can get more ...
0
votes
0answers
11 views

tfp.Distributions.Categorical.sample() is picking the same action everytime after certain iterations

I have written a code for an RL agent such that at each state the model calculates the probabilities of all possible actions and samples one action randomly to proceed further. To acheive this, I have ...
0
votes
0answers
19 views

FrozenLake-v0 not training using REINFORCE

I am implementing a simple REINFORCE (policy gradient) algorithm for openAI's FrozenLake-v0 environment. However, it does not seem to learn anything at all. I have used the same neural architecture ...
3
votes
0answers
37 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
0
votes
0answers
28 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
2
votes
1answer
27 views

Feature Extraction for printer classification

I need some advice. I am currently trying to do a printer classification with ML/DL. What do I have? 11 colored-images with high resolution from 8 different inkjet-printers (in total 88 images) I have ...
2
votes
0answers
21 views

Carlini-Wagner attack - what are "objective functions"?

I'm trying to understand the paper by Carlini and Wagner on deep neural networks adversarial attacks. On page 44, in Section V-A, it is explained how loss function to the described problem was ...
-1
votes
0answers
32 views

Generative Models for Graphs (Training and Evaluation) [closed]

I am reading this paper Learning Deep Generative Models of Graphs, I understood the decision building process to generate new nodes and connect them to existing ones. but, I am stuck at section 4.2 ...
3
votes
0answers
17 views

Do any practical deep learning algorithms deal with tensors containing non-real entries?

In deep learning, most of the applications are from text and images. Both text and images can be converted into a tensor of real numbers. Other than both mentioned above, there may be some other real-...
-1
votes
1answer
36 views

Is it possible to train an RL agent using images?

I have an image which consists of a start and an end point, the journey has some obstacles which have to be avoided. Is it possible to train an RL agent using such images to find the best path ...
2
votes
1answer
24 views

How many unique angles of an object do you need in your image training set in order to correctly classify it?

I'm interested in using ResNet-50 to classify images of objects for around 1000 unique classes. I'm wondering if there is any way to estimate how many unique angles I need in my training set to ...
2
votes
0answers
16 views

Is it a good practice to split sparse from dense features?

I have a mixture of real (float) and categorical features to use as input in a neural network. I encode the categorical features using one-hot / multi-hot encoding. If I want to use all the features ...
3
votes
2answers
346 views

How should I read a deep learning paper?

I have a background in mathematics and I am accustomed to reading papers with lemma and proofs. When I see a deep learning paper, they seem to be of practical nature. How can I improve my reading and ...
0
votes
1answer
23 views

Is PositionalEncoding needed for using Transformer models correctly?

I am trying to make a model that uses a Transformer to see the relationship between several data vectors, but the order of the data is not relevant in this case, so I am not using the Positional ...
0
votes
1answer
47 views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
0
votes
0answers
18 views

Why do we run $QK^T$ in self-attention when it can be simplified?

$$ Q = \pmb x W^Q \\ V = \pmb x W^V $$ So $$ \begin{align*}\\ QV^T &= \pmb x W^Q (\pmb x W^V)^T \\ &= \pmb x W^Q(W^V)^T \pmb x^T \\ &= \pmb x M \pmb x^T \end{align*} $$ So you ...
2
votes
0answers
10 views

Can a GIoU loss (generalized intersection over union) be used after an STN module (spatial transformer network)?

I have a model that uses an STN module for number detection and Mean Squared Error loss. But I would like to replace it for GIoU, because MSE doesn't take into account how much of the target area has ...
0
votes
1answer
45 views

Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?

I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset ...
1
vote
2answers
51 views

Proximal Policy Optimization for continuous control problem

I am using clipped PPO to train a neural network to act as the controller for steering an aircraft, and am finding that my networks aren't learning. The goal is to keep the aircraft flying to cover ...
1
vote
1answer
38 views

How will MLOps and lifelong learning be complementary?

According to [1], in MLOps, continuous training is a new property, unique to ML systems, that's concerned with automatically retraining and serving the models. While lifelong/incremental learning ...
1
vote
1answer
25 views

Best practice for handling letterboxed images for non fully-convolutional deep learning networks?

I'm working on a depth estimation network. It has two outputs: A relative depth map A scalar for scaling the relative depth map into an absolute depth map. This second output uses dense layers so ...
0
votes
0answers
11 views

Self-Supervised learning model that labels the similar/not similar output?

I want to first reference the following SimCLR framework illustration to explain better what I'm asking. Lets say that after I found out of the image is not similar to the cat, can I actually predict ...
0
votes
1answer
56 views

how to decide the optimum model?

I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
2
votes
0answers
34 views

Decreasing number of neurons in CNN

the conventional way of creating a CNN is using increasing number of neurons: ...
1
vote
1answer
13 views

How to label unsupervised data for deep learning multi-classification

I have unlabeled credit card transaction data that has the following columns: ...
1
vote
1answer
58 views

What is the name of this letter $\mathcal{J}$?

What is the name of this letter $\mathcal{J}$ in the following deep learning equation? And what alphabet it is from? $$\mathcal{J} = \frac{1}{m} \sum_{i=1}^m \mathcal{L}^{(i)}$$
2
votes
0answers
76 views

Why does the number of input tokens to an LSTM have an impact on the convergence of Integrated Gradients?

Background I am computing the attribution scores for a simple LSTM model using Integrated Gradients. This method defines the contribution of a feature to a model prediction by integrating over the ...
0
votes
0answers
25 views

How to apply Deep Learning techniques to unlabeled data for Anomaly Detection

I'm fairly new to the field of deep learning and would like to ask which deep learning techniques can be used for anomaly detection in unlabeled data. For example, let's say I want to detect anomalous ...
1
vote
0answers
28 views

Is there any deepfake detectors with multiple deep learning models in the classifier component?

I observed that the deepfake detectors are of two types as Deep learning-based (DL-based) and machine learning-based (Non-DL methods) models. In those DL-based deepfake detectors, the model consists ...
1
vote
1answer
17 views

Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1, and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
2
votes
1answer
42 views

What are the major layers in a Vision Transformer?

Currently, I am studying deepfake detection using deep learning methods. Convolution neural networks, recurrent neural networks, long-short term memory networks, and vision transformers are famous ...
1
vote
0answers
50 views

What is the most suitable measure of the distance between two VAE's latent spaces?

The problem I'm trying to solve is as follows. I have two separate domains, where inputs do not have the same dimensions. However, I want to create a common feature space between both domains using ...
1
vote
0answers
13 views

What is the best way to train a text-based regressor model?

I want to build a deep learning model that can predict a continuous value (LogP in this case) given text inputs (SMILES notations in this case), the dataset is as illustrated below. SMILES notations ...
1
vote
0answers
66 views

What is uncentered variance and how it becomes equal to mean square in Adam?

I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean In this statement, the author is talking about ...
1
vote
0answers
30 views

Why does research on faster Transformers focus on the query-key product?

A lot of recent research on Transformers has been devoted to reducing the cost of the self-attention mechanism: $$\text{softmax}\left(\frac{Q K^T}{\sqrt{d}} \right)V,$$ As I understand it, the runtime,...
1
vote
0answers
79 views

Understanding gumbel-softmax backpropagation in Wav2Vec papers

I'm studying the series of Wav2Vec papers, in particular, the vq-wav2vec and wav2vec 2.0, and have a problem understanding some details about the quantization procedure. The broader context is this: ...
2
votes
1answer
29 views

Can I use the transformers for the prediction of historical data?

Can I use the transformers for the prediction of wind power with the historical data? Dataset Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind ...
3
votes
3answers
106 views

Should I allow NN to infer relationships of inputs?

This question is assuming a sequential, deep neural network Given some features [X1, X2, ... Xn], I'm trying to predict some value Y. The raw data available to me contains feature ...
0
votes
0answers
24 views

What can be recommended reputed deep learning journal(s) or conference(s) that do accept architectures in comparatively less time?

In the domain of deep learning, architectures are highly important. Many research journals and conferences do accept the proposed architectures as the contribution. Since the domain of deep learning ...
3
votes
1answer
38 views

Is there any reason behind bias towards max pooling over avg pooling?

Consider the following excerpt taken from the chapter named Using convolutions to generalize from the textbook titled Deep Learning with PyTorch by Eli Stevens et al. Downsampling could in principle ...
1
vote
0answers
17 views

What are the defining moments that make community realise the potential of deep learning?

Consider the following paragraph from the chapter named pre-trained models from the textbook titled Deep Learning with PyTorch by Eli Stevens et al. The AlexNet architecture won the 2012 ILSVRC by a ...
2
votes
2answers
47 views

When would it make sense to perform a gradient descent step for each term of a loss function with multiple terms?

I am training a neural network using a mini-batch gradient descent algorithm. Now, consider the following loss function, which is composed of 2 terms. $$L = L_{\text{MSE}} + L_{\text{regularization}} \...
1
vote
0answers
31 views

What is the effect of gradient clipping by norm on the performance of a model?

It is recommended to apply gradient clipping by normalization in case of exploding gradients. The following quote is taken from here answer One way to assure it is exploding gradients is if the loss ...
0
votes
0answers
23 views

Is the multi-head attention in the transformer a weighted adjacency matrix?

Are multi-head attention matrices weighted adjacency matrices? The job of the multi-head-attention mechanism in transformer models is to determine how likely a word is to appear after another word. In ...

1
2 3 4 5
34