Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

Are there any recommendations on initialising a single parameter in deep learning?

I want to initialize a parameter, which is a single real number in my model. If you want the role of the parameter in the model, you can assume it as the parameter to multiply with the output of the ...
0
votes
0answers
19 views

Fine tuning BERT for token level classification

I want to try self-supervised and semi-supervised learning for my task, which relates to token-wise classification for the 2 sequences of sentences (source and translated text). The labels would be ...
0
votes
0answers
11 views

preprocessing of time series data, each line consist of a time series

Imagine having a dataset (almost 100000 observations) composed of 365 columns (1 year) and each index (or observation) will then be representing time-series data. In my case, each observation (time-...
0
votes
0answers
33 views

Predict placement of an object in 3D space

I am trying to find a way to train a model to predict the correct placement of entities like a tree, dog and cat in a natural 3D environment. Any help regarding how I could use textual data to learn ...
0
votes
0answers
19 views

Generate gaming sequences of inputs like how GANs do for art

I need some insight on a subject given the fact that I'm not a researcher, but I am a software engineer. I want to build a model that would recommend (or generate) different paths (play sequences) in ...
0
votes
1answer
25 views

Best way to resize 3d to 2d matrix

I have a (5, 128, 768) matrix, that is, I have 5 embedding spaces of shape (128, 768). Since they all keep a relation, and for ...
0
votes
0answers
14 views

Proper loss function for regression with uniform target distribution

I'm doing some simulations and I would like to estimate a real number that is uniformly distributed between minValue and maxValue...
1
vote
1answer
44 views

Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?

I am looking at appendix C of the VAE paper: It says: C.1 Bernoulli MLP as decoder In this case let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Bernoulli whose ...
0
votes
1answer
68 views

What is the definition of a trace of a tensor?

Tensor is a multi-dimensional ordered collection of elements, which is used in deep learning to store and process data as well as intermediate steps. We are aware of the trace of a two-dimensional ...
0
votes
1answer
32 views

Non-sliding kernels for location-aware processing in Convolutional Neural Networks

My understanding of how CNN operates in image detection is through the use of kernels that slide through the image to detect features (edges and so on). So a single kernel could potentially be ...
0
votes
0answers
28 views

Object detection: when there's only 1 object in each image

Good day. I have a custom dataset for object detection, which has imbalance that each image has only one object annotation. I trained the object detection model(Efficientdet-dx) on TensorFlow object ...
0
votes
0answers
15 views

Picking Order with expert knowledge: Modeling issues

I want to guess the optimal picking order. The orders have 10-50 positions and the optimal picking order not available. (The picking order depends on several things: (not every box is stackable; some ...
0
votes
0answers
24 views

Encoding Image Priors into CNN

There's a core problem with all of ML which I haven't really seen made explicit: the issue is every model needs to have an assumption on the structure of the data you learn and this assumption needs ...
0
votes
0answers
18 views

What are some applications of virtual try-on other than in the fashion industry?

I've been considering doing research in virtual try-on technology. There are various computer vision techniques that go into this, but I was wondering if there is any potential application of virtual ...
0
votes
0answers
70 views

Why is val accuracy 100% within 2 epochs and incorrectly predicting new images? (1,000 images per class when training)

My CNN tensorflow model reports 100% validation accuracy within 2 epochs. But it incorrectly predicts on single new images. (It is multiclass problem. I have 3 classes). How to resolve this? Can you ...
0
votes
0answers
126 views

How to predict using softmax having separate inputs and outputs?

I am new to Deep Learning. Having completed the coursera courses and read something from Deep Learning with Python, I am trying to implement one idea using DL. There is a number of user equipment (UE) ...
0
votes
0answers
22 views

Why batch normalization before upsampling is giving worse results?

I am training a model to generate images. The model contains 5+5 layers: ...
3
votes
1answer
71 views

What is the stride information of an image referring here?

In convolutional neural networks, the convolution and pooling operations have a parameter known as stride, which decides the amount of jump the kernel needs to do on the input image. You can get more ...
0
votes
0answers
34 views

tfp.Distributions.Categorical.sample() is picking the same action everytime after certain iterations

I have written a code for an RL agent such that at each state the model calculates the probabilities of all possible actions and samples one action randomly to proceed further. To acheive this, I have ...
0
votes
0answers
24 views

FrozenLake-v0 not training using REINFORCE

I am implementing a simple REINFORCE (policy gradient) algorithm for openAI's FrozenLake-v0 environment. However, it does not seem to learn anything at all. I have used the same neural architecture ...
3
votes
0answers
51 views

Are there neural networks with (hard) constraints on the weights?

I don't know too much about Deep Learning, so my question might be silly. However, I was wondering whether there are NN architectures with some hard constraints on the weights of some layers. For ...
0
votes
2answers
44 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
1
vote
1answer
37 views

Feature Extraction for printer classification

I need some advice. I am currently trying to do a printer classification with ML/DL. What do I have? 11 colored-images with high resolution from 8 different inkjet-printers (in total 88 images) I have ...
1
vote
0answers
36 views

Where do the objective functions proposed in this paper by Carlini-Wagner attack come from?

I'm trying to understand the paper by Carlini and Wagner on deep neural networks adversarial attacks. On page 44, in Section V-A, it is explained how the loss function to the described problem was ...
2
votes
0answers
19 views

Do any practical deep learning algorithms deal with tensors containing non-real entries?

In deep learning, most of the applications are from text and images. Both text and images can be converted into a tensor of real numbers. Other than both mentioned above, there may be some other real-...
-2
votes
1answer
42 views

Is it possible to train an RL agent using images?

I have an image which consists of a start and an end point, the journey has some obstacles which have to be avoided. Is it possible to train an RL agent using such images to find the best path ...
2
votes
1answer
28 views

How many unique angles of an object do you need in your image training set in order to correctly classify it?

I'm interested in using ResNet-50 to classify images of objects for around 1000 unique classes. I'm wondering if there is any way to estimate how many unique angles I need in my training set to ...
2
votes
0answers
18 views

Is it a good practice to split sparse from dense features?

I have a mixture of real (float) and categorical features to use as input in a neural network. I encode the categorical features using one-hot / multi-hot encoding. If I want to use all the features ...
4
votes
2answers
395 views

How should I read a deep learning paper?

I have a background in mathematics and I am accustomed to reading papers with lemma and proofs. When I see a deep learning paper, they seem to be of practical nature. How can I improve my reading and ...
0
votes
1answer
45 views

Is Positional Encoding always needed for using Transformer models correctly?

I am trying to make a model that uses a Transformer to see the relationship between several data vectors, but the order of the data is not relevant in this case, so I am not using the Positional ...
0
votes
1answer
52 views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
0
votes
0answers
20 views

Why do we run $QK^T$ in self-attention when it can be simplified?

$$ Q = \pmb x W^Q \\ V = \pmb x W^V $$ So $$ \begin{align*}\\ QV^T &= \pmb x W^Q (\pmb x W^V)^T \\ &= \pmb x W^Q(W^V)^T \pmb x^T \\ &= \pmb x M \pmb x^T \end{align*} $$ So you ...
2
votes
0answers
12 views

Can a GIoU loss (generalized intersection over union) be used after an STN module (spatial transformer network)?

I have a model that uses an STN module for number detection and Mean Squared Error loss. But I would like to replace it for GIoU, because MSE doesn't take into account how much of the target area has ...
0
votes
1answer
49 views

Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?

I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset ...
1
vote
2answers
52 views

Proximal Policy Optimization for continuous control problem

I am using clipped PPO to train a neural network to act as the controller for steering an aircraft, and am finding that my networks aren't learning. The goal is to keep the aircraft flying to cover ...
2
votes
1answer
50 views

How will MLOps and lifelong learning be complementary?

According to [1], in MLOps, continuous training is a new property, unique to ML systems, that's concerned with automatically retraining and serving the models. While lifelong/incremental learning ...
2
votes
1answer
29 views

Best practice for handling letterboxed images for non fully-convolutional deep learning networks?

I'm working on a depth estimation network. It has two outputs: A relative depth map A scalar for scaling the relative depth map into an absolute depth map. This second output uses dense layers so ...
0
votes
0answers
13 views

Self-Supervised learning model that labels the similar/not similar output?

I want to first reference the following SimCLR framework illustration to explain better what I'm asking. Lets say that after I found out of the image is not similar to the cat, can I actually predict ...
-1
votes
1answer
59 views

how to decide the optimum model?

I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
2
votes
0answers
36 views

Decreasing number of neurons in CNN

the conventional way of creating a CNN is using increasing number of neurons: ...
0
votes
1answer
16 views

How to label unsupervised data for deep learning multi-classification

I have unlabeled credit card transaction data that has the following columns: ...
1
vote
1answer
61 views

What is the name of this letter $\mathcal{J}$?

What is the name of this letter $\mathcal{J}$ in the following deep learning equation? And what alphabet it is from? $$\mathcal{J} = \frac{1}{m} \sum_{i=1}^m \mathcal{L}^{(i)}$$
1
vote
0answers
85 views

Why does the number of input tokens to an LSTM have an impact on the convergence of Integrated Gradients?

Background I am computing the attribution scores for a simple LSTM model using Integrated Gradients. This method defines the contribution of a feature to a model prediction by integrating over the ...
0
votes
0answers
28 views

How to apply Deep Learning techniques to unlabeled data for Anomaly Detection

I'm fairly new to the field of deep learning and would like to ask which deep learning techniques can be used for anomaly detection in unlabeled data. For example, let's say I want to detect anomalous ...
0
votes
0answers
36 views

Is there any deepfake detectors with multiple deep learning models in the classifier component?

I observed that the deepfake detectors are of two types as Deep learning-based (DL-based) and machine learning-based (Non-DL methods) models. In those DL-based deepfake detectors, the model consists ...
0
votes
1answer
17 views

Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1, and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
2
votes
1answer
51 views

What are the major layers in a Vision Transformer?

Currently, I am studying deepfake detection using deep learning methods. Convolution neural networks, recurrent neural networks, long-short term memory networks, and vision transformers are famous ...
0
votes
0answers
73 views

What is the most suitable measure of the distance between two VAE's latent spaces?

The problem I'm trying to solve is as follows. I have two separate domains, where inputs do not have the same dimensions. However, I want to create a common feature space between both domains using ...
1
vote
0answers
13 views

What is the best way to train a text-based regressor model?

I want to build a deep learning model that can predict a continuous value (LogP in this case) given text inputs (SMILES notations in this case), the dataset is as illustrated below. SMILES notations ...
1
vote
1answer
164 views

What is uncentered variance and how it becomes equal to mean square in Adam?

I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean In this statement, the author is talking about ...

1
2
3 4 5
33