Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Why can't we use only keys to calculate self-attention?

I was reading about the self-attention mechanism and the paper suggests to have 3 things to be computed: Key, Query and Value. As far as I understood the reason for having Value is to allow ...
0 votes
0 answers
7 views

Video Processing [closed]

I should write a chapter about video (components of video and its characteristiques and the pre-processing and processing etc.) But I can't find resources that can help me to write this chapter, can ...
0 votes
1 answer
180 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
-1 votes
1 answer
337 views

Difference between FLAX and pytorch

I am going through a code written in FLAX instead of pytorch .Can someone please explain what is the difference between these two deep learning frameworks?
0 votes
1 answer
65 views

What does crop size refer to in DeepLabv3 paper?

In the paper in which DeepLabv3 is presented, the authors are mentioning that: "For atrous convolution with large rates to be effective, large crop size is required; otherwise, the filter ...
0 votes
2 answers
398 views

What does a value of -1.000 mean in MS COCO Metrics for Object Detection

I am training some Object-Detection-Models from the TensorFlow Object Detection API and got from the evaluation with MS COCO metrics the following results for Average Precision: IoU = 0.5;0.9 maxDets =...
0 votes
0 answers
9 views

What’s more efficient in multihead attention: multiply QKV by W_i then split or linearly project QKV h times into dimensions d_k?

I’m looking to bridge two implementations of multihead attention. Approach 1: Multiply and Split Each of the queries, keys, and values is multiplied by a separate square weight matrix of size (...
2 votes
1 answer
34 views

How is a LLM able to override its prior knowledge through In-Context Learning?

I came across a Google's blog (https://research.google/blog/larger-language-models-do-in-context-learning-differently/) discussing large language models (LLMs) and how we can overried their prior ...
0 votes
1 answer
86 views

How to use a NN for seq2seq tasks?

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
0 votes
2 answers
93 views

How does high entropy targets relate to less variance of the gradient between training cases?

I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this: When the soft targets have high entropy, they provide much ...
1 vote
2 answers
1k views

Can I extend Graph Convolutional Networks to graphs with weighted edges?

I'm researching spatio-temporal forecasting utilising GCN as a side project, and I am wondering if I can extend it by using a graph with weighted edges instead of a simple adjacency matrix with 1's ...
0 votes
1 answer
17 views

What is $z|y$ in Conditional Adversarial Nets?

I am currently going through Conditional Adversarial Nets (CGANs) and the modified objective function of the two-player minimax game is stated as follows: $$\min_G \max_D V(D, G)=\mathbb E_{x\sim p_{...
0 votes
0 answers
9 views

Incredibly High CrossEntropyLoss in Sequence-to-Sequence Generation

I'm trying to do SMILES chemical representation prediction from a large dataset (Around 5M Samples) to teach it do predict another downstream task. The model's part responsible for generating the data ...
0 votes
1 answer
33 views

How to get Complexity per Layer, Sequential Operations and Maximum Path Length in CNN architecture?

In the paper Attention is all you need, here is Table 1, can someone explain what architecture is referred to in the "Convolution" row and hence describe the other 3 columns in it? The other ...
3 votes
1 answer
97 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
2 votes
6 answers
748 views

How do I increase the size of an (almost) balanced dataset?

I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for ...
1 vote
2 answers
48 views

Dealing with incomplete file sets for a CNN for medical imaging regression problem

I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types: flair.nii.gz mprage.nii.gz swi....
0 votes
1 answer
66 views

Best way to generate a human face over a face generated by FaceFormer framework?

FaceFormer framework generates a talking face from audio, focusing on the lip and face movement when a person talks. Now from that what would be the best way to generate a human face on top of that? I ...
1 vote
1 answer
362 views

A way to leverage machine learning to reduce DFS/BFS search time on a tree graph?

I'm not very knowledgeable in this field but I'm wondering if any research or information already exists for the following situation: I have some data that may or may not look similar to each other. ...
0 votes
0 answers
24 views

Segment a spectrogram into a series of images by (constant) beats per minute to train a Deep Neural Network

I have a .csv file with information about a soundtrack and it is divided into beats (per minute), which are ordered by row. As in: the index corresponds to each beat, and the columns have info about ...
3 votes
2 answers
254 views

How to fix time dimension in time varying data-sets using deep learning model for classification?

Dataset Description I am working on the famous ABIDE Autism Datasets. The dataset is very big in the sense that it has more than 1000 subjects containing half of them as autistic and the other half as ...
3 votes
1 answer
197 views

Are there tabular datasets where deep neural networks outperform traditional methods?

Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin? I'd prefer tabular datasets rather than ...
6 votes
1 answer
109 views

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems? If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say ...
1 vote
1 answer
39 views

Reach optimal values by not decreasing gradient

Is it possible to reach the optimal values ​​for the parameters by not applying the decreasing gradient in some layers?
0 votes
1 answer
140 views

In a face database containing multiple images per subject, how do we determine the face image which is most suited for face recognition?

Let us imagine a face database with several subjects, each subject having multiple face images. How do we determine which is the best face suitable for face recognition purposes?
0 votes
2 answers
96 views

Performance of augmented dataset with or without original images

I am training on yolo and I had a small dataset. I decided to increase it by augmenting it with rotation, shearing, etc to increase the size and increase accuracy. Now I have seen augmented datasets ...
-1 votes
1 answer
56 views

Can anyone please explain the Recurrent Neural Network calculation shown in the picture?

As you can see, this is a recurrent neural network. I want to understand how the calculations are being made. Please, be as detailed as possible no matter how simple or self-explanatory the ...
1 vote
1 answer
486 views

Handling imbalanced data with multiple targets

I have the model which has 3 outputs (it is a regression task, I have the angle of the steering wheel, brake and acceleration). I can divide my values to some smaller bins and in this way I can change ...
0 votes
1 answer
287 views

Hot to calculate Maximum Normalized log Probability for Active Learning with BERT

I have encountered difficulties understanding the calculation of Maximum Normalized Log Probabilities acording to Shen et al.. With n being the sequence length, yi the label of word i. Xij is the ...
0 votes
1 answer
18 views

Determining optimal data size for generalization in transformer encoders, particularly for Time-Series signal data

I'm currently experimenting with training a model that employs a single transformer encoder on time-series signal data. Despite having a relatively small dataset of around 50 examples, each with a ...
0 votes
0 answers
24 views

Neural Network with Incorrect Calculation Better than Correct One

I have designed my own neural network and discovered an error. During backpropagation, instead of inserting the Z-values into the derivative of the activation function, I inserted the A-values. The ...
0 votes
0 answers
22 views

No matter how I change a loss function I get it equal to infinity

I am a bioinformatician, and at the moment I am working with a dataset containing ~12.3 million mutations for ~5500 individuals. The goal is to perform binary classification. I use this framework to ...
8 votes
1 answer
2k views

When to use Tanh?

When and why would you not use Tanh? I just replaced ReLU with Tanh and my model trains about 2x faster, reaching 90% acc within 500 steps. While using ReLU it reached 90% acc in >1000 training ...
0 votes
0 answers
17 views

Preparation of multivariate time series data

I am doing a university project on index/stock price prediction. I plan to use a combined cnn-lstm model, and I have several different types of data: Open High Low Close Volume, values, fundamental ...
2 votes
2 answers
583 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
0 votes
0 answers
5 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
1 vote
1 answer
1k views

How to use the LSTM layer in PPO architecture?

What is the best way of using the LSTM layer in PPO architecture? Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? Should I ...
0 votes
1 answer
71 views

Using naive bayesian vs. transformer-based architecture model for human-annotated data?

I have a reddit dataset with thousands of online posts over the economy and inflation. We have used human-annotation on 60% of posts to determine whether users blame the following entities over the ...
0 votes
1 answer
88 views

Advanced / Complex Neural Network (System) Design

In addition to basic forward networks, many books cover other basic network designs like CNN's and RNN's. However, they don't really go any further than that, explaining things like common approaches ...
1 vote
0 answers
20 views

Why completely two different algorithms are being used in Deep Q Learning?

I'm a new student in reinforcement learning. Recently, I've been studying about different algorithms of RL. But I'm quite surprized that there are some algorithms which are named as "same" ...
0 votes
0 answers
9 views

Is it possible to have lower ECE but worse reliable curve?

I am new to the calibration concept for classification. I have tried temperature scaling on my model's results. However, after applying temperature scaling, the reliable curve got worse despite ...
0 votes
1 answer
364 views

Object Classification: How to decide which detected region is a RoI for classification?

I am working on a project where I am working on the Flickr-47 dataset to do logo detection and classification. My approach is to first finetune a YOLO v5 model with high recall to detect as many "...
0 votes
1 answer
68 views

What algorithm to use to classify data by spatial relations?

Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, ...
4 votes
2 answers
301 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
1 vote
1 answer
452 views

How to normalize for perceptual loss when training neural net from scratch?

Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) ...
1 vote
1 answer
665 views

Is Softmax Necessary as the Activation Function for Self-Attention Mechanisms?

I’m curious about the mathematical reasoning behind the use of the softmax function as the activation function in self-attention mechanisms within neural networks. Specifically, I’m interested in ...
0 votes
0 answers
50 views

UNets with a pretrained network as the encoder portion of U-Net

UNets with a pretrained network (like VGG16 or InceptionV3 or ResNet, or …) as the encoder portion of U-Net are common. However I'm struggling to understand how the 1D encoded second-to-last layer is ...
5 votes
3 answers
3k views

Last linear layer of the decoder of a transformer

I am learning the transformers architecture from these two sources: https://arxiv.org/pdf/1706.03762.pdf https://jalammar.github.io/illustrated-transformer/ I just wanted to ask about the final step ...
0 votes
1 answer
136 views

Machine Learning Methods commonly used when data are scarse

It is well-known that deep neural networks require lots of data to perform reliably and well. A commonly-cited statistic is that you need at least 10,000 examples per class for a classification ...
-1 votes
1 answer
840 views

How to retrain a Facenet model with the triplet loss function?

I want to calculate the similarity or distance of two faces. I'm using Python. I have read and done what this tutorial says. However, the result is not good (the similarity of same faces and ...

1
2 3 4 5
41