Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Feature extraction with or without data augmentation, what's the difference?

Here's an extract from Chollet's deep learning book about using pre-trained CNN to predict class from a photo set : At this point, there are two ways you could proceed: Running the convolutional ...
0
votes
0answers
19 views

What happens in an LSTM's Encoder?

I am currently learning about LSTM models and have a question: I am not sure what is happening inside the Encoder; I know, that the Encoder gives a hidden vector for the Decoder but I still have the ...
0
votes
0answers
11 views

Evaluating an embedding network for accuracy

I aim to do action recognition in videos on a private dataset. To test with the existing state-of-the-art implementation that other guys published their code on Github, like the one here: IIC. Here ...
0
votes
0answers
6 views

How can I build and train mode for Arabic word embedding from scratch using BERT and share the model on hugging face?

my project is (building an Arabic word embedding model). I want to build my own model on hugging face like (aubmindlab/AraBERT model) for Arabic language using Bert for word embedding. How can I start ...
1
vote
0answers
46 views

Can the law of iterated expectation be used on the inner expectation of the DQN cost function described in the DQN paper

Is the expression for the DQN cost function, Equation (2) of the DQN paper $$\begin{align}L_1 &= E_{\mu,\pi}\left[\left(y_i - q(s,a;\theta)\right)^2\right]\\ &=E_{\mu,\pi}\left[\left(E_{\...
0
votes
1answer
16 views

Handling imbalanced data with multiple targets

I have the model which has 3 outputs (it is a regression task, I have the angle of the steering wheel, brake and acceleration). I can divide my values to some smaller bins and in this way I can change ...
1
vote
0answers
26 views

What are the benefits of Cross Stage Partial Connections over Residual Connections?

Cross Stage Partial Connections (CSPC) try to solve the next problems: Reduce the computations of the model in order to make it more suitable for edge devices. Reduce memory usage. Better ...
0
votes
1answer
30 views

Can you add more layers in Transformers for classification [closed]

I am working on NLP classification task and I am using the AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=5) method. ...
0
votes
0answers
25 views

Is it possible to use GPT-3 with voice cloning to have GPT-3 respond in a custom voice?

I'm curious about three things here. Is it possible to use GPT-3 along with voice cloning to have GPT-3 either respond in a custom voice or possibly create an app that can do the same sort of thing? ...
1
vote
0answers
12 views

Why identity mapping is so hard for deeper neural network as suggested by Resnet paper?

In resnet paper they said that a deeper network should not produce more error than its shallow counterpart since it can learn the identity map for the extra added layer. But empirical result shown ...
1
vote
0answers
48 views

Setting up a deep learning architecture for multi-dimensional data

The input data is thousands, millions of 4x1000 matrices. Each row consists of 3 small natural numbers (1000 combinations) and a corresponding real number between 0 and 1. The output is a 1x1000 ...
0
votes
1answer
8 views

How to use mixed data for image segmentation?

I have a task for which I have to do image segmentation (cancer detection on MRIs). If possible, I would also like to include clinical data (i.e. numeric/categorical data which comes in the form of a ...
1
vote
0answers
30 views

How to restrain a model's outputs to a certain range without affecting its representative capacity?

CONTEXT I am trying to build a regression model that finds the optimal parameters for a given input. The data I am using are point clouds, with N points and ...
-1
votes
0answers
27 views

I'm designing a Neural Network in Matlab using the Sigmoid Function to complete a specific goal that should be trivial- and I am stuck

For a graduate school project, I'm crafting the Neural Netowrk that I breifly talked about in the title. I've been working on it for a while. This might sound contradictory, but I'm very happy with ...
1
vote
0answers
17 views

Using numerical/categorical data and image data to detect objects

Let's say that I want to create a program capable of detecting lamps on some pictures. Those pictures can be, for instance, of a room, a street, etc. I would like to know if the following is possible: ...
6
votes
2answers
289 views

Is there an ideal range of learning rate which always gives a good result almost in all problems?

I once read somewhere that there is a range of learning rate within which learning is optimal in almost all the cases, but I can't find any literature about it. All I could get is the following graph ...
2
votes
0answers
21 views

What are the metrics to be used for unsupervised monocular depth estimation in computer vision?

I am currently replicating the results of this paper. In this paper they have not mentioned how they are evaluating the results as no ground truth is available for comparison. Same goes for other ...
1
vote
1answer
20 views

Is there a methodology for splitting up annotated orthophotos into smaller photos that retain the original bounding boxes?

I'm trying to train an object detection algorithm (i.e. YOLOv4 Scaled, Faster R-CNN) on data taken from large orthophotos. Let's say I have one class, and I label the entire orthophoto with bounding ...
0
votes
1answer
22 views

Are the Word2Vec encoding available online [closed]

I am trying to do an NLP project and was wondering if there is anywhere online where the Word2Vec encoding are stored. I want to search up a word and see what its encoding is. I have tried looking but ...
4
votes
1answer
141 views

Will there be some promising techniques that can make AI greener and affordable in the future?

The recent advances in machine learning were mostly achieved by the hardware, and the hardware is said to continue driving the development of AI, but I was still shocked by this thread which reads ...
0
votes
1answer
23 views

NLP: Are hashtags tokenised?

I am exploring a potential NLP project and was wondering what generally is done with the hashtags words (e.g. #hello). Are those words ignored? is the # removed and the word tokenised? is it tokenised ...
0
votes
0answers
12 views

Why do the inception score and the Fréchet inception distance use the inception network and not another network?

So I was researching about the evaluation of GANs and found these two metrics which seem to be the most popular. I understand that the main ideia is to apply the data to a pre-trained network in order ...
0
votes
0answers
15 views

How to augment 2.5D keypoints?

I am currently working on 3D hand pose estimation. The idea is to first detect the 2.5D pose representation and then obtain 3D pose with the help of camera parameters. For some reason, I was trying to ...
0
votes
0answers
26 views

Understanding policies in helicopter control in the paper by Andrew Ng et al

I was going through this paper on helicopter flight control using reinforcement learning by Andrew Ng et al. It defines two policy classes to learn two policies, one for hovering the helicopter and ...
0
votes
1answer
37 views

What kind of deep learning model does latest version of AlphaFold use for protein folding problem?

I understand there are multiple versions used in AlphaFold. What kind of deep learning model does the more advanced version use? CNN, RNN, or something else? (Additionally, is there an open-source ...
1
vote
0answers
16 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
0
votes
0answers
22 views

How to change the number of input neurons in embedding layer?

I was building a recommender system using Tensorflow recommenders (TFRS) library . I was following the official tutorial for ranking model , where they have used two-tower model. The part where I have ...
1
vote
0answers
24 views

Does the order of data augmentation and normalization matter?

What is the preferred order of data augmentation and normalization? Is it the former followed by the latter?
-1
votes
0answers
20 views

How to detect and eliminate output bias in a neural network?

I am working on applying Neural networks for a regression problem on images and the outputs of the model that I trained are always of a lower value than the actual values. Does this certify as a bias ...
3
votes
2answers
63 views

Is it possible to do face recognition with just the eyes?

Assuming the input photo is focused on a person's face, if the person is wearing a surgical mask, most face recognition software fail to identify the subject's face. Most facial landmark models are ...
0
votes
0answers
18 views

How is the marginal likelihood of wide valleys higher than that of narrow valleys when optimizing a cost function?

I am reading the paper Entropy-sgd: Biasing gradient descent into wide valleys by Chaudhari et al. From what I understand, wide valleys tend to generalize better than sharp ones because they are ...
2
votes
1answer
268 views

Can ML/DL solve my classification problem?

I'm new to AI but would still like to try and get a project off the ground. I've read a lot about ML/DL the past few days but I just can't figure out if my problem can be solved with ML/DL. What I'm ...
0
votes
0answers
16 views

How to chose a specific contextual embedding?

I am learning deep learning using a plethora of online resources. I have been using mostly word2vec and gloVe for my NMT model. I recently came across the concept of contextual-embedding, and I see ...
1
vote
1answer
22 views

Why are the weights of the previous layers updated only considering the old values of the weights of the later layer, not the updated values?

Why are the weights of a neural net updated only considering the old values of the later layer, not the already updated values? I use this example to explain my problem. When applying the ...
0
votes
0answers
36 views

What are the popular approaches to Q-value approximation?

I need the q-value for my RL training, there are some approaches: Brute-force the action sequence (this won't work for long sequence) Use classic algorithm to optimise and estimate (this ain't much ...
3
votes
0answers
17 views

How are Ground truth provided to each Pyramid map in RetinaNet or YOLOv3 Paper? How is the mapping of Feature Pyramids done to Ground Truth

SO the YOLO V3 and RetinaNet both uses the Feature pyramids which look something like this: (except b and e which have one ...
1
vote
0answers
18 views

Why does the relativistic discriminator increase the probability that generated data are real and decrease the probability that real data are real?

I was reading the ESRGAN whitepaper, where I came across this line: Relativistic discriminator [2] is developed not only to increase the probability that generated data are real but also to ...
0
votes
0answers
32 views

Accuracy goes straight for about 200 epochs then start increasing

Can anyone explain the following observation? Why did the accuracies keep to be a straight line with a very smooth decrease of loss? Is this because of the learning rate or other reasons? Some info: ...
1
vote
0answers
10 views

How to predict multiple set of coordinates (of bounding boxes) for signboards text localization through neural network?

I am creating a signboard translation application from scratch. I have images of signboards where there are multiple texts and I have the corresponding set of coordinates of bounding boxes for ...
6
votes
0answers
49 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
0
votes
0answers
17 views

How does the embeddings work in vision transformer from paper?

I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to ...
0
votes
1answer
44 views

Factors that causing totally different outcomes from an exactly same model and datasets

Here is a model that trains time series data in (batch, step, features) way. I have kept the random state for train test split function the same. Every parameter below the same, running the model ...
1
vote
0answers
18 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
2
votes
1answer
71 views

Aren't scores in the Wasserstein GAN probabilities?

I am quite new to GAN and I am reading about WGAN vs DCGAN. Relating to the Wasserstein GAN (WGAN), I read here Instead of using a discriminator to classify or predict the probability of generated ...
0
votes
0answers
28 views

Is using a LSTM, CNN or any other neural network model on top of a Transformer(using hidden states) overkill?

I have recently come across transformers, I am new to Deep Learning. I have seen a paper using CNN and BiLSTM on top of a transformer, the paper uses a transformer(XLM-R) for sentiment analysis in ...
0
votes
1answer
42 views

Is the working of RNNs, LSTM and GRU sequential or parallel?

You take any blog or any example and all they tell you about is the given picture below. It has 4 different matrices and 3 of whose weights are shared. So, I'm wondering how is this achieved in ...
0
votes
0answers
14 views

How to deal with dynamically changing input tensor in neural networks without padding?

I have a dataset about the monitored health/growth of a community of people. The dataset has tensor shaped (batch_size, features, person, window), where: person==...
0
votes
0answers
40 views

Alpha Zero does not converge for Connect 6, a game with huge branching factor - why?

I have a problem with applying alpha zero self-play to a game (Connect 6) with a huge branching factor (30,000 on average). I have implemented the MCTS as described but I found that during the MCTS ...
2
votes
1answer
64 views

What do the variables in the cross-correlation formula mean?

I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in Goodfellow's Deep Learning (page 329), I can't quite understand what $m$ ...
1
vote
0answers
16 views

1
2 3 4 5
29