Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

598 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
14
votes
1answer
149 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
12
votes
0answers
402 views

How can Viv generate new code based on some user's query?

I have been looking into Viv, an artificial intelligent agent in development. Here is a demonstration of Viv (by Dag Kittlaus). Based on what I understand, this AI can generate new code and execute it ...
11
votes
3answers
374 views

Can some one help me understand this paragraph from Nvidia's progressive gan paper?

Furthermore, we observe that mode collapses traditionally plaguing GANs tend to happen very quickly, over the course of a dozen minibatches. Commonly they start when the discriminator overshoots, ...
7
votes
0answers
678 views

Is there a difference in the architecture of deep reinforcement learning when multiple actions are performed instead of a single action?

I've built a deep deterministic policy gradient reinforcement learning agent to be able to handle any games / tasks that have only one action. However, the agent seems to fail horribly when there are ...
6
votes
2answers
57 views

Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?

I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. ...
6
votes
1answer
145 views

Extending FaceNet’s triplet loss to object recognition

FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper) such that any two faces of the same identity will have a small Euclidean distance, and such ...
6
votes
1answer
48 views

Video summarization similar to Summe's TextRank

We have the popular TextRank API which given a text, ranks keywords and can apply summarization given a predefined text length. I am wondering if there is a similar tool for video summarization. ...
5
votes
0answers
57 views

Training and inference for highly-context-sensitive information

What is the best way to train / do inference when the context matters highly as to what the inferred result should be? For example in the image below all people are standing upright, but because of ...
5
votes
1answer
236 views

What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

I am new to the object recognition community. Here I am asking about the broadly accepted ways to calculate the error rate of a deep CNN when the network produces different results using the same data....
5
votes
1answer
135 views

Is one big network faster than several small ones?

The basis of my question is that a CNN that does great on MNIST is far smaller than a CNN that does great on ImageNet. Clearly, as the number of potential target classes increases, along with image ...
4
votes
1answer
46 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
4
votes
0answers
64 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
4
votes
0answers
59 views

Why don't the neural networks inside LSTM cells contain hidden layers?

I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers. Why don't LSTM cells have more complex neural ...
4
votes
0answers
26 views

What Are the key Plugins And Features to implement in AI Assistant Personal Project?

Hope you're doing great I'm doing a personal project for android, I want to make an app in Android Studio. The App will be an AI Assistant Like Siri, Jarvis , etc. The idea in mind is to make the app ...
4
votes
0answers
569 views

What are the differences between Yolo v1 and CenterNet?

I recently read a new paper (late 2019) about a one-shot object detector called CenterNet. Apart from this, I'm using Yolo (V3) one-shot detector, and what surprised me is the close similarity between ...
4
votes
0answers
54 views

When to use RMSE as opposed to MSE and vice versa?

I understand that RMSE is just the square root of MSE. Generally, as far as I have seen, people seem to use MSE as a loss function and RMSE for evaluation purposes, since it exactly gives you the ...
4
votes
2answers
48 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
4
votes
0answers
330 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
4
votes
1answer
59 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
4
votes
0answers
115 views

How did the OpenAI 5 for Dota concatenate units?

I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully ...
4
votes
0answers
98 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
4
votes
0answers
49 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
4
votes
0answers
80 views

Is there any theoretical capabilities of apply deep successor representations with A3C algorithm?

Deep Successor Representations(DSR) has given better performance in tasks like navigation when it compares to normal model-free RL tasks. Basically, DSR is a hybrid of model-free RL and Model-Based RL....
4
votes
2answers
2k views

What is the purpose of “reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval.”?

I am a deep learning beginner recently reading this book "Deep learning with Python", the example explains the process of implementing a greyscale image classification using MNIST in keras, in the ...
4
votes
0answers
452 views

Mountain car problem with images - not converging

I'm trying to find the optimal policy for the mountain car problem using deep Q learning with images as input, however, I cannot find a way to get my Q function to give me good solutions (I followed ...
4
votes
1answer
99 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable length input padded with 0. However the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...
4
votes
0answers
297 views

What are some interesting recent papers that synthesize symbolic AI with Deep Learning?

A lot of people seem to be under the impression that combining GOFAI and contemporary AI will make models more general. I'm particularly interested in reasoning through analogy or case-based reasoning....
4
votes
0answers
150 views

Game AI - Modify image classification model for analog output

I'm developing a Game AI which tries to master racing simulation. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. I had two main issues ...
3
votes
0answers
28 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
3
votes
0answers
31 views

DQN Agent with a 2D matrix as input in Keras

I have a Reinforcement Learning environment where the state is a 2D matrix with 0s and 1s (only one column with the value of 1 in each row). Example: ...
3
votes
0answers
35 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
3
votes
1answer
58 views

Is batch learning with gradient descent equivalent to “rehearsal” in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
3
votes
0answers
41 views

Would a different learning rate for every neuron and layer mitigate or solve the vanishing gradient problem?

I'm interested in using the sigmoid (or tanh) activation function instead of RELU. I'm aware of RELU advantages on faster computation and no vanishing gradient problem. But about vanishing gradient, ...
3
votes
1answer
130 views

What are the keys and values of the attention model for the encoder and decoder in the “Attention Is All You Need” paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
3
votes
0answers
41 views

Where does reinforcement learning actually show up in Deepmind's game engines?

From the brief research I've done on the topic, it appears that the way Deepmind's Alphazero or Muzero makes decisions is through Monte Carlo tree searches, where in the randomized simulations allows ...
3
votes
0answers
63 views

Simplification of expected reward under the limit in continuous tasks

I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
3
votes
0answers
145 views

Understanding the results of “Visualizing and Understanding Convolutional Networks”

I am trying to understand the results of the paper Visualizing and Understanding Convolutional Networks, in particular the following image: What are these 3x3 blocks and their 9 cells representing? ...
3
votes
0answers
102 views

What is symbol-to-number differentiation?

I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient ...
3
votes
0answers
47 views

Are there any commonly used discontinuous activation functions?

Are there any commonly used activation functions (e.g. that take values in $(0,.5)\cup (.5,1)$)? Preferably for classification? Why? I was looking for commonly used activation functions on Google, ...
3
votes
0answers
32 views

What is the difference between “out-of-distribution (generalisation)” and “(meta)-transfer learning”?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
3
votes
0answers
119 views

How to correctly implement self-play with DQN?

I have an environment where an agent faces an equal opponent, and while I've achieved OK performance implementing DQN and treating the opponent as a part of the environment, I think performance would ...
3
votes
1answer
148 views

Should I prefer the model with the lowest validation loss or the highest validation accuracy to deploy?

I trained a ResNet20 on Cifar10 and obtained the following learning curves. From the figures, I see at epoch 52, my validation loss is 0.323 (the lowest), and my validation accuracy is 89.7%. On the ...
3
votes
0answers
186 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
3
votes
0answers
41 views

How can I compare EEG data with accelerometer data in 1 algorithm?

I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
3
votes
0answers
364 views

Why isn't there a model playing FPS like CoD or Battlefield already existing?

Assuming we had an unlimited time to train a model and a very powerful machine to use our model in real-time (hello quantum computer), I'd like to know why no one could achieve to build an AI able to ...
3
votes
0answers
46 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
3
votes
0answers
32 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
3
votes
0answers
46 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
3
votes
0answers
39 views

Can sequence-to-sequence models be used to convert source code from one programming language to another?

Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
3
votes
0answers
38 views

How should I make output layer of my neural network so that I can get outputs ranging from [-20,-1]

I am trying to make a neural network which takes in 0 and 1 as it's input and should give me output ranging from [-20,-1].I am using three layers with sigmoid as the activation function .How should I ...

1
2 3 4 5
12