Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

258 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9
votes
0answers
60 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data so we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
9
votes
1answer
196 views

Deep Networks and generalisation of Hopfield Networks

Hopfield Nets are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimise the energy function when all neurons are set equal the ...
9
votes
5answers
310 views

Can an AI be trained to generate the outline of a story?

I know that one of the recent fads right now is to train a neural network to generate screenplays and new episodes of e.g. the Friends or The Simpsons, and that's fine: it's interesting and might be ...
7
votes
1answer
441 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
7
votes
3answers
246 views

Can some one help me understand this paragraph from Nvidia's progressive gan paper?

Furthermore, we observe that mode collapses traditionally plaguing GANs tend to happen very quickly, over the course of a dozen minibatches. Commonly they start when the discriminator overshoots, ...
6
votes
1answer
107 views

Synapses automatically select it's neurons

I know the basics of Artificial Neural Networks. For instance; make dot product with the weights and every neuron from previous layer. Adjust the weight by error. And done, That is how I see neural ...
6
votes
3answers
90 views

How much can the addition of new features improve the performance?

How much can the addition of new features improve the performance of the model during the optimization process? Let's say I have a total of 10 features. Suppose I start the optimisation process using ...
5
votes
1answer
37 views

Why do very deep non resnet architectures perform worse compared to shallower ones for the same iteration? Shouldn't they just train slower?

My understanding of the vanishing gradient problem in deep networks is that as backprop progresses through the layers the gradients become small, and thus training progresses slower. I'm having a hard ...
4
votes
0answers
22 views

Extending FaceNet’s triplet loss to object recognition

FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper) such that any two faces of the same identity will have a small Euclidean distance, and such ...
4
votes
0answers
19 views

Video summarization similar to Summe's TextRank

We have the popular TextRank API which given a text, ranks keywords and can apply summarization given a predefined text length. I am wondering if there is a similar tool for video summarization. ...
4
votes
0answers
83 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
4
votes
0answers
78 views

Catastrophic Forgetting on Pong Environment using DQN

I am running a basic DQN on the Pong environment. Not a CNN, just a 3 layer linear neural net with ReLUs. It seems to work for the most part, but at some point my model suffers from catastrophic ...
4
votes
0answers
46 views

How did the OpenAI 5 for Dota concatenate units?

I am no expert in the field of AI so I apologize if this is a simple/easy question. I was trying to implement a network similar to OpenAI's for another game and I noticed that I did not fully ...
4
votes
0answers
44 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
4
votes
2answers
319 views

Is it possible to use AI to reverse engineer software?

I was thinking of something of the sort: Build a program (call this one fake user) that generates lots and lots and lots of data based on the usage of another program (call this one target) using ...
4
votes
1answer
95 views

What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

I am new to the object recognition community. Here I am asking about the broadly accepted ways to calculate the error rate of a deep CNN when the network produces different results using the same data....
4
votes
0answers
77 views

Is there any theoretical capabilities of apply deep successor representations with A3C algorithm?

Deep Successor Representations(DSR) has given better performance in tasks like navigation when it compares to normal model-free RL tasks. Basically, DSR is a hybrid of model-free RL and Model-Based RL....
4
votes
0answers
138 views

Game AI - Modify image classification model for analog output

I'm developing a Game AI which tries to master racing simulation. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. I had two main issues ...
3
votes
0answers
22 views

Vector normalization by a neural network

I'm wondering if there is a NN that can achieve the following task: Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\...
3
votes
1answer
57 views

Is normalizing the data a way to improve generalization?

There are many known ways to overcome overfitting or make a model generalize better to unseen data. Here I would like to ask if normalizing/standardizing/similiraizing the train and test data is a ...
3
votes
2answers
30 views

What kind of output should be used for predicting angles in DNNs?

I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles? For example, output the angle in radians cyclic nature of the ...
3
votes
0answers
17 views

What parameters can be tweaked to avoid a generator or discriminator loss collapsing to zero when training a DC-GAN?

Sometimes when I am training a DC-GAN on an image dataset, similar to the DC-GAN PyTorch example (https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html), either the Generator or ...
3
votes
1answer
77 views

How can I build an AI with NLP that read stories

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet. The idea is as follows: Let's say we have a story (in the text) that has 10 ...
3
votes
0answers
41 views

Should noise (such as OU) be decreased over time in actor / critic algorithms?

In most of RL algorithms I saw, there is a coefficient that reduces actions exploration over time, to help convergence. But in Actor-Critic, or other algorithms (A3C, DDPG, ...) used in continuous ...
3
votes
0answers
39 views

DQN Agent not learning anymore - what can I do to fix this?

I am trying to use Deep-Q-Learning to learn an ANN which controls a 7-DOF robotic arm. The robotic arm must avoid an obstacle and reach a target. I have implemented a number of state-of-art ...
3
votes
0answers
70 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
3
votes
3answers
395 views

How to implement a Continuous Control of a quadruped robot with Deep Reinforcement Learning in Pybullet and OpenAI Gym?

Description I have designed this robot in URDF format and its environment in pybullet. Each leg has a minimum and maximum value of movement. What reinforcement algorithm will be best to create a ...
3
votes
1answer
66 views

Should the input to the negative log likelihood loss function be probabilities?

I am trying to train a supervised model where the output from the model is output of a linear function $WX + b$. Kindly note that I'm not using any softmax or $\log$ softmax on the result of the ...
3
votes
1answer
245 views

Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)

Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision ...
3
votes
2answers
991 views

What is the purpose of “reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval.”?

I am a deep learning beginner recently reading this book "Deep learning with Python", the example explains the process of implementing a greyscale image classification using MNIST in keras, in the ...
3
votes
1answer
65 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable length input padded with 0. However the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...
3
votes
0answers
167 views

Understanding multi iteration update of model in Policy Gradient PPO algorithm

I have a general question about the updating of the network/model in the PPO algorithm. If I understand it correctly, there are multiple iterations of weight updates done on the model with data that ...
3
votes
0answers
137 views

What is the relation between the definition of learnability of Vapnik and Gold and learnability of neural networks?

Gold showed that a language can be learned only if it contains a finite set of sentences. We know that deep neural networks can implement any function. Does this contradict the Gold's result? What ...
3
votes
1answer
2k views

Training AI to play NES/SNES games on NN python

I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey ...
2
votes
0answers
22 views

Are there methods that allow deep networks to learn object categorization in a self-supervised way?

When training a deep network to learn object classification from a set like ImageNet, we minimize the cross entropy between the ground truth and the predicted categories. This is done in a supervised ...
2
votes
0answers
11 views

How to choose the suitable Neural Network Architecture for Regression Tasks

so I'm working on a Project where I want to predict the Vehicle Position from the Vehicle Data like speed, acceleration etc.. now the data that I have comes also with a timestamp for each sample ( I ...
2
votes
0answers
37 views

When to use RMSE as opposed to MSE and vice versa?

I understand that RMSE is just the square root of MSE. Generally, as far as I have seen, people seem to use MSE as a loss function and RMSE for evaluation purposes, since it exactly gives you the ...
2
votes
0answers
31 views

Image to image regression in tensorflow

I am working on an image to image regression task which requires me to develop a deep learning model that takes in a sequence of 5 images and return another image. The sequence of 5 images and the ...
2
votes
0answers
36 views

Feasibility of using machine learning to obtain self-consistent solutions

I am a physicist and I don't have much background on machine learning or deep learning except taking a couple of courses on statistics. In physics, we often simulate a model by means of two-way ...
2
votes
0answers
18 views

How can I detect fast and slow motion in videos?

I'm trying to detect if a given video shot is fast or slow motion. Basically, I need to calculate a "video motion" score in a given video sequence, meaning how fast or slow motion the video is. For ...
2
votes
0answers
39 views

Spike detection in time series using Artificial Neural Networks

I'm quite new in ANNs. I intend to use ANNs for predicting spike points in time series right before they happen. I've already used LSTM for another scenario, and I know that they can be used in ...
2
votes
0answers
25 views

How do deep fakes get the right encoding for both people?

Deep fakes work by using a single encoder but then having a different decoder for different people. But I wondered what if the encoder encodes say "closed eyes" of person A as the same code for "...
2
votes
0answers
39 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
2
votes
0answers
60 views

How can I incrementally train a Yolo model without catastrophic forgetting?

I have successfully trained a Yolo model to recognize k classes. Now I want to train by adding k+1 class to the pre-trained weights (k classes) without forgetting previous k classes. Ideally, I want ...
2
votes
1answer
29 views

Spikes in of Train and Test error

I learn a DNN for image recognition. During each epoch, I calculate mean loss in the training set. After each epoch, I calculate loss and number of errors over both training and test set. The problem ...
2
votes
0answers
21 views

How to voxelize multiple frames at the time and append them together?

I'm trying to implement this approach for object detection and tracking. In this approach, the first step is voxelize each frame to construct a 3D tensor, the second step is to append multiple voxels ...
2
votes
0answers
14 views

What are some neural network models that can use auxiliary info during training for image segmentation?

What are some deep learning models that can use supplementary information other than RGB channels for image segmentation? For example imagine a poorly shot image of a river (blue) that shows a gap, ...
2
votes
0answers
33 views

What are the benefits of using the state information that maintains the graph structure?

When you applying a graph structured data to the graph convolution network, what are the benefits of using the state information that maintains the graph structure?
2
votes
1answer
33 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
2
votes
0answers
17 views

Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases

I was working on CNN. I modified the training procedure on runtime. As we can see the validation loss and validation accuracy, The yellow curve does not fluctuate much. Green curve and Red curve ...