A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [dropout]

For questions related to the concept of dropout, which refers to the dropping out units in a neural network (NN), during the training of the NN, so that to avoid overfitting. The dropout method is a regularisation technique, which was introduced in "Dropout: A Simple Way to Prevent Neural Networks from Overfitting" (2014) by Nitish Srivastava et al.

Filter by
Sorted by
Tagged with
3
votes
1answer
47 views

Can dropout layers not influence LSTM training?

I am working on a project that requires time-series prediction (regression) and I use LSTM network with first 1D conv layer in Keras/TF-gpu as follows: ...
6
votes
2answers
114 views

Why is dropout favoured compared to reducing the number of units in hidden layers?

Why is dropout favored compared to reducing the number of units in hidden layers for the convolutional networks? If a large set of units leads to overfitting and dropping out "averages" the response ...
3
votes
1answer
61 views

Can Google's patented ML algorithms be used commercially?

I just find that Google patents some of the widely used machine learning algorithms. For example: System and method for addressing overfitting in a neural network (Dropout?) Processing images using ...
1
vote
1answer
74 views

Dropout causes too much noise for network to train

I am using dropout of different values to train my network. The problem is, dropout is contributing almost nothing to training, either causing so much noise the error never changes, or seemingly ...
3
votes
2answers
729 views

Is pooling a kind of DropOut

If I got well the global idea of DropOut it allows to improve the sparsity of the information that come from one layer to another by setting some weights to zero. In another hand, pooling, let's say ...
2
votes
1answer
39 views

Should I remove the units of a neural network or increase dropout?

When adding dropout to a neural network, we are randomly removing a fraction of the connections (setting those weights to zero for that specific weight update iteration). If the dropout probability is ...
3
votes
2answers
118 views

Price Movement Forecasting Issue

I am working on a project for price movement forecasting and I am stuck with poor quality predictions. At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the ...
1
vote
1answer
76 views

Usefulness of Dropout for non-overfitting network

My neural network is simple enough and does not overfit. Dropout is a regularization technique for reducing overfitting in neural networks From Wikipedia Adding Dropout in a non-overfitting ...
12
votes
1answer
2k views

Why do you not see dropout layers on reinforcement learning examples?

I've been looking at reinforcement learning, and specifically playing around with creating my own environments to use with the OpenAI Gym AI. I am using agents from the stable_baselines project to ...
8
votes
1answer
114 views

5 years later, are maxout networks dead, and why?

Maxout networks were a simple yet brilliant idea of Goodfellow et al. from 2013 to max feature maps to get a universal approximator of convex activations. The design was tailored for use in ...
2
votes
1answer
208 views

Action Probability with Thompson Sampling in Deep Reinforcement Learning

In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
5
votes
2answers
650 views

What to do if CNN cannot overfit a training set on adding dropout?

I have been trying to use CNN for a regression problem. I followed the standard recommendation of disabling dropout and overfitting a small training set prior to trying for generalization. With a 10 ...
2
votes
1answer
696 views

What are the counterparts of non-linearities and dropout in fully convolutional networks?

I am trying to replicate the fully convolutional networks (FCN) concept described here for semantic segmentation. It seems people have successfully trained such models by removing fully connected ...
4
votes
1answer
605 views

Does a bias also have a chance to be dropped out in Dropout layer?

Suppose that you have 80 neurons in a layer, where one neuron is bias. Then you add a dropout layer after the activation function of this layer. In this case, does it have a chance to drop out the ...