Hot answers tagged

35 votes
Accepted

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of ...
Dr. Snoopy's user avatar
  • 1,320
15 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to ...
Apollys supports Monica's user avatar
11 votes
Accepted

Effect of batch size and number of GPUs on model accuracy

This should make a difference, but how big is the difference heavily depends on your task. However, generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but ...
Clement's user avatar
  • 1,735
9 votes

Can LSTM neural networks be sped up by a GPU?

From Nvidia www (https://developer.nvidia.com/discover/lstm): Accelerating Long Short-Term Memory using GPUs The parallel processing capabilities of GPUs can accelerate the LSTM training and ...
pasaba por aqui's user avatar
7 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard ...
Peteris's user avatar
  • 883
6 votes
Accepted

Can LSTM neural networks be sped up by a GPU?

I found that there are cuDNN accelerated cells in Keras, for example, https://keras.io/layers/recurrent/#cudnnlstm. They are very fast. The normal LSTM cells are faster on CPU than on GPU.
Dieshe's user avatar
  • 289
6 votes
Accepted

Deep Q-Learning poor convergence on Stochastic Environment

The inputs that you describe seem like they should be sufficient for a DQN-based agent to learn a good strategy for playing Minesweeper, regardless of whether or not the starting layout changes. The ...
Dennis Soemers's user avatar
  • 10.2k
4 votes
Accepted

Accuracy too high too fast?

It actually depends on a couple of things here - How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly ...
Dishant Sheth's user avatar
4 votes

Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

I'm going to post another guess to this question - it won't be a complete answer, but hopefully it'll provide some direction towards finding a more legitimate answer. The feed-forward networks as ...
mortonjt's user avatar
4 votes
Accepted

Why are traditional ML models still used over deep neural networks?

Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models? Of course, the model that achieves state-of-the-art ...
nbro's user avatar
  • 40.2k
3 votes
Accepted

How can I prevent the CNN from classifying a new input into one of the existing labels (it was trained with) when the input has a new different label?

You can introduce another class to your network - "not a book". After that, you will need to add new data to your dataset, random images that do not contain books to classify and train your network on ...
Andrew's user avatar
  • 276
3 votes

How to constraint the output value of a neural network?

There are many ways of constraining the network's output. Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by ...
BlueMoon93's user avatar
3 votes

Will a neural network always predict the correct label if it sees the exact same input during training and testing?

No, Neural Networks do not have such a guarantee. In fact, I don't believe any kind of classifier in the entire field of Machine Learning has such a guarantee, though some may be slipping my mind... ...
Dennis Soemers's user avatar
  • 10.2k
3 votes

Adding BERT embeddings in LSTM embedding layer

Instead of using the Embedding() layer directly, you can create a new bertEmbedding() layer and use it instead. ...
skillsmuggler's user avatar
3 votes

Entropy term in Proximal Policy Optimization (PPO) becomes undefined after few training epochs

I browsed through some other implementations of PPO and they all add small offset (1e-10) to prevent undefined log(0). I did that and the training works now.
Chintan Trivedi's user avatar
3 votes

Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

1) The math is the exact same, so from an optimization or mathematical perspective there is no difference 2) Here are my guesses to a possible answer. Habit: People may just call one over the ...
mshlis's user avatar
  • 2,359
3 votes
Accepted

Is CNN capable of extracting the descriptive statistics features

CNNs learn convolutional filters that get trained on finding local, recurring patterns in some kind of image/volume data. 1D convolution is actually a thing, but I think what would be more suitable ...
Daniel B.'s user avatar
  • 815
3 votes
Accepted

How to describe an keras Model in a scientific report

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. ...
Balraj Ashwath's user avatar
3 votes

Effect of batch size and number of GPUs on model accuracy

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely ...
Ketil Malde's user avatar
3 votes

Why are traditional ML models still used over deep neural networks?

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply ...
Michael Grogan's user avatar
3 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder ...
Rich Chase's user avatar
3 votes
Accepted

Is my GRU model under-fitting given this plot of the training and validation loss?

When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your ...
techPirate99's user avatar
3 votes
Accepted

Why MLP cannot approximate a closed shape function?

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, ...
Neil Slater's user avatar
  • 31.7k
3 votes
Accepted

What is the need for so many filters in a CNN?

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are ...
nbro's user avatar
  • 40.2k
3 votes
Accepted

Can predictions of a neural network using ReLU activation be non-linear (i.e. follow the pattern) outside of the scope of trained data?

It isn't too surprising to see behaviour like this, since you're using $\mathrm{ReLU}$ activation. Here is a simple result which explains the phenomenon for a single-layer neural network. I don't have ...
htl's user avatar
  • 1,000
3 votes

What is the proper way to process continuous sequence data, such as time-series, using the Transformer?

Instead of using a token embedding you can use a linear layer. For an input of (10, 5, 4) - (sequence length, batch size, features) you can create a linear layer: ...
Dylan Kerler's user avatar
3 votes
Accepted

Are these book example CNN results realistic?

I was able to run the code without "any" modifications on Tensorflow 2.4.0, just had to replace the imports: ...
NikoNyrh's user avatar
  • 757
3 votes
Accepted

Weights initialization once the Neural Network is trained

Regarding your first code snippet, there is no weight storing or continuation of training between the different CV folds whatsoever: each model is trained anew with the respective training data of ...
desertnaut's user avatar
  • 1,034
3 votes

Why is a simple regression problem so hard for an MLP to learn?

An interesting problem. This network has only 933 trainable parameters, and obtains MeanAbsolutePercentageError of 0.01 - 0.04. It is based on a softmax activation, ...
NikoNyrh's user avatar
  • 757
3 votes
Accepted

If the output is 0.09, does this mean that the prediction is class 1 or 0?

If you are using binary cross-entropy as loss function, the output of Sigmoid functions represent the probability of $\mathbb{P}[y=1]$. Hence, it means that $\mathbb{P}[y=0] = 1 - \mathbb{P}[y=1]$. ...
OmG's user avatar
  • 1,816

Only top scored, non community-wiki answers of a minimum length are eligible