# Tag Info

32

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model. This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...

14

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to have some powerful understanding when restricted to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in the works....

7

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have ...

6

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...

4

It actually depends on a couple of things here - How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred. As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes ...

4

As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, ...

3

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. encoder-decoder arch., LSTM) specifics of connections between network layers (e.g. residual-, dense-, skip-connections) specifics of individual components of the ...

3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...

3

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder how well an "other" class would actually work. The ways in which "other" differs from "digit" has infinite variability (or at least its only limitation is the ...

3

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, tanh or hyperbolic tangent). Assuming at least one hidden layer, then the universal approximation theorem applies. How closely you can approximate any given function is limited by the number of ...

2

This is probably the most major factor: model.compile(loss='categorical_crossentropy', optimizer='adam') you have set the loss function for a multiclass classifier. It is going to have some weird results when values - either predicted or target - are outside of range 0..1 You should use this instead: model.compile(loss='mean_squared_error', optimizer='...

2

You can try using a multi-input model. Here is a recent post with a similar discussion, with the required architecture defined in the answer. Instead of combining the separate models, you can create a model which uses image and numerical data side by side. Keras allows you to use different types of data using multi input structure via functional API. And ...

2

I try to answer the things I know for sure: One effect of bigger images is the increasing computation time due to more pixels (input to your training) 4.Grayscaling reduces the information, which might decrease training time, but also model performance (accuracy, precision, recall). What I have seen is that grayscaling is used in for example face detection ...

2

Neural networks can have a lot of different structures. CNNs can have a number of parameters that ranges from a few thousands to several millions. In general you aim to increase the number of filters and reduce the first 2 dimensions, as you go deeper in the network. So if you had Conv -> pool -> Conv -> pool -> ... , you could do for example ...

2

A couple of points: Have you firstly scaled your data, e.g. using MinMaxScaler? This could be one reason why your loss readings remain high. Additionally, consider that while Dropout can be useful for reducing overfitting, it is not necessarily a panacea. Let's take an example of using LSTM to forecast fluctuations in weekly hotel cancellations. Model ...

2

Ok, I made a dumb mistake. Please disregard the above panicked ranting. In case anyone else bumps up against something like this however, the issue was resolved just by remembering to clear my session at the end of each run: from keras import backend as K ...code... K.clear_session()

2

Use seed for random functions. For example if you are using numpy random function from numpy.random import seed seed(1) Read more about reproducible results here, https://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Set PYTHONHASHSEED environment variable at a fixed value import os os.environ['PYTHONHASHSEED'] = str(1) https://...

2

See comprehensive answer here; to paste a snippet, below is complete code for fixing a random seed: def reset_seeds(reset_graph_with_backend=None): if reset_graph_with_backend is not None: K = reset_graph_with_backend K.clear_session() tf.compat.v1.reset_default_graph() print("KERAS AND TENSORFLOW GRAPHS RESET") # ...

2

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account? Well, an LSTM model is not ...

2

I'm an amateur with neural networks, but I will illustrate my understanding of how this problem comes to be. First, lets see how trivial neural network classifies 2D input into two classes : But in case of complex neural network, the input space is much bigger and the sample data points are much more clustered with big chunks of empty space between them: ...

2

Neither of the above mentioned methods could be a potent indicator of the performance of a model. A simple way to train the model just enough so that it generalizes well on unknown datasets would be to monitor the validation loss. Training should be stopped once the validation loss progressively starts increasing over multiple epochs. Beyond this point, ...

2

$f(x) = x^2 + b$ is a polynomial (more precisely, a parabola) so it is continuous, thus, a neural network (with at least one hidden layer) should be able to approximate that function (given the universal approximation theorem). After a very quick look at your code, I noticed you aren't using an activation function for your dense layers (i.e. your activation ...

2

There have been many researches in dynamic difficulty adjustment (DDA). I see this one is quite explaining: AI for Dynamic Difficulty Adjustment in Games. However, there are many factors when we are trying to do dynamic difficulty adjustment. As explained in paper above, one major problem is it is sometimes hard to make sure the created model will still ...

2

Yes this looks a lot like overfitting. The clue is in the low and slowly decreasing training loss compared to the large increases in validation loss. One simple fix would be to stop training around epoch 50, taking the best cross validation result to select the most general network at that point. However, anything that works to improve stable generalisation ...

2

I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options. I see a little thing that might be making your model a little worse, though: You're concatenating "relu" with "sigmoid". Placing two different nature values in the same array may make it more difficult ...

2

When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your model worked on real world users.However, if your model must have to measure the accuracy try it with the RMSE score as it will penalise you more for being ...

2

You have 3 inputs going to 3 nodes in the input layer. Each connection has a weight so you have 3 X 3 =9 weights. Plus each node has a bias weight so that adds 3 more weights for a total of 12. Your output layer has 3 inputs and is a single node so you have 3 weights for the inputs to the node plus a bias weight for a total of 4. So the total weights in ...

1

The probability map / output isn't produced by your loss function, but your output layer, which is activated either by softmax or sigmoid. In other words, your dice loss output is also a probability map. It's simply very confident in itself. If you forget about the problem with potential overfitting for a moment and train your binary crossentropy model ...

1

Calculating parameter number in a CNN is very straightforward. CNN is composed of different filters which is essentially a 3d tensor. CNN weights are shared meaning they are used multiple times, and reused in different locations. Each layer have n tensors each with dimension w * h * c where w = width, h = height, c = channels (the input channel size), ...

Only top voted, non community-wiki answers of a minimum length are eligible