Hot answers tagged

33

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model. This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...


15

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to have some powerful understanding when restricted to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in the works....


8

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...


7

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have ...


4

As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, ...


3

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder how well an "other" class would actually work. The ways in which "other" differs from "digit" has infinite variability (or at least its only limitation is the ...


3

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, tanh or hyperbolic tangent). Assuming at least one hidden layer, then the universal approximation theorem applies. How closely you can approximate any given function is limited by the number of ...


3

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account? Well, an LSTM model is not ...


3

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. encoder-decoder arch., LSTM) specifics of connections between network layers (e.g. residual-, dense-, skip-connections) specifics of individual components of the ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


3

See comprehensive answer here; to paste a snippet, below is complete code for fixing a random seed: def reset_seeds(reset_graph_with_backend=None): if reset_graph_with_backend is not None: K = reset_graph_with_backend K.clear_session() tf.compat.v1.reset_default_graph() print("KERAS AND TENSORFLOW GRAPHS RESET") # ...


3

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are the learnable parameters of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-...


2

Use seed for random functions. For example if you are using numpy random function from numpy.random import seed seed(1) Read more about reproducible results here, https://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Set PYTHONHASHSEED environment variable at a fixed value import os os.environ['PYTHONHASHSEED'] = str(1) https://...


2

A couple of points: Have you firstly scaled your data, e.g. using MinMaxScaler? This could be one reason why your loss readings remain high. Additionally, consider that while Dropout can be useful for reducing overfitting, it is not necessarily a panacea. Let's take an example of using LSTM to forecast fluctuations in weekly hotel cancellations. Model ...


2

Neural networks can have a lot of different structures. CNNs can have a number of parameters that ranges from a few thousands to several millions. In general you aim to increase the number of filters and reduce the first 2 dimensions, as you go deeper in the network. So if you had Conv -> pool -> Conv -> pool -> ... , you could do for example ...


2

Calculating parameter number in a CNN is very straightforward. CNN is composed of different filters which is essentially a 3d tensor. CNN weights are shared meaning they are used multiple times, and reused in different locations. Each layer have n tensors each with dimension w * h * c where w = width, h = height, c = channels (the input channel size), ...


2

I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options. I see a little thing that might be making your model a little worse, though: You're concatenating "relu" with "sigmoid". Placing two different nature values in the same array may make it more difficult ...


2

Yes this looks a lot like overfitting. The clue is in the low and slowly decreasing training loss compared to the large increases in validation loss. One simple fix would be to stop training around epoch 50, taking the best cross validation result to select the most general network at that point. However, anything that works to improve stable generalisation ...


2

There have been many researches in dynamic difficulty adjustment (DDA). I see this one is quite explaining: AI for Dynamic Difficulty Adjustment in Games. However, there are many factors when we are trying to do dynamic difficulty adjustment. As explained in paper above, one major problem is it is sometimes hard to make sure the created model will still ...


2

$f(x) = x^2 + b$ is a polynomial (more precisely, a parabola) so it is continuous, thus, a neural network (with at least one hidden layer) should be able to approximate that function (given the universal approximation theorem). After a very quick look at your code, I noticed you aren't using an activation function for your dense layers (i.e. your activation ...


2

Neither of the above mentioned methods could be a potent indicator of the performance of a model. A simple way to train the model just enough so that it generalizes well on unknown datasets would be to monitor the validation loss. Training should be stopped once the validation loss progressively starts increasing over multiple epochs. Beyond this point, ...


2

I'm an amateur with neural networks, but I will illustrate my understanding of how this problem comes to be. First, lets see how trivial neural network classifies 2D input into two classes : But in case of complex neural network, the input space is much bigger and the sample data points are much more clustered with big chunks of empty space between them: ...


2

Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models? Of course, the model that achieves state-of-the-art performance depends on the problem, available datasets, etc., so a comprehensive comparison between traditional ML models and deep neural networks is not ...


2

When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your model worked on real world users.However, if your model must have to measure the accuracy try it with the RMSE score as it will penalise you more for being ...


2

You have 3 inputs going to 3 nodes in the input layer. Each connection has a weight so you have 3 X 3 =9 weights. Plus each node has a bias weight so that adds 3 more weights for a total of 12. Your output layer has 3 inputs and is a single node so you have 3 weights for the inputs to the node plus a bias weight for a total of 4. So the total weights in ...


2

Regularizer's are used as a means to combat over fitting.They essentially create a cost function penalty which tries to prevent quantities from becoming to large. I have primarily used kernel regularizers. First I try to control over fitting using dropout layers. If that does not do the job or leads to poor training accuracy I try the Kernel regularizer. I ...


2

A 2D convolution is a convolution where the kernel has the same depth as the input, so, in theory, you do not need to specify the depth of the kernel, if you know the depth of the input. I don't know which library you are referring to (although you tagged your post with TensorFlow and Keras), but, in TensorFlow, you only need to specify the width and height ...


2

Assume the image can contain objects of class $C_1 \dots C_c$. Assume a set of additional inputs that has a meaning of questions as "contains the image a C_i or C_j or ... ?". The main problem for the system is classify the image in classes $C_i$. Second problem is answer the implicit question proposed by the remainder inputs. Thus, better combine ...


2

The main benefit of deep learning is that you don't have to manually design features. Classic Machine Learning algorithms always include the Feature engineering step, whereas neural networks are able to crate features automatically during learning. The classic example is CNN. In the first layer it creates simple features that representing lines, the last ...


1

You don't need to manage negative rewards separately, if you implemented the algorithm correctly it will work regardless if the rewards are negative or not. You seem to be using rewards for the loss but you should be using the return which is the sum of the rewards for some state action pair from that point until the end of trajectory. You also seem to be ...


Only top voted, non community-wiki answers of a minimum length are eligible