Hot answers tagged

33

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model. This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...


15

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to have some powerful understanding when restricted to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in the works....


10

This should make a difference, but how big is the difference heavily depends on your task. However, generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small, the batch/minute will be very low and therefore decreasing training speed severely. However,...


9

From Nvidia www (https://developer.nvidia.com/discover/lstm): Accelerating Long Short-Term Memory using GPUs The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and 140x higher throughput during inference when compared to ...


7

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have ...


5

I found that there are cuDNN accelerated cells in Keras, for example, https://keras.io/layers/recurrent/#cudnnlstm. They are very fast. The normal LSTM cells are faster on CPU than on GPU.


4

It actually depends on a couple of things here - How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred. As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes ...


3

No, Neural Networks do not have such a guarantee. In fact, I don't believe any kind of classifier in the entire field of Machine Learning has such a guarantee, though some may be slipping my mind... For an easy counterexample, consider what happens if you have two instances with precisely identical inputs, but different output labels. If your classifier is ...


3

You can introduce another class to your network - "not a book". After that, you will need to add new data to your dataset, random images that do not contain books to classify and train your network on that data. So when your network won't see a book it will output high probability for "not a book" class, if an image with a book will be shown to the network ...


3

The inputs that you describe seem like they should be sufficient for a DQN-based agent to learn a good strategy for playing Minesweeper, regardless of whether or not the starting layout changes. The inputs contain all information that is necessary. However, the problem certainly becomes much easier (probably too easy) if the initial problem is always the ...


3

Instead of using the Embedding() layer directly, you can create a new bertEmbedding() layer and use it instead. # Sample code # Model architecture # Custom BERT layer bert_output = BertLayer(n_fine_tune_layers=10)(bert_inputs) # Build the rest of the classifier dense = tf.keras.layers.Dense(256, activation='relu')(bert_output) pred = tf.keras.layers....


3

I'm going to post another guess to this question - it won't be a complete answer, but hopefully it'll provide some direction towards finding a more legitimate answer. The feed-forward networks as suggested by Vaswani are very reminiscent of the sparse autoencoders. Where the input / output dimensions are much greater than the hidden input dimension. If you ...


3

CNNs learn convolutional filters that get trained on finding local, recurring patterns in some kind of image/volume data. 1D convolution is actually a thing, but I think what would be more suitable for your case is using Recurrent Neural Nets. They are specifically designed for working on time series-es of heterogeneous data. Update: I would like to ...


3

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. encoder-decoder arch., LSTM) specifics of connections between network layers (e.g. residual-, dense-, skip-connections) specifics of individual components of the ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


3

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account? Well, an LSTM model is not ...


3

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder how well an "other" class would actually work. The ways in which "other" differs from "digit" has infinite variability (or at least its only limitation is the ...


3

When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your model worked on real world users.However, if your model must have to measure the accuracy try it with the RMSE score as it will penalise you more for being ...


3

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, tanh or hyperbolic tangent). Assuming at least one hidden layer, then the universal approximation theorem applies. How closely you can approximate any given function is limited by the number of ...


3

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are the learnable parameters of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-...


3

I was able to run the code without "any" modifications on Tensorflow 2.4.0, just had to replace the imports: import keras from keras.datasets import mnist ... -> import tensorflow.keras as keras from tensorflow.keras.datasets import mnist ... Output: Epoch 1/12 469/469 [==============================] - 4s 7ms/step - loss: 2.2914 - accuracy: 0....


2

I am new to AI but this is something I can think of. There might be other much better ways. Or even functions in scikit learn to do it 1) create a list of all the 22 models 2) iterate over the models one by one and use model.predict() for each model and store the hotencoded output to another list or numpy array. 3) Take average of the output list or ...


2

I think it is not a very good idea because Im pretty sure you used for learning all these 22 CNN same images and even same way for giving them a batches of images. So basically in a result you would have almost the same 22 classifiers.


2

There are many ways of constraining the network's output. Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by 180 to adapt to your output. Since the layer is part of the optimization process, gradients will be learned correctly. However, there are a few issues with it, ...


2

After training, all standard models are deterministic (the process each input goes thru is set). In essence, during training the model attempts to learn the distribution of the training dataset. Whether it is able to depends on the size of the model, if it is big enough, it can simply "memorize" all the training samples and result in perfect accuracy on ...


2

The probability map / output isn't produced by your loss function, but your output layer, which is activated either by softmax or sigmoid. In other words, your dice loss output is also a probability map. It's simply very confident in itself. If you forget about the problem with potential overfitting for a moment and train your binary crossentropy model ...


2

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network. Then, there are several methods and considerations: You can use the weight resulted from your first ...


2

Why would giving my AI more data make it perform worse? A lot of possible reasons: In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others. Data quality: In practice, improving the quality of data often ...


2

I had a similar problem with a 2D convolution on a hexagonal grid while working on a diffusion problem and stumbled upon this question. Rather than using cube coordinates, you could use doubled coordinates, which I found much easier to save in a 2D array. An example kernel that only changes the direct neighbours and the cell itself would be this kernel = ...


2

I cannot comment much on your setup for inputs and outputs. It seems adequate to get some control, but does not cover the fully Markov state for the game, so I would expect that will limit the agent from ever being truly optimal. I would expect it to learn to play the game though, if you were implementing Q learning with a neural network correctly. In your ...


Only top voted, non community-wiki answers of a minimum length are eligible