Hot answers tagged

33

Keras is a simple and high-level neural networks library, written in Python, that works as a wrapper for Tensorflow and Theano. It's easy to learn and use. Using Keras is like working with Lego blocks. It was built so that people can do quick experiments and proofs-of-concept before launching into a full-scale build process. With that in mind, it was made ...


33

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model. This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...


15

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to have some powerful understanding when restricted to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in the works....


8

From nvidia www (https://developer.nvidia.com/discover/lstm): Accelerating Long Short-Term Memory using GPUs The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and 140x higher throughput during inference when compared ...


8

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...


7

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have ...


4

I found that there are cuDNN accelerated cells in Keras for example: https://keras.io/layers/recurrent/#cudnnlstm They very fast. The normal LSTM cells are faster on CPU then on GPU. Also see here for a comparisem: https://wiki.eniak.de/ml/geschwindigkeitsvergleich_keras_lstm_und_cudnnlstm


4

It actually depends on a couple of things here - How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred. As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes ...


4

As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, ...


3

The inputs that you describe seem like they should be sufficient for a DQN-based agent to learn a good strategy for playing Minesweeper, regardless of whether or not the starting layout changes. The inputs contain all information that is necessary. However, the problem certainly becomes much easier (probably too easy) if the initial problem is always the ...


3

No, Neural Networks do not have such a guarantee. In fact, I don't believe any kind of classifier in the entire field of Machine Learning has such a guarantee, though some may be slipping my mind... For an easy counterexample, consider what happens if you have two instances with precisely identical inputs, but different output labels. If your classifier is ...


3

Instead of using the Embedding() layer directly, you can create a new bertEmbedding() layer and use it instead. # Sample code # Model architecture # Custom BERT layer bert_output = BertLayer(n_fine_tune_layers=10)(bert_inputs) # Build the rest of the classifier dense = tf.keras.layers.Dense(256, activation='relu')(bert_output) pred = tf.keras.layers....


3

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. encoder-decoder arch., LSTM) specifics of connections between network layers (e.g. residual-, dense-, skip-connections) specifics of individual components of the ...


3

See comprehensive answer here; to paste a snippet, below is complete code for fixing a random seed: def reset_seeds(reset_graph_with_backend=None): if reset_graph_with_backend is not None: K = reset_graph_with_backend K.clear_session() tf.compat.v1.reset_default_graph() print("KERAS AND TENSORFLOW GRAPHS RESET") # ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


3

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account? Well, an LSTM model is not ...


3

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder how well an "other" class would actually work. The ways in which "other" differs from "digit" has infinite variability (or at least its only limitation is the ...


3

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, tanh or hyperbolic tangent). Assuming at least one hidden layer, then the universal approximation theorem applies. How closely you can approximate any given function is limited by the number of ...


3

Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content? The filters (aka kernels) are the learnable parameters of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-...


2

input. rather than concatenating four consecutive frames, try using the difference between two consecutive frames as input. network architecture. a vanilla multi layer net will work better than a convnet in this case. you dont really need spatial/translational invariance for pong, especially if you use frame differences as input.


2

I am new to AI but this is something I can think of. There might be other much better ways. Or even functions in scikit learn to do it 1) create a list of all the 22 models 2) iterate over the models one by one and use model.predict() for each model and store the hotencoded output to another list or numpy array. 3) Take average of the output list or ...


2

I have found a hack to integrate keras with tensorflow object detection API. This hack works if you have trained the keras classification model with tensorflow backend. If above is the case you can extend the classification model to a object detection model by first converting the keras checkpoint to a tensorflow checkpoint then in the object detection API ...


2

I think it is not a very good idea because Im pretty sure you used for learning all these 22 CNN same images and even same way for giving them a batches of images. So basically in a result you would have almost the same 22 classifiers.


2

Why would giving my AI more data make it perform worse? A lot of possible reasons: In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others. Data quality: In practice, improving the quality of data often ...


2

I had a similar problem with a 2D convolution on a hexagonal grid while working on a diffusion problem and stumbled upon this question. Rather than using cube coordinates, you could use doubled coordinates, which I found much easier to save in a 2D array. An example kernel that only changes the direct neighbours and the cell itself would be this kernel = ...


2

There are many ways of constraining the network's output. Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by 180 to adapt to your output. Since the layer is part of the optimization process, gradients will be learned correctly. However, there are a few issues with it, ...


2

After training, all standard models are deterministic (the process each input goes thru is set). In essence, during training the model attempts to learn the distribution of the training dataset. Whether it is able to depends on the size of the model, if it is big enough, it can simply "memorize" all the training samples and result in perfect accuracy on ...


2

Your question is really broad.. But there are many manuals and tutorials which describe how to do it step by step. My personal favorite would be: Practical Text Classification With Python and Keras - It has a very detailed explanation of every step of the implementation while remaining practical. You can also try: A Word2Vec Keras tutorial - This one ...


2

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network. Then, there are several methods and considerations: You can use the weight resulted from your first ...


2

Ok, I made a dumb mistake. Please disregard the above panicked ranting. In case anyone else bumps up against something like this however, the issue was resolved just by remembering to clear my session at the end of each run: from keras import backend as K ...code... K.clear_session()


Only top voted, non community-wiki answers of a minimum length are eligible