New answers tagged

4

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model. This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...


0

Seems like you have a CUDA version conflict. Remove the existing CUDA 10.2 and install CUDA 10.0 (going by your missing libraries, it requires a v10.0). You can find the archived releases here: https://developer.nvidia.com/cuda-toolkit-archive


0

We also work with Python in our company. One of the sphere that we use it for is fast prototyping and building highly scalable web applications. For over two decades, our Python developers have been providing businesses with full-stack web-development services, client-server programming and administration. We help our clients build high-load web portals, ...


1

To emphasize (and this is not emphasized in this answer), in the case of neural networks, the biases or, more precisely, the connections (or weights) between biases and other neurons are also learnable parameters, so the back-propagation algorithm calculates a gradient of the loss function that contains the partial derivatives with respect to the connections ...


1

In a simple feed-forward network, each artificial neuron has a separate bias value. This allows for greater flexibility for the output layer function than if each neuron had to use a single whole-layer bias. Although not an absolute requirement, without this arrangement it may become very hard to approximate some functions. Moving from a bias vector to a ...


3

TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. Both provide machine learning libraries to accomplish various tasks and do the job. TensorFlow is a powerful and deep learning tool with active visualization and ...


1

Here's a link to some benchmarks that should give you some insights. In my experience (I've used systems with both 1080s and V100s) I've found that as of about a year ago, a lot of the common tools couldn't use the V100s well. Until we started doing some manual optimization, the 1080s were comparable if not better on common tasks. Of course, once we put ...


4

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


2

See comprehensive answer here; to paste a snippet, below is complete code for fixing a random seed: def reset_seeds(reset_graph_with_backend=None): if reset_graph_with_backend is not None: K = reset_graph_with_backend K.clear_session() tf.compat.v1.reset_default_graph() print("KERAS AND TENSORFLOW GRAPHS RESET") # ...


3

As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, ...


2

Use seed for random functions. For example if you are using numpy random function from numpy.random import seed seed(1) Read more about reproducible results here, https://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Set PYTHONHASHSEED environment variable at a fixed value import os os.environ['PYTHONHASHSEED'] = str(1) https://...


0

The whole idea behind those distributed optimization methods is that data should be local in every node/worker. Thus, if you only send the loss value to the central node, this node can't compute the gradients of this loss, and thus can't do any training. However, if you don't want to send gradients, a family of distributed optimization algorithms called ...


Top 50 recent answers are included