Hot answers tagged

4

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


3

Some other details you could mention are: total number of model parameters (e.g. 1.2M or 0.15M) & depth of the network (e.g. 38-layered network) family/style of the network architecture (e.g. encoder-decoder arch., LSTM) specifics of connections between network layers (e.g. residual-, dense-, skip-connections) specifics of individual components of the ...


3

As @codeblooded said, you should set random seed for numpy and keras, and also set pythonhashseed. The seeds set the state of the random number generator which makes the results different. This method only works when you train the network on CPU. The problem with getting same result on GPU every single time is that cuDNN is not deterministic. Specifically, ...


2

Use seed for random functions. For example if you are using numpy random function from numpy.random import seed seed(1) Read more about reproducible results here, https://machinelearningmastery.com/reproducible-results-neural-networks-keras/ Set PYTHONHASHSEED environment variable at a fixed value import os os.environ['PYTHONHASHSEED'] = str(1) https://...


2

A couple of points: Have you firstly scaled your data, e.g. using MinMaxScaler? This could be one reason why your loss readings remain high. Additionally, consider that while Dropout can be useful for reducing overfitting, it is not necessarily a panacea. Let's take an example of using LSTM to forecast fluctuations in weekly hotel cancellations. Model ...


2

Neural networks can have a lot of different structures. CNNs can have a number of parameters that ranges from a few thousands to several millions. In general you aim to increase the number of filters and reduce the first 2 dimensions, as you go deeper in the network. So if you had Conv -> pool -> Conv -> pool -> ... , you could do for example ...


2

See comprehensive answer here; to paste a snippet, below is complete code for fixing a random seed: def reset_seeds(reset_graph_with_backend=None): if reset_graph_with_backend is not None: K = reset_graph_with_backend K.clear_session() tf.compat.v1.reset_default_graph() print("KERAS AND TENSORFLOW GRAPHS RESET") # ...


2

This question is very broad, so let me attempt to answer it using my own background in time series analysis. As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account? Well, an LSTM model is not ...


1

Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models? Of course, the model that achieves state-of-the-art performance depends on the problem, available datasets, etc., so a comprehensive comparison between traditional ML models and deep neural networks is not ...


Only top voted, non community-wiki answers of a minimum length are eligible