8 votes
Accepted

Neural network for game

Neural networks do not directly take actions in games. Instead, some code needs to supply the current state of the game to the neural network, interpret its output and take the action. Typically yet ...
Neil Slater's user avatar
  • 30.3k
4 votes
Accepted

Which epoch is the best for me to choose?

The first question is not well defined. What does normal mean? It is clearly decreasing, so that is good. If you are asking if the fluctuations are normal, yes that is not uncommon. This is not an ...
chessprogrammer's user avatar
3 votes

Does it make sense to apply batch normalization to a batch size of 1?

It's not possible to use batch normalization with a batch size of 1. Batch normalization requires you to calculate the variance of activation values in the current batch, and variance is undefined for ...
Johann's user avatar
  • 31
2 votes
Accepted

Why is it called multi-headed attention?

The original paper "Attention is all you need" mentions the following. " Instead of performing a single attention function with $ d_{model} $-dimensional keys, values and queries, we ...
Sathishkumar Thirumalai's user avatar
2 votes
Accepted

In multilayer perceptron neural networks, the more complex a dataset is, the more difficult it will be to find adequate initial random weights?

but produce the same result. this is a fake news going around the ML community, they will have comparable performances, but the predictions won't be the same. And no, initialization does not aim to ...
Alberto's user avatar
  • 1,358
1 vote

What are the techniques used to initialize weights for neural networks?

Both Keras and PyTorch provide a range of initialization classes and functions. Of these, probably the two most commonly used are the Glorot (Xavier in Pytorch) and He (Kaiming in pytorch) ...
Lynn's user avatar
  • 131
1 vote

Types of activation functions used in neural networks

Let me give a very easy explanation... Activation Functions in neural networks are useful as they introduce non-linearity into the model, which helps in complex data pattern understanding. Sigmoid ...
Hiren Namera's user avatar
1 vote

Is size of trained model on disk a good measure of model complexity?

The title of your question asks about model complexity. Yet the body of your question talks about this metric as being useful for embedded systems like a smartphone, which have more limited memory. I ...
chessprogrammer's user avatar
1 vote

Can I do incremental learning with different loss function in neural networks?

Yes this is possible. Many hyperparameters of training can be changed at any time when training a neural network model, and that can include the objective function. You can also change the dataset, ...
Neil Slater's user avatar
  • 30.3k
1 vote

How can I deal with random weights initialisation when predicting a time-series sine function?

From comments, you are testing your ideas with a very small neural networks. The highly variable end result with large dependency on initial conditions is a common result of working with small numbers ...
Neil Slater's user avatar
  • 30.3k
1 vote
Accepted

What is the potential issue of nested neural networks

Isn't this just a very short recurrent neural network? Same issues apply, although they are less severe since you aren't applying as many recurrent iterations. Once you start "nesting" them ...
NikoNyrh's user avatar
  • 737

Only top scored, non community-wiki answers of a minimum length are eligible