# Tag Info

Accepted

### Why are the initial weights of neural networks randomly initialised?

You shouldn't assign all to 0.5 because you'd have the "break symmetry" issue. http://www.deeplearningbook.org/contents/optimization.html Perhaps the only property known with complete ...
• 1,380

### How to train a neural network for a round based board game?

Great question! NN is very promising for this type of problem: Giraffe Chess. Lai's accomplishment was considered to be a pretty big deal, but unfortunately came just a few months before AlphaGo ...
• 6,067

### Can LSTM neural networks be sped up by a GPU?

From Nvidia www (https://developer.nvidia.com/discover/lstm): Accelerating Long Short-Term Memory using GPUs The parallel processing capabilities of GPUs can accelerate the LSTM training and ...
• 1,251
Accepted

### Is it okay to use publicly available Instagram videos to train an AI?

Under US copyright law, this is probably fair use ...but beware of memorization. You may run into more trouble if the AI outputs things very similar to the original work. Also, consult a lawyer to ...
• 196

### How to train a neural network for a round based board game?

I'm a chess player and my answer will be only on chess. Training a neural network with reinforcement learning isn't new, it has been done many times in the literature. I'll briefly explain the common ...
• 1,380

### Why are the initial weights of neural networks randomly initialised?

The initial weights in a neural network are initialized randomly because the gradient based methods commonly used to train neural networks do not work well when all of the weights are initialized to ...
Accepted

### Is pooling a kind of dropout?

Dropout and Max-pooling are performed for different reasons. Dropout is a regularization technique, which affects only the training process (during evaluation, it is not active). The goal of dropout ...
• 426
Accepted

### Is a GPU always faster than a CPU for training neural networks?

This changes according to your data and complexity of your models. See following article by microsoft. Their conclusion is The results suggest that the throughput from GPU clusters is always ...
• 196

### What is the name of a human-inspired machine learning approach?

If it was based on how the human brain learns, it might have used hebbian learning. One example for such a network would be HTM.
• 4,007
Accepted

### What are the best known gradient-free training methods for deep learning?

There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing,...
Accepted

### What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)

Usually the problem is to fit the model into video RAM. If it does not, you cannot train your model at all without big efforts (like training parts of the model separately). If it does, time is your ...
• 176

### How do I choose the optimal batch size?

Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch: If you have a small training set, use batch gradient descent (m < 200) In ...
Accepted

### What is the "thing" which is trained in AI model training

This answer applies to Machine Learning (ML) part of AI, as that seems to be what you are asking about. Please bear in mind that AI is still a broad church, including many other techniques than ML. ML,...
• 23.8k
Accepted

### What is the difference between training and testing in reinforcement learning?

What is reinforcement learning? In reinforcement learning (RL), you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. On each time step $t$, ...
• 33.8k

### Should I continue training if the neural network attains 100% training accuracy?

First of all, as mentioned by @Neil Slater in the comment - you need to have three splits into the train, validation and test set. One sometimes disregards the difference between validation and test ...

### What is the reason we loop over epochs when training a neural network?

I am not expert on optimization, but I can share with you my knowledge of the topic. I think that the source of your confusion is that you assumed that, after the first epoch, we have reached a local ...
• 33.8k
Accepted

### How to shorten the development time of a neural network?

Your scenario is common. The most straightforward approach is to subsample your data randomly. Unless your data or your model has strong bias, your performance to the smaller data set should be ...
• 1,380
Accepted

### How can action recognition be achieved?

There are several approaches as to how this can be achieved. One recent study from 2015 about Action Recognition in Realistic Sports VideosPDF uses the action recognition framework based on the three ...
• 10k

### How can action recognition be achieved?

This study from 2012 uses 3D convolutional neural networks (CNN) for automated recognition of human actions in surveillance videos. The 3D CNN model extracts features from both the spatial and the ...
• 10k
Accepted

### Why is my test error lower than the training error?

You use dropout during traing to reduce overfitting, but this reduces the training accuracy. The dropout will not be used during testing, therefore the accuracy will be higher. That's normal behavior ...
• 1,644
Accepted

### How to refine K-means clustering on a data set?

The usual parameters to adjust in a k-means: Number of clusters (recall many clusters can have same label). Distance definition (euclidean is the most basic, Gauss is an improvement) Selection of ...
• 1,251
Accepted

### Is random initialization of the weights the only choice to break the symmetry?

Randomising just b sort of works, but setting w to all zero causes severe problems with vanishing gradients, especially at the ...
• 23.8k
Accepted

### Can LSTM neural networks be sped up by a GPU?

I found that there are cuDNN accelerated cells in Keras, for example, https://keras.io/layers/recurrent/#cudnnlstm. They are very fast. The normal LSTM cells are faster on CPU than on GPU.
• 269

### Why does training an SVM take so long? How can I speed it up?

The most likely explanation is that you're using too many training examples for your SVM implementation. SVMs are based around a kernel function. Most implementations explicitly store this as an NxN ...
• 8,867

### Which layer in a CNN consumes more training time: convolution layers or fully connected layers?

As CNN contains convolution operation, but DNN uses constructive divergence for training. CNN is more complex in terms of Big O notation. For reference: See Convolutional Neural Networks at ...

### How do I choose the optimal batch size?

From the blog A Gentle Introduction to Mini-Batch Gradient Descent and How to Configure Batch Size (2017) by Jason Brownlee. How to Configure Mini-Batch Gradient Descent Mini-batch gradient descent ...

### Iteratively and adaptively increasing the network size during training

Neuroevolution Through Augmenting Topologies or NEAT may be what you are referring to. The original paper by Kenneth O. Stanley is here NEAT combines a neural network and a genetic algorithm. Instead ...
• 319