3
The AlphaZero paper mentions an "evaluation" step that seems to deal with the the problem similar to yours:
... we evaluate each new neural network checkpoint against the current best network $f_{\theta_*}$ before using it for data generation ... Each evaluation consists of 400 games ... If the new player wins by a margin of > 55% (to avoid ...
2
You have two questions in one.
Is it maxpool that ruins the model?
I would say no, the maxpool is a standard operation for convolution networks, it down-samples the intermediate representation to reduce the necessary computations, improve the regularization, and adds translation invariance to some degree. Originally averaging was used to downsample over ...
2
Can residual connections be beneficial when we have a small training dataset?
The usual rule of data science investigations applies here: Try it, measure the results, then you will know.
It is very hard to tell, a priori, whether a specific architectural or hyperparameter choice will impact the performance of a neural network on a given problem.
In this ...
2
Not quite sure about RNN & LSTM (and it always depends on the task), but for CNN the answer is clearly no; CNN routinely include FC layers. Quoting from the highly popular (and recommended) Stanford course CS231n: Convolutional Neural Networks for Visual Recognition:
ConvNet Architectures
We have seen that Convolutional Networks are commonly made up of ...
1
This seems to be a known problem, and intuitively seems reasonable. You might be interested in the paper Adversarial Training Can Hurt Generalization.
The authors suggest that this might be because training on the perturbed data requires the model to learn more robust features, which means more samples are required to obtain performance comparable to a model ...
1
The authors use so-called embeddings, it's a form to represent the images in some meaningful vector form.
The procedure to get embedding as follows. First, keep in mind most of the popular convolutional net architectures starts with convolutional layers and then have few fully connected layers. Then do the following.
Train the full network with one-hot ...
1
It looks like your network is overfitting, because the training loss carries on decreasing to zero even though validation loss levels off, and then starts to increase again.
I would guess that your network is essentially "memorising" the training examples because you're getting a near zero loss in training.
You could try:
applying some form of ...
1
Accuracy is a good measure if our classes are evenly split, but is very misleading if we have imbalanced classes.Always use caution with accuracy. You need to know the distribution of the classes to know how to interpret the value.
1
The Standard Image Captioning Pipeline is to train the model in a single batch(or mini-batch) i.e. get the features from the CNN Image encoder and then feed that into an RNN decoder (features + Real Captions) to produce output captions for the Image.
The training loop in PyTorch would look something like this:
# zero the parameter gradients
decoder.zero_grad(...
Only top voted, non community-wiki answers of a minimum length are eligible
