New answers tagged convolutional-neural-networks
-1
The networks with one-dimensional convolution over the temporal dimensions are called Temporal Convolutional Networks. The authors of this study claim that TCNs are comparable to RNNs in performance on common tasks associated with RNNs.
1
This seems to be a known problem, and intuitively seems reasonable. You might be interested in the paper Adversarial Training Can Hurt Generalization.
The authors suggest that this might be because training on the perturbed data requires the model to learn more robust features, which means more samples are required to obtain performance comparable to a model ...
1
The authors use so-called embeddings, it's a form to represent the images in some meaningful vector form.
The procedure to get embedding as follows. First, keep in mind most of the popular convolutional net architectures starts with convolutional layers and then have few fully connected layers. Then do the following.
Train the full network with one-hot ...
3
The AlphaZero paper mentions an "evaluation" step that seems to deal with the the problem similar to yours:
... we evaluate each new neural network checkpoint against the current best network $f_{\theta_*}$ before using it for data generation ... Each evaluation consists of 400 games ... If the new player wins by a margin of > 55% (to avoid ...
0
Here are a couple of Kaggle Kernels, Notebooks and Tutorials for Image Captioning.
Kaggle Kernel | Neural Image Captioning: 🌄 -> 💬
Kaggle Kernel | Show Attend and Tell
Kaggle Kernel | Flickr Image Captioning : TPU, TF2 & Glove
Tensorflow Tutorial | Image captioning with visual attention
Show and Tell: A Neural Image Caption Generator by Vinyals ...
2
Not quite sure about RNN & LSTM (and it always depends on the task), but for CNN the answer is clearly no; CNN routinely include FC layers. Quoting from the highly popular (and recommended) Stanford course CS231n: Convolutional Neural Networks for Visual Recognition:
ConvNet Architectures
We have seen that Convolutional Networks are commonly made up of ...
1
It looks like your network is overfitting, because the training loss carries on decreasing to zero even though validation loss levels off, and then starts to increase again.
I would guess that your network is essentially "memorising" the training examples because you're getting a near zero loss in training.
You could try:
applying some form of ...
0
You may use Keras model which is provided in TensorFlow. Keras model lets you evaluate a batch of different number of samples compared to when training.
For example, when training:
myModel.fit(x=X, y=Y, batch_size=BATCH_SIZE, epochs=EPOCHS, verbose=1)
Evaluate every single sample to get output loss of each sample (https://www.tensorflow.org/api_docs/python/...
0
You are heading in the right direction to make an audio-based classifier. This is not quite the same as providing a similarity metric between two pieces of audio, but it may do as a first attempt. You could use the "probability that this audio is a Queen song" as a proxy for similarity.
First of all, is 300 songs even enough?
Nowhere near enough ...
2
Can residual connections be beneficial when we have a small training dataset?
The usual rule of data science investigations applies here: Try it, measure the results, then you will know.
It is very hard to tell, a priori, whether a specific architectural or hyperparameter choice will impact the performance of a neural network on a given problem.
In this ...
1
Accuracy is a good measure if our classes are evenly split, but is very misleading if we have imbalanced classes.Always use caution with accuracy. You need to know the distribution of the classes to know how to interpret the value.
2
You have two questions in one.
Is it maxpool that ruins the model?
I would say no, the maxpool is a standard operation for convolution networks, it down-samples the intermediate representation to reduce the necessary computations, improve the regularization, and adds translation invariance to some degree. Originally averaging was used to downsample over ...
1
The Standard Image Captioning Pipeline is to train the model in a single batch(or mini-batch) i.e. get the features from the CNN Image encoder and then feed that into an RNN decoder (features + Real Captions) to produce output captions for the Image.
The training loop in PyTorch would look something like this:
# zero the parameter gradients
decoder.zero_grad(...
0
I think your notations are unclear, but I can give an answer based on what you probably meant. For example, $\frac{\partial{L}}{\partial{W^x}}$ should be replaced by $(\nabla_{W^x_{j:}}L)_{j=1, ...,n}$ (assuming everything stays in $\mathbb{R}^n$). Also your expression for $\frac{\partial{L}}{\partial{W^x}}$ is wrong, even accounting for the notation.
Since $...
Top 50 recent answers are included
