People who code: we want your input. Take the Survey
9

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...


5

You should choose the model A. The loss is just a differentiable proxy for accuracy. That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine the data which produce high loss and check for presence of overfitting or incorrect labels. If the higher loss is due to a regularizer then reducing the ...


4

In machine learning, the accuracy is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives ($\mathrm {TP}$) and true negatives ($\mathrm {TN}$), so the usual formula to calculate the accuracy is the following one (your first one). \begin{align} \text{Accuracy}=\frac {\...


3

It is common during the training of Neural Networks for accuracy to improve for a while and then get worse -- in general, This is caused by over-fitting. It's also fairly common for the Neural Network to "get UNLUCKY and get knocked into a BAD sectors of parameter space corresponding to a sudden decrease in accuracy -- sometimes it can recover from this ...


3

There are a few issues you need to address first. Normalise your data. You should try and keep your values for each input in a good range, otherwise you're never going to train anything useful. A simple way of doing this could be to divide each value by the maximum value for that input. This will ensure they are between 0 and 1, or you could divide by the ...


3

This partly answer to question 1. There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data. The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure ...


3

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space. In addition, how this is actually parallelized might make a difference, including the order of operations and ...


3

For the precision metric for example you have: $$ Precision = \frac{TP}{TP+FP}, $$ with TP = True Positive and FP = False Positive. Imagine you have the following values: Image 1: $TP = 2, FP = 3$ Image 2: $TP = 1, FP = 4$ Image 3: $TP = 3, FP = 0$ The precision scores as you calculated will be: Image 1: $2/5$ Image 2: $1/5$ Image 3: $1$ Your average ...


3

You should note that both your results are consistent with a "true" probability of 87% accuracy, and your measurement of a difference between these models is not statistically significant. With an 87% accuracy applied at random, then there is approx 14% chance of getting the two extremes of accuracy you have observed by chance if samples are chosen randomly ...


2

You can not use error to reliably measure accuracy. Error is best used as a measure of how fast the model is currently learning. As an example, using different loss functions (cross entorpy vs MSE) results in massively different values for the error at similar accuracy. Also considering this, an error of 0.0000000001 quite often has lower validation set ...


2

Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead. A true positive is when you predict a car in a place and there is a car in that place. A predicted positive is every car you ...


2

It is explained in this CrossValidated post. Top1 accuracy means the best guess (class with highest probability) is the correct result 58.9% of the time, while top5 accuracy means the correct result is in the top 5 best guesses (5 classes with highest probabilities) 87.7% of the time.


2

Neither of the above mentioned methods could be a potent indicator of the performance of a model. A simple way to train the model just enough so that it generalizes well on unknown datasets would be to monitor the validation loss. Training should be stopped once the validation loss progressively starts increasing over multiple epochs. Beyond this point, ...


2

The short answer is no, you shouldn't do that. There is a "distribution shift" thing when you have different x-y relation on the validation set then on the train set. The distribution shift would deteriorate your model performance and you should try to avoid that. The reason it's bad - ok, you find the way to fix the model for validation data, but ...


2

You have two questions in one. Is it maxpool that ruins the model? I would say no, the maxpool is a standard operation for convolution networks, it down-samples the intermediate representation to reduce the necessary computations, improve the regularization, and adds translation invariance to some degree. Originally averaging was used to downsample over ...


1

Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers). For CNNs all the standard convolutional layers - Conv1D, Conv2D and Conv3D,- have the padding argument. The padding values can be valid or same for 2d and 3d convolutions. And extra causal type of padding is possible for 1d ...


1

In your case the most probable explanation would be the case of overfitting. The model with too many hidden layers have lots of parameters. By means of all these parameters the model is remembering stuff from the training data itself instead of generalizing by learning the useful patterns. As a rule of thumb if you increase the number of hidden layers more ...


1

Accuracy is a good measure if our classes are evenly split, but is very misleading if we have imbalanced classes.Always use caution with accuracy. You need to know the distribution of the classes to know how to interpret the value.


1

I'll try to answer on more general questions Is it ok that model performs better on validation, then on train? It's certainly fine if you use techniques like dropout or data augmentation and the difference is not that big. Because in case of dropout for train you use part of the network, and for validation the whole. I'm suspicious my model is too good. ...


1

In fact, choosing smartly the values of the image augmentation can help the performance of your system. Where I work we developed an object detector for cars. We had the following image augmentation parameters: Apect ratio distorsion: it changed the cars dimensions Additive noise: it blurred the image Change colorspace: change the cars colors Saturation ...


1

I shall suggest one more popular metric for this. Davies Bouldin Score (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.davies_bouldin_score.html#sklearn.metrics.davies_bouldin_score). You can also take a look at the clustering metrics in scikit documentation (https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics).


1

You can compute "Silhouette Coefficient" for your aim. Its values mean: 1: Means clusters are well apart from each other and clearly distinguished. 0: Means clusters are indifferent, or we can say that the distance between clusters is not significant. -1: Means clusters are assigned in the wrong way. Also other measures such as purity and mutual ...


1

Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at this question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented. Additional note: If you use batch ...


1

This looks like it could be a homework problem, so consider updating it with the homework tag if so. The second model has the same precision, but worse recall, than model 1. Therefore we would rather have model 1 than model 2. The third model has worse recall than model 1, and worse precision than model 1, therefore we would rather have model 1 than model ...


1

Just as a general remark, notice that technically we don't use the term "accuracy" for regression settings, such as yours - only for classification ones. If RMSE is 'in the units of the quantity being estimated', does this mean we can say: "The network is on average (1-SQRT(0.019))*100 = 86.2% accurate"? No. The advantage of the RMSE, as you have ...


1

I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.


1

It depends on your application! Imagine a binary classifier that is always very "confident" - it always assigns P=100% to Class A and 0% to Class B, or vice versa (sometimes wrong, never uncertain!). Now imagine a "humble" model that is perhaps fractionally less accurate, but whose probabilities are actually meaningful (when it says "Class A with ...


Only top voted, non community-wiki answers of a minimum length are eligible