1
The required shape of the tensor $T$ depends on the shape of other tensors that are involved in the same operations of that same tensor $T$ and the required/desired shape of the resulting tensor, in the same way that the number of columns of the matrix $M \in \mathbb{R}^{n \times m}$ needs to match the number of rows of the matrix $M' \in \mathbb{R}^{n' \...
1
I commonly use softmax for all 2-class or k-class problems, basically, because I always like to have an output node for each class. For sigmoid, i.e., logistic, you cannot estimate MSE for each sample using the relationship
$E_i = \sum_c^C (y_c - \hat{y}_c)^2$,
where $C$ is the number of classes, $y_c$ is 0 or 1 for true class membership, and $\hat{y}_c$ is ...
1
Sigmoid is used for binary cases and softmax is its generalized version for multiple classes. But, essentially what they do is over exaggerate the distances between the various values.
If you have values on a unit sphere, apply sigmoid or softmax on those values would lead to the points going to the poles of the sphere.
1
This seems to be a known problem, and intuitively seems reasonable. You might be interested in the paper Adversarial Training Can Hurt Generalization.
The authors suggest that this might be because training on the perturbed data requires the model to learn more robust features, which means more samples are required to obtain performance comparable to a model ...
1
Accuracy can sometimes be a very coarse metric. When it is applied to three class problems, people often take the class label with maximum predicted probability and predict that. The probabilities of the individual labels are ignored. I'd recommend that as well as accuracy you calculate sensitivity and specificity for each class and the area under the ROC ...
Only top voted, non community-wiki answers of a minimum length are eligible
