0
$\begingroup$

I'm a bit of a CNN newbie, and I'm trying to train one to image classify pictures of pretty similar looking particles. I'm making the inputs and labels by hand from a set of 48x48 grayscale images, and labeling them with a one-hot vector based on their position in the sequence (for example, the 400/1000th image might have a one-hot in the 4th position if I have 10 categories in the run). I'm using sigmoidal output activation and categorical cross entropy loss. I've played around with a few different optimizers, as well. I'm implementing in python keras.

Unfortunately, although I have pretty good accuracy numbers for the training and validation, when I actually look at the outputs being produced, it generally gives multiple categories, which is not at all what I want. For example, if I have 6 categories and a label of 3, it might give the following probability vector:

[ .99 .98 1.0 .99 0.02 0.05 ]

It was my understanding that categorical cross entropy would not allow this type of categorization, and yet it is prevalent in my code. I am under the impression that I'm doing something fundamentally wrong, but I cant figure out what. Any help would be appreciated.

Improve this question
$\endgroup$
3
  • $\begingroup$ when you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1. $\endgroup$
    – Brale
    Apr 8, 2019 at 13:57
  • $\begingroup$ I knew it would be a rookie mistake! Thank you! That fixed that issue, at least. If you want to post this as a full answer I will mark it as a correct fix. $\endgroup$
    – Fred E
    Apr 8, 2019 at 14:18
  • $\begingroup$ Yeah sure, thanks. $\endgroup$
    – Brale
    Apr 8, 2019 at 14:40

1 Answer 1

Reset to default
1
$\begingroup$

When you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .