4
$\begingroup$

When training a deep network to learn object classification from a set like ImageNet, we minimize the cross entropy between the ground truth and the predicted categories. This is done in a supervised way. It is my understanding that you can separate categories in an unsupervised way using principal component analysis, but I have never seen that in a deep network. I am curious if this can be done easily in the last case. One possible way to do this would be to minimize a loss that favors categorization into one-hot vectors (this would only guarantee that an image is classified into a single category, rather that the correct category, though). Has this been done, or is there any reason why not?

$\endgroup$
1
$\begingroup$

I did an experiment, took a trained densenet121 and kept the bottom layers. I trained the FC layer to a softmax and then to a lambda layer that normalized the vector. I trained the network with imagenet to make the outputs the most far a away from (1,1,1,1,1...1) as possible, so I would get one hot vectors. I did, but the network trained to a single category (put all in the same hot vector). Then I added a penalty that encourages to make vectors different is their features are also different. it improved a little, a dozen categories instead of one, bout noting close to a thousand, the available number.

I am posting this so nobody wastes his time in a silly idea like this one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.