I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A < B < C, e.g. low, medium, high, very high. What loss function and activation function for the output layer should I use to take advantage of the class hierarchy, so that true A and predicted C is penalized more than true A and predicted B?

My ideas are:

1) To assign some value to each category, use one output unit with the sigmoid activation and RMS loss function. Then to assign each class to an interval, e.g. 0-033 - class A, 0.33-0.66 class B 0.66-1 - class C. It seem to do the trick, but can favor the extreme categories over the middle ones.

2) Use K softmax output units, integer labels instead of one-hot encoded and the sparse categorical crossentropy loss function. In this case I am not sure how exactly sparse categorical crossentropy works and if it really takes into account the hierarchy.

  • $\begingroup$ Welcome to SE:AI! $\endgroup$ – DukeZhou Apr 14 at 22:17
  • $\begingroup$ Thank you Duke ;) Really nobody can help? $\endgroup$ – Karol Borkowski Apr 15 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.