0
$\begingroup$

Let's say I have a binary classification problem and I want to solve it by means of FC neural net. So which approach will be correct: 1) define the last layer of NN like this linear(h, 1) and use Binary Cross entropy 2) define the last layer of NN like this linear(h, 2) and use Cross entropy. It seems for me that these two approaches are similar. Is it correct?

P.S. This question is more theoretical one, so no any technicalities related to particular framework.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Binary cross-entropy loss is a specific case for cross-entropy loss. Theoretically, one can also use the normal cross-entropy loss for binary classification. Binary cross-entropy is probably computationally a lot faster than the general cross-entropy loss and is simpler to understand. However, it does exactly the same as cross-entropy loss if performed on two classes.

So the only question left is: 1 or 2 output nodes for binary classification? If you use a sigmoid activation for the variant with one output node and the softmax activation for the variant with two output nodes, there is mathematically/theoretically very little difference. One could argue that there are more weights in the network with two output nodes, assuming the rest of the architecture is the same, but that is about it.

Similar question with more mathematical answers on stats SE.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .