2
$\begingroup$

What is the use of softmax function? Why was it used at the end of fully connected layer in convolution neural network?

$\endgroup$
1
$\begingroup$

The main purpose of the softmax function is to transform the (unnormalised) output of $K$ units (which is e.g. represented as a vector of $K$ elements) of a fully-connected layer to a probability distribution (a normalised output), which is often represented as a vector of $K$ elements, each of which is between $0$ and $1$ (a probability) and the sum of all these elements is $1$ (a probability distribution).

In the case of a classification task, the $i$th element of the vector produced by the softmax function corresponds to the probability of the input of the network of belonging to the $i$th class (e.g. a dog).

$\endgroup$
6
  • $\begingroup$ but can it be replaced to argmax function (taking element with higher value as 1, everythin else is 0) $\endgroup$ Jun 16 '19 at 16:52
  • $\begingroup$ @user8426627 You could do that, but you might lose the probabilistic interpretation of the results (classification). At the end, you will have to make a decision, so you will choose one (or more) of those outputs (anyway). The most obvious decision is to choose the class with the highest probability, but this might not always be the case. $\endgroup$
    – nbro
    Jun 16 '19 at 16:59
  • $\begingroup$ i just want to figure out, does softmax gains benefit when packpropagating from it compare to just use linear difference $\endgroup$ Jun 16 '19 at 17:04
  • $\begingroup$ @user8426627 I'm not sure what you mean by "linear difference". If you're asking for the advantages of the softmax function (compared to other functions), then maybe this is another question. $\endgroup$
    – nbro
    Jun 16 '19 at 17:21
  • $\begingroup$ the square los, what has a derivative expected - output. It is related to question, since it is about "What is the use of softmax function" if we have other one $\endgroup$ Jun 16 '19 at 17:25
1
$\begingroup$

In short the softmax function helps with multi-classification i.e output of more than one of two possibilities. It works well this the categorical cross entropy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.