5
$\begingroup$

I am working on a MLP neural networks, using supervised learning (2 classes and multi-class classification problems). For the hidden layers, I am using $\tanh$ (which produces an output in the range $[-1, 1]$) and for the output layer a softmax (which gives the probability distribution between $0$ and $1$). As I am working with supervised learning, should be my targets output between 0 and 1, or $-1$ and $1$ (because of the $\tanh$ function), or it does not matter?

The loss function is quadratic (MSE).

$\endgroup$
  • $\begingroup$ Quadratic mse for softmax is not recommended. Try cross entropy loss. $\endgroup$ – DuttaA Mar 8 at 13:59
1
$\begingroup$

For this particular classification problem, I would recommend you using a softmax function whose output range is [0,1].
The sum of all outputs should be 1, so an advantage of using a softmax function is that you get a percentage of how confident the network is in this classification.

Side note: As DuttaA has commented, cross entropy loss is a better loss function than the quadratic mean squared error.

$\endgroup$
0
$\begingroup$

your targets should be in the same range as your output functions other wise your loss function wont be accurate, with supervised learning your trying to reduce the loss of your output against your targets so in this case your targets should be the true/optimal probability distribution for that set of input data. Im from the midwest so obligatory "cant compare apples to oranges" here ;)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.