For a project, I've trained multiple networks for multiclass classification all ending with a ReLU activation at the output.

Now the output logits are not probabilities.

Is it valid to get the probability of each class by applying a softmax function at the end, after training?

  • $\begingroup$ Hello. Welcome to Artificial Intelligence Stack Exchange! Can you please explain 1. why you trained more than one neural network to solve the same problem, 2. why you used the ReLU at the last layers of these neural nets, 3. how exactly you trained the neural nets? Did you compare the (vector) labels with the output produced by the last ReLU, 3.1 you got good accuracy with this approach? $\endgroup$
    – nbro
    Commented Nov 22, 2021 at 12:08
  • $\begingroup$ Hi, thank you. 1. It's not the exact same problem, it's for slightly differing datasets and slight differing architectures. 2. This is a constraint set for research purposes. 3. As usual, using backprop - output decision was decided using argmax over the last relu output vector; and yes test/validation accuracy and loss are satisfying $\endgroup$
    – user452306
    Commented Nov 22, 2021 at 12:23
  • $\begingroup$ My main and only question, is if we can convert post-hoc some relu-output to some 'propabilities' in a valid way $\endgroup$
    – user452306
    Commented Nov 22, 2021 at 12:24


You must log in to answer this question.