# Questions tagged [softmax]

For questions related to the softmax function, which a function that takes as input a vector of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponentials of the input numbers. The softmax is often used as the activation function of the output layer of a neural network.

10 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
33 views

This is a question I posted here. I am asking it on this StackExchange branch as well, so that more people who could potentially answer get to see the question. In the A3C algorithm from the original ...
43 views

### How am I supposed to code equation 4.57 from the book "Machine Learning: An Algorithmic Perspective"?

Consider the equation 4.57 (p. 108) from section 4.6 of the Book Machine Learning: An Algorithmic Perspective, where the derivative of the softmax function is explained \delta_o(\kappa) = (y_\kappa -...
33 views

### Isn't it true that using max over a softmax will be much slower because there is not a smooth gradient?

Isn't it true that using max over a softmax will be much slower because there is not a smooth gradient? Max basically zeros out the gradients of all the non-maximum values. Especially at the beginning ...
36 views

### What do the authors of this paper mean by the bias term in this picture of a neural network implementation?

I am reading a paper implementing a deep deterministic policy gradient algorithm for portfolio management. My question is about a specific neural network implementation they depict in this picture (...
17 views

### Why use two different embeddings for actions in this paper?

I was reading this paper Top-𝐾 Off-Policy Correction for a REINFORCE Recommender System and I'm wondering is there a particular advantage to use different embeddings for actions, one embedding is ...
43 views

### Which solutions are there to the problem of having too large activations before the softmax (or sigmoid) layer?

I'm trying to build a neural network (NN) for classification using only N-bit integers for both the activations and weights, then I will train it with some heuristic algorithm, based only on the NN ...
39 views

### How is it possible that the softmax combined with the MSE in a molecule classification task performs than than the cross-entropy?

I'm working on a GNN project associated with molecule classification. The project is to classify if the atom in the molecule will initiate a certain reaction. For example, a molecule can be ...
33 views

### Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...