# Questions tagged [cross-entropy]

For questions related to the concept of cross-entropy in the context of artificial intelligence. For example, when the cross-entropy is used as a loss function to train a neural network.

5 questions
Filter by
Sorted by
Tagged with
12k views

### How do I handle negative rewards in policy gradients with the cross-entropy loss function?

I am using policy gradients in my reinforcement learning algorithm, and occasionally my environment provides a severe penalty (i.e. negative reward) when a wrong move is made. I'm using a neural ...
• 209
1k views

### Which loss function should I use in REINFORCE, and what are the labels?

I understand that this is the update for the parameters of a policy in REINFORCE: $$\Delta \theta_{t}=\alpha \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) v_{t},$$ where $v_t$ is ...
• 590
5k views

I am studying logistic regression for binary classification. The loss function used is cross-entropy. For a given input $x$, if our model outputs $\hat{y}$ instead of $y$, the loss is given by $$\text{... • 3,830 2 votes 1 answer 2k views ### How does the implementation of the VAE's objective function equate to ELBO? For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO.$$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z)) The above is the definition ...
• 797
The question looks foolish, but I think cross-entropy is somewhat weird as a cost function. As a cost function for linear regression, the mean square error $\sum_{i=1}^{n} (y_i - (ax_i+b)) ^2$ seems ...