All Questions
Tagged with cross-entropy deep-learning
8 questions
0
votes
0
answers
13
views
Incredibly High CrossEntropyLoss in Sequence-to-Sequence Generation
I'm trying to do SMILES chemical representation prediction from a large dataset (Around 5M Samples) to teach it do predict another downstream task. The model's part responsible for generating the data ...
- 101
0
votes
1
answer
456
views
What is the correct loss function for binary classification: Cross entropy or Binary cross entropy?
Let's say I have a binary classification problem and I want to solve it by means of FC neural net. So which approach will be correct: 1) define the last layer of NN like this ...
- 23
1
vote
1
answer
78
views
What loss function will be correlated with classification metrics?
Recently I developed a custom training algorithm for deep learning models, based on evolutionary algorithms. Details are not important, except that it also uses decreasing regular cross entropy loss ...
- 1,280
3
votes
0
answers
3k
views
Focal Loss vs Weighted Cross Entropy Loss
Weighted Focal Loss is defined like so
$FL(p_t) = -\alpha_t log(p_t) (1-p_t)^\gamma $
Whereas weighted Cross Entropy Loss is defined like so
$CE(p_t) = -\alpha_t log(p_t)$
Some blog posts try to ...
- 789
1
vote
1
answer
733
views
How to compute the gradient of the cross-entropy loss function with respect to the parameters with softmax activation function?
I've seen plenty of examples of people doing Sigmoid + MSE backpropagation implementations, yet I do not seem to understand how to implement backpropagation as stated in the title in the case of multi-...
- 115
3
votes
2
answers
332
views
Where is the mistake in my derivation of the GAN loss function?
I was pondering on the loss function of GAN, and the following thing turned out
\begin{aligned}
L(D, G)
& = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{x \sim p_g(x)} [\log(1 - D(x)] \...
- 324
1
vote
1
answer
154
views
Why does PyTorch use a different formula for the cross-entropy?
In my understanding, the formula to calculate the cross-entropy is
$$
H(p,q) = - \sum p_i \log(q_i)
$$
But in PyTorch nn.CrossEntropyLoss is calculated using this ...
- 2,819
4
votes
1
answer
127
views
How to formalize learning in terms of information theory?
Consider the following game on a MNIST dataset:
There are 60000 images.
You can pick any 1000 images and train your Neural Network without access to the rest of images.
Your final result is ...
- 141