Timeline for How to obtain a formula for loss, when given an iterative update rule in gradient descent?
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 21, 2019 at 3:52 | comment | added | Hanzy | The only part of the loss function I can’t discern is why the author subtracts (0.0001*entropies[i]) at each step of calculating the loss. | |
| Apr 21, 2019 at 1:54 | comment | added | Hanzy | @nbro the entropy function you describe is the information theory entropy function found here. The entropy function you refer to with two distributions is the Cross-entropy function, which essentially compares how similar two distributions are. The entropy function used here is the average rate at which information is produced by a stochastic source of data. | |
| Mar 14, 2019 at 22:59 | comment | added | Gulzar | Sorry about that, i had to put that project aside for too many weeks and. I hope i will get back to it asap | |
| Mar 14, 2019 at 21:18 | comment | added | nbro | @Gulzar If you have figured out more details behind my explanations and the linked source code, feel free to add them as an answer to your own question! | |
| Mar 14, 2019 at 21:16 | vote | accept | Gulzar | ||
| Feb 14, 2019 at 16:51 | comment | added | Gulzar |
Thanks for all the details! However, my question was mostly theoretical. In most papers i've seen, the update rules are formulated like the pseudocode equations I linked in the question. I wanted to know how IN THE GENERAL CASE I would convert such equations to a loss function. Pseudocode would be enough for this purpose. I'm sorry, but I still can't figure out the equation which loss=... came from.
|
|
| Feb 12, 2019 at 18:09 | history | answered | nbro | CC BY-SA 4.0 |