Questions tagged [entropy]
For questions about the concept of (information) entropy in the context of artificial intelligence.
9
questions
1
vote
1
answer
69
views
Why is Soft Q Learning not an Actor Critic method?
I've been reading these two papers from Haarnoja et. al.:
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement
Learning with a Stochastic Actor
Reinforcement Learning with Deep Energy-...
0
votes
0
answers
65
views
What is the relationship between entropy in thermodynamics and entropy in information theory?
BACKGROUND: In thermodynamics, entropy $S$ is a measure of disorder and is given by
$${\displaystyle S=k_B\log(W)},$$
where $k_B$ is Boltzman's constant and $W$ is the number of microstates.
In ...
- 882
0
votes
0
answers
9
views
Is there any point in continuing the training of an agent when entropy (of output probabilities) is low?
I'm working with a PPO agent with a small, discrete action space (3 possible actions, 1 of which is always masked depending on the state).
Premise 1:
My understanding is that the "entropy" ...
- 312
1
vote
0
answers
35
views
Can entropy bonus be used with state-independent log std for stochastic policies?
In this blog article by openai, they say the std of the exploration distribution must be state-dependent, i.e. an output of the policy network, so it works with the entropy bonus, which is an integral ...
- 131
1
vote
0
answers
22
views
Why exclude the first entropy bonus term in the soft Q-function in SAC?
Based on OpenAI Spinning Up description of Soft Actor Critic (SAC) the soft Q-function is defined as
and as they say
Q value is changed to include the entropy bonuses from every timestep except the ...
- 111
0
votes
1
answer
58
views
What does the product of probabilities raised to own powers used for entropy calculation quantify?
Suppose $X$ is a random variable taking $k$ values.
$$Val(X) = \{x_1, x_2, x_3, \cdots, x_k\} $$
Then what is the following expression of $N(X)$ called in literature if exists? What does it signify?
$$...
- 3,491
0
votes
1
answer
60
views
How does high entropy targets relate to less variance of the gradient between training cases?
I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this:
When the soft targets have high entropy, they provide much ...
1
vote
0
answers
96
views
How does NN follows law of energy conservation?
Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
- 67
3
votes
2
answers
5k
views
How to calculate the entropy in the ID3 decision tree algorithm?
Here is the definition of the entropy
$$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$
Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$,...
- 131