Questions tagged [entropy]

For questions about the concept of (information) entropy in the context of artificial intelligence.

Filter by
Sorted by
Tagged with
1
vote
0answers
52 views

How does NN follows law of energy conservation?

Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
3
votes
1answer
4k views

How to calculate the entropy in the ID3 decision tree algorithm?

Here is the definition of the entropy $$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$ Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$,...