9
votes
Accepted
Are decision tree learning algorithms deterministic?
Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
Generally, yes. Most decision tree learners, like the common ID3 and ...
- 9,392
3
votes
3
votes
Accepted
Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?
Yes, if you can assume that your data is separable on the features given then ID3 will find a decision tree for it (Note: this will not necessarily be an optimal tree, or even a good tree). To ...
- 587
2
votes
Accepted
What is the intuition behind the entropy formula used in the ID3 algorithm?
Please, take a look at Understanding Shannon's Entropy metric for Information.
The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign ...
- 176
Only top scored, non community-wiki answers of a minimum length are eligible