9 votes
Accepted

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? Generally, yes. Most decision tree learners, like the common ID3 and ...
3 votes

How to calculate the entropy in the ID3 decision tree algorithm?

Suppose you have data: ...
  • 1,345
3 votes
Accepted

Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?

Yes, if you can assume that your data is separable on the features given then ID3 will find a decision tree for it (Note: this will not necessarily be an optimal tree, or even a good tree). To ...
2 votes
Accepted

What is the intuition behind the entropy formula used in the ID3 algorithm?

Please, take a look at Understanding Shannon's Entropy metric for Information. The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign ...
  • 176

Only top scored, non community-wiki answers of a minimum length are eligible