5 votes
Accepted

Formal definition of the Object Detection problem

This is just an idea Given a set of pixels, the task is to decide: Which pixel is the center of an object? What is the size of the bounding boxes with the center is the pixel in part 1? Formula, ...
user avatar
  • 817
3 votes
Accepted

How does the Kullback-Leibler divergence give "knowledge gained"?

You can know it better, if you know the concept of entropy: Information entropy is the average rate at which information is produced by a stochastic source of data. The information content (also ...
user avatar
  • 1,663
2 votes
Accepted

How can a machine learning problem be reduced as a communication problem?

Information-theoretic view of Bayesian learning I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible? Yes, this is ...
user avatar
  • 33.8k
1 vote

Why would the Dice coefficient be more suitable than mutual information when you don't want 0-0 matches to be significant?

Their reasoning is that mutual information is symmetric, giving equal value to 1s and 0s, as it is derived from information theory, where they are just two symbols used to encode a message, with ...
user avatar
  • 5,062
1 vote
Accepted

Applications of Information Theory in Machine Learning

Apart from the entropy and the cross-entropy, which are widely used in deep learning and you seem to be aware of, there is also the Kullback-Leibler divergence (also known as relative entropy), which ...
user avatar
  • 33.8k
1 vote

Compressing text using AI by sending only prediction rank of next word

If you have a fixed predictor, then yes. If the predictor is not fixed but deterministic, the feasibility depends on the effort needed to update the predictor, and ensuring that messages include a ...
user avatar
1 vote

Is it possible to have the latent vector of an auto-encoder with size 1?

According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But ...
user avatar
1 vote

Can feature engineering change the selection of the model according to the minimum description length?

I think the wrong assumption here is that you've forgotten the cost of encoding the new features! MDL should be considered relative to the original or raw dataset. The idea is that you want to find ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible