# Tag Info

Accepted

### Formal definition of the Object Detection problem

This is just an idea Given a set of pixels, the task is to decide: Which pixel is the center of an object? What is the size of the bounding boxes with the center is the pixel in part 1? Formula, ...
• 912
Accepted

### How can a machine learning problem be reduced as a communication problem?

Information-theoretic view of Bayesian learning I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible? Yes, this is ...
• 41k
Accepted

### How does the Kullback-Leibler divergence give "knowledge gained"?

You can know it better, if you know the concept of entropy: Information entropy is the average rate at which information is produced by a stochastic source of data. The information content (also ...
• 1,826
Accepted

### Applications of Information Theory in Machine Learning

Apart from the entropy and the cross-entropy, which are widely used in deep learning and you seem to be aware of, there is also the Kullback-Leibler divergence (also known as relative entropy), which ...
• 41k
Accepted

### Classifier performance if data are deterministic

This is an important question that goes back to the fundamentals of machine learning theory. The primary objective of using machine learning to solve a problem is as the following: For a given set of ...
• 441
1 vote
Accepted

### How Mutual Information is related to uncertainty

Yes, you can think of mutual information as entropy carried by that information $Y$ Say you have a dice $X$ and you want to predict the outcome, then your belief is a discrete uniform one with $p=1/6$ ...
• 2,333
1 vote
Accepted

### Calculating mutual information between layer outputs and targets in a neural network

NPEET uses a K Nearest Neighbors estimation for mutual information and they note in section 4.3 of their provided pdf doc that there is a chance for negative values in the estimated mutual information....
• 126
1 vote

### Why would the Dice coefficient be more suitable than mutual information when you don't want 0-0 matches to be significant?

Their reasoning is that mutual information is symmetric, giving equal value to 1s and 0s, as it is derived from information theory, where they are just two symbols used to encode a message, with ...
• 5,397
1 vote

### Compressing text using AI by sending only prediction rank of next word

If you have a fixed predictor, then yes. If the predictor is not fixed but deterministic, the feasibility depends on the effort needed to update the predictor, and ensuring that messages include a ...
1 vote

### Is it possible to have the latent vector of an auto-encoder with size 1?

According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But ...
1 vote

### Can feature engineering change the selection of the model according to the minimum description length?

I think the wrong assumption here is that you've forgotten the cost of encoding the new features! MDL should be considered relative to the original or raw dataset. The idea is that you want to find ...
• 9,297

Only top scored, non community-wiki answers of a minimum length are eligible