Skip to main content
The 2024 Developer Survey results are live! See the results
5 votes
Accepted

Why has statistics-based AI become more popular than other forms of AI?

The availability of large data sets. In symbolic/rule-based AI, the 'knowledge' has to be hand-coded, usually by experts. This is expensive and limited to small-scale problems only. In statistical AI/...
Oliver Mason's user avatar
  • 5,397
4 votes

How much statistics is involved in AI?

Many people without a formal/solid background in statistics (e.g. without knowing exactly what the central limit theorem (CLT) states) are doing research on machine learning, which is a very big and ...
nbro's user avatar
  • 41k
4 votes
Accepted

Why is Standard Deviation based on L2 Variance and not L1 Variance

I've found the answer, the L2 is Standard Deviation, and the L1 is Mean Deviation. Standard deviation describes the variation better and the values are always different on different sets of X while ...
Dan D.'s user avatar
  • 1,293
4 votes
Accepted

What does it mean when a model "statistically outperforms" another?

Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will always give you a better-performing model. However, since you ...
alltom's user avatar
  • 176
3 votes

Why does model overfitting lead to poor generalization?

The issue is the generalisation. If your model fits perfectly to your training data, it then depends whether your training data is a good reflection of the actual data you will encounter 'in the wild'....
Oliver Mason's user avatar
  • 5,397
3 votes

What is the difference between model and data distributions?

Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the data generating distribution. Generative models classify the samples ...
Djib2011's user avatar
  • 3,193
3 votes
Accepted

How does $\mathbb{E}$ suddenly change to $\mathbb{E}_{\pi'}$ in this equation?

Also, in general, in the conditional expectation, which distribution do we compute the expectation with respect to? From what I have seen, in $\mathbb{E}[X|Y]$, we always calculate the expected value ...
David's user avatar
  • 4,940
2 votes

What is Bayes' theorem?

Bayes' theorem relates conditional probabilities: $$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$
dyedgreen's user avatar
2 votes

How much statistics is involved in AI?

I work in NLP, and use very little statistics. Actually, almost nothing I do can be classed as 'serious' statistics. So yes, AI is a wide area, and in my company there is a group that does machine ...
Oliver Mason's user avatar
  • 5,397
2 votes
Accepted

Research paths/areas for improving the performance of CNNs when faced with limited data

Some research areas that come to mind which can be useful when faced with a limited amount of data: Regularization: Comprises different methods to prevent the network from overfitting, to make it ...
HelloGoodbye's user avatar
2 votes
Accepted

Which generalization of standard deviation to use for multidimensional input normalization

This idea is sometimes applied in computer vision, under the name of Whitening Transform, or ZCA sphering transform. The name whitening comes from signal processing, since removing correlation from a ...
Edoardo Guerriero's user avatar
2 votes

Coherence is classifying time series data

The following paper from Amazon Alexa Research refers to this topic as keyword spotting (KWS), and more specifically, to wake up words as wake word (WW) spotting. Jose, C., Mishchenko, Y., Senechal, ...
Brian O'Donnell's user avatar
2 votes

Why does model overfitting lead to poor generalization?

Note that if you know the function your data was generated from, you do not even need machine learning. For example, if you generated the data with $y = f(x) = x^2$, you already have the function to ...
nbro's user avatar
  • 41k
2 votes
Accepted

What does it mean to "learn a distribution", and what does it contain?

Learning the distribution: When we talk about learning a distribution, we are essentially trying to capture the underlying statistical properties of the data. In other words, we try to capture the ...
Robin van Hoorn's user avatar
1 vote

Non constant Feature Importance

I think you can derive new features that should be more stable over time using PCA. A more simple approaches is to calculate the technical indicators of these numerical features such as RSI, moving ...
Gene's user avatar
  • 11
1 vote

What is the difference between q and p in Statistical Notation(used in VAE)?

These letters have no special meaning. It's just a convention. So you can denote these distributions with other letters, like $a$, $b$, $c$, etc. $p$ is probably common because it probably stands for ...
nbro's user avatar
  • 41k
1 vote

Why is laplace distribution less sensitive to outliers than normal distribution?

Your reference doesn't claim Laplace distribution is less sensitive to outliers than normal distribution, the intended claim is the absolute error loss is more robust (less sensitive to outliers) than ...
cinch's user avatar
  • 2,551
1 vote
Accepted

What is $D_i$ in "Common Principal Components Analysis"?

$D_{i}$ merely refers to the diagonal matrix that captures the variance for each principal component specific to class $i$. We can pool data because the $C$ matrix is the same for every class, meaning ...
abkg's user avatar
  • 91
1 vote

What is wrong in reasoning here in classification for defect detection?

Alpha represents the significance level, or the probability that you will make a Type I error by rejecting the null hypothesis when it is actually true - in other words, the probability that you're ...
Nuclear Hoagie's user avatar
1 vote

Distribution of a log-normal random variable with fixed dimension

Due to the fact that your multivariate-normal distribution is independent (the covariance matrix is diagonal), it will be more intuitive to treat each dimension as being its 'own' random variable; i.e....
David's user avatar
  • 4,940
1 vote
Accepted

Markov's Decision Process - calculate value in each iteration

The value iteration algorithm defines the following update rule (reference is slide 11 in this MIT course): $$V_{i+1}(s) = \max_{a}\{R(s,a) + \gamma E_{s'\sim T(.|s,a)} V_i(s')\}$$ for all states $s$, ...
Raphael Lopez Kaufman's user avatar
1 vote

What can be an example other than batch normalization that uses statistics of batches?

Here's some examples: Group Normalization Layer Normalization Switchable Normalization Attentive Normalization Spectrl Normalization Notice how in general different normalization techniques are ...
Edoardo Guerriero's user avatar
1 vote

What is the definition of "confidence interval" around a (complicated) function?

Off the top of my head, I don't know the very specific definition of confidence interval (or whether it's only defined for the parameters of a model), as I am not a statistician. In any case, ...
nbro's user avatar
  • 41k
1 vote
Accepted

What would be the reason behind using plots (such as box-plots or histograms) for ML development?

At a basic level, these kinds of low-dimensional plots where you look at one or two variables at a time can help to give you a sense of what types of relationships you might expect to see, such as ...
Nuclear Hoagie's user avatar
1 vote

Can AI be understood as a generalized statistics tool?

My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the "best fit" to some form of ...
nbro's user avatar
  • 41k
1 vote
Accepted

Why is probability that at least one hypothesis out of $k$ being consistent with $m$ training examples $k(1- \epsilon)^m$?

Let $A$ and $B$ be two events. In general, the probability that either $A$ or $B$ occurs is defined as $$ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $$ If $A$ and $B$ are disjoint, i.e. ...
nbro's user avatar
  • 41k

Only top scored, non community-wiki answers of a minimum length are eligible