4

Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will always give you a better-performing model. However, since you can retrain many times to get a distribution of models, you can use a statistical test like the T-Test to say "algorithm A usually produces a better model ...


3

Also, in general, in the conditional expectation, which distribution do we compute the expectation with respect to? From what I have seen, in $\mathbb{E}[X|Y]$, we always calculate the expected value over distribution $X$. No, for $\mathbb{E}[X|Y]$ we take expectation of $X$ with respect to the conditional distribution $X|Y$, i.e. $$\mathbb{E}[X|Y] = \...


3

Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the data generating distribution. Generative models classify the samples by trying to learn the distribution of the data. In most cases, either the model is incapable of doing so, or the training samples aren't enough to properly ...


3

I've found the answer, the L2 is Standard Deviation, and the L1 is Mean Deviation. Standard deviation describes the variation better and the values are always different on different sets of X while the Mean Deviation gives the same values sometimes. *Footnote: Why square the differences? If we just add up the differences from the mean ... the negatives ...


2

Bayes' theorem relates conditional probabilities: $$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$


2

Some research areas that come to mind which can be useful when faced with a limited amount of data: Regularization: Comprises different methods to prevent the network from overfitting, to make it perform better on the validation data but not necessarily on the training data. In general, the less training data you have, the stronger you want to regularize. ...


1

My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the "best fit" to some form of problem. Is that correct? I know it isn't. But is it vaguely correct? No. It's not correct, in my opinion, not even vaguely and in many ways. AI is not (...


1

Let $A$ and $B$ be two events. In general, the probability that either $A$ or $B$ occurs is defined as $$ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $$ If $A$ and $B$ are disjoint, i.e. they cannot happen at the same time, then $P(A \text{ and } B) = 0$, so the above formula becomes $$ P(A \text{ or } B) = P(A) + P(B) $$ If the probability of ...


Only top voted, non community-wiki answers of a minimum length are eligible