People who code: we want your input. Take the Survey
4

Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will always give you a better-performing model. However, since you can retrain many times to get a distribution of models, you can use a statistical test like the T-Test to say "algorithm A usually produces a better model ...


3

Also, in general, in the conditional expectation, which distribution do we compute the expectation with respect to? From what I have seen, in $\mathbb{E}[X|Y]$, we always calculate the expected value over distribution $X$. No, for $\mathbb{E}[X|Y]$ we take expectation of $X$ with respect to the conditional distribution $X|Y$, i.e. $$\mathbb{E}[X|Y] = \...


3

Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the data generating distribution. Generative models classify the samples by trying to learn the distribution of the data. In most cases, either the model is incapable of doing so, or the training samples aren't enough to properly ...


3

I've found the answer, the L2 is Standard Deviation, and the L1 is Mean Deviation. Standard deviation describes the variation better and the values are always different on different sets of X while the Mean Deviation gives the same values sometimes. *Footnote: Why square the differences? If we just add up the differences from the mean ... the negatives ...


2

Bayes' theorem relates conditional probabilities: $$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$


2

Some research areas that come to mind which can be useful when faced with a limited amount of data: Regularization: Comprises different methods to prevent the network from overfitting, to make it perform better on the validation data but not necessarily on the training data. In general, the less training data you have, the stronger you want to regularize. ...


1

At a basic level, these kinds of low-dimensional plots where you look at one or two variables at a time can help to give you a sense of what types of relationships you might expect to see, such as linear, non-linear, or periodic relationships, which can steer you toward an appropriate family of models. You wouldn't want to use a linear model to predict data ...


1

I no longer really use validation that much, but rather only training and testing. Why? Because I mostly follow Ron Kohavi's (Stanford Univ) approach to CV. I have done a lot of validation but it seemed to be overkill, essentially causing me to ask why I have this very small-sampled parameter watch on the side from which I am supposed to learn from. You ...


1

My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the "best fit" to some form of problem. Is that correct? I know it isn't. But is it vaguely correct? No. It's not correct, in my opinion, not even vaguely and in many ways. AI is not (...


1

Let $A$ and $B$ be two events. In general, the probability that either $A$ or $B$ occurs is defined as $$ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $$ If $A$ and $B$ are disjoint, i.e. they cannot happen at the same time, then $P(A \text{ and } B) = 0$, so the above formula becomes $$ P(A \text{ or } B) = P(A) + P(B) $$ If the probability of ...


Only top voted, non community-wiki answers of a minimum length are eligible