# Tag Info

10

There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - that it has been sampled fairly from the population that the learning algorithm has been set up to approximate. The term i.i.d. stands for "independent and ...

10

There are several examples. For example, one instance of using Statistical AI from my workplace is: Analyzing the behavior of the customer and their food-ordering trends, and then trying to upsell by recommending them the dishes which they might like to order/eat. This can be done through the apriori and FP-growth algorithms. We then, automated the ...

9

We typically think of machine learning models as modeling two different parts of the training data--the underlying generalizable truth (the signal), and the randomness specific to that dataset (the noise). Fitting both of those parts increases training set accuracy, but fitting the signal also increases test set accuracy (and real-world performance) while ...

9

Noise in the data, to a reasonable amount, may help the network to generalize better. Sometimes, it has the opposite effect. It partly depends on the kind of noise ("true" vs. artificial). The AI FAQ on ANN gives a good overview. Excerpt: Noise in the actual data is never a good thing, since it limits the accuracy of generalization that can be achieved ...

8

Suppose that we have some optimization criterion $J(x)$, which we aim to optimize (maybe maximize, maybe minimize), which we can compute for a single example $x$. In an "ideal world", where we have no restrictions on computation time and memory, we would generally want to run training algorithms on the complete "ground truth" population. For example, if we'...

8

Yes and no! There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a Weibull distribution, or another extreme value model, from data. The bigger issue is with known-unknowns vs. unknown-unknowns. If you know that rare events are possible (as with, say, earthquake ...

6

"Assuming that we have sufficient data..." — that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still something of a black box: you train it, and then you get a mapping from input to output. But you don't really know how that mapping is achieved. It's not only about ...

5

The University of Maryland published some slides (PDF) from an introductory presentation on this topic. The fourth page explains why SRL is interesting. "Traditional statistical machine learning approaches" process one sort of thing in which there is some uncertaintly. Image identification is a good example of that. "Traditional ILP/relational learning ...

4

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are ...

4

There are many online services that use statistical neural networks for recommendations. For example, we have a well known service here in Russia that could give it's users recommendations for movies and shows to watch and books to read. Its recommendation core is based on many things known about a user: what movies/books he or she loves and what not, ...

4

According to Wikipedia: A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. Answer to your question: To build any neural network ...

3

What is a statistical model? According to Anthony C. Davison (in the book Statistical Models), a statistical model is a probability distribution constructed to enable inferences to be drawn or decisions made from data. The probability distribution represents the variability of the data. Are neural networks statistical models? Do neural networks construct or ...

3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...

3

From this document, as you found here, $X$ is an observed variable and $Z$ is a hidden variable; $p(X)$ is the density function of $X$. The posterior distribution of the hidden variables can then be written as follows using the Bayes’ Theorem: $$p(Z|X) = \frac{p(X|Z)p(Z)}{p(X)} = \frac{p(X|Z)p(Z)}{\int_Zp(X,Z)}$$ Now base on what you post, if we denote ...

3

This formulation/interpretation can indeed be confusing (or even misleading), as the output of a neural network is usually deterministic (i.e. given the same input $x$, the output is always the same, so there is no sampling), and there isn't really a probability distribution that models any uncertainty associated with the parameters of the network or the ...

3

Not necessarily. The neural network (or whatever else you use) is a model of what you are trying to do, and usually models are not able to perfectly model reality, as it is too complex. A noise term is generally used to represent that, ie the imperfection of the model's relationship with the actual world.

3

Many people without a formal/solid background in statistics (e.g. without knowing exactly what the central limit theorem (CLT) states) are doing research on machine learning, which is a very big and fundamental subfield of AI that has a big overlap with statistics, or using machine learning to solve problems. So, in my view, you don't need to learn ...

3

First of all, I don't know of any textbook that clarifies these terms, but, although I am not a statistician, in addition to the other answer, one possible way to look at it is as follows. You use probability theory to model your problem. For example, if it's a classification problem, you could define the conditional probability distribution $p(y \mid x)$, ...

2

Not strictly examples of AI, but related to the greater AI project: But us in the psychology / cognitive science side of things sure love our bayesian modelling! In fact there are people who believe that a theory grounded in such analysis would ultimately bring us to a unified theory of the brain and cognition! Unfortunately to my knowledge, these theories ...

2

You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools. We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning. You are describing the essence of ...

2

2

The problem you're trying to address can, in some sense, be viewed as a Feature Selection problem. If you look for literature using only those words, you're not going to find what you're looking for though. In general, "Feature Selection" simply refers to the problem where you already have a large amount of features, and you're simply deciding to select ...

2

In the YouTube depiction of CS294-112 fall 2017 lecture 3 Reinforcement Learning, Levine, the transition of the finite horizon expected reward to a form where each transition is decoupled from the entire Markov chain of state-action marginals is explained between $t_{video}$ = 44:04 and $t_{video}$ = 45:22. At t=44:29, the probabilistic expectation where no ...

2

Are you thinking something like Information Gain? Information Gain basically uses the concept of information entropy to determine if splitting a variable is useful.

2

The use of KL provides a more intuitive way of what the ELBO is attempting to maximize. Basically, we want to find a posterior approximation such that $p(z\mid x) \approx q(z)\in\mathcal{Q}$ $$KL(q(z)\parallel p(z\mid x)) \rightarrow \min_{q(z)\in\mathcal{Q}}$$ As a result of this, while finding this optimal posterior approximation, we maximize the ...

2

Let's say we have $a$ - constant and $\epsilon \sim \mathcal{N}(0,\sigma)$, then: $$\mathbb{E}\left[(a+\epsilon)^2\right] = \mathbb{E}\left[a^2\right] + 2 \mathbb{E}\left[a\right]\mathbb{E}\left[\epsilon\right] + \mathbb{E}\left[\epsilon^2\right]$$ Expectations of constants are just the constants: $\mathbb{E}[a] = a$ and $\mathbb{E}[a^2] = a^2$ The mean of ...

2

I work in NLP, and use very little statistics. Actually, almost nothing I do can be classed as 'serious' statistics. So yes, AI is a wide area, and in my company there is a group that does machine learning, so they probably use a lot more of it than I do. Previously I worked in conversational AI. Again, very little to no statistics at all. I would contest ...

1

Yes, you can use sklearn's confusion_matrix. To explicitly extract the false positives and negatives, you can do from sklearn.metrics import confusion_matrix y_true = [0, 1, 0, 1] y_pred = [1, 1, 1, 0] tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()

1

A linear model has a linear decision boundary. So in the case of the question, you need to draw two multimodal distributions whose domains do not overlap at all and then you can just say the linear model would should all to the right of some number would be class 1 and all to the left would be class 2

1

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean. In other words, the past of the process contains no information on the development of the process in the distant future. And, this might be the reason that you are getting poor forecasts. ...

Only top voted, non community-wiki answers of a minimum length are eligible