Skip to main content
11 votes

Why exactly do neural networks require i.i.d. data?

There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - ...
Neil Slater's user avatar
  • 32.7k
10 votes
Accepted

What are some examples of Statistical AI applications?

There are several examples. For example, one instance of using Statistical AI from my workplace is: Analyzing the behavior of the customer and their food-ordering trends, and then trying to upsell by ...
Dawny33's user avatar
  • 1,371
9 votes

How does noise affect generalization?

We typically think of machine learning models as modeling two different parts of the training data--the underlying generalizable truth (the signal), and the randomness specific to that dataset (the ...
Matthew Gray's user avatar
  • 4,272
9 votes
Accepted

How does noise affect generalization?

Noise in the data, to a reasonable amount, may help the network to generalize better. Sometimes, it has the opposite effect. It partly depends on the kind of noise ("true" vs. artificial). The AI FAQ ...
Franck Dernoncourt's user avatar
8 votes

Is Nassim Taleb right about AI not being able to accurately predict certain types of distributions?

Yes and no! There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a Weibull distribution, or another extreme ...
John Doucette's user avatar
8 votes

Why exactly do neural networks require i.i.d. data?

Suppose that we have some optimization criterion $J(x)$, which we aim to optimize (maybe maximize, maybe minimize), which we can compute for a single example $x$. In an "ideal world", where we have ...
Dennis Soemers's user avatar
  • 10.3k
7 votes
Accepted

Is traditional machine learning obsolete given that neural networks typically outperform them?

"Assuming that we have sufficient data..." — that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still ...
Oliver Mason's user avatar
  • 5,397
5 votes

Are neural networks statistical models?

According to Wikipedia: A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger ...
Ta_Req's user avatar
  • 101
5 votes
Accepted

Are neural networks statistical models?

What is a statistical model? According to Anthony C. Davison (in the book Statistical Models), a statistical model is a probability distribution constructed to enable inferences to be drawn or ...
nbro's user avatar
  • 40.9k
5 votes
Accepted

Does the correlation between inputs affect the model performance?

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies ...
nbro's user avatar
  • 40.9k
5 votes
Accepted

What is Statistical relational learning?

The University of Maryland published some slides (PDF) from an introductory presentation on this topic. The fourth page explains why SRL is interesting. "Traditional statistical machine learning ...
Ben N's user avatar
  • 2,609
5 votes
Accepted

How can supervised learning be viewed as a conditional probability of the labels given the inputs?

This formulation/interpretation can indeed be confusing (or even misleading), as the output of a neural network is usually deterministic (i.e. given the same input $x$, the output is always the same, ...
nbro's user avatar
  • 40.9k
4 votes
Accepted

What are the differences in scope between statistical AI and classical AI?

Statistical AI, arising from machine learning, tends to be more concerned with inductive thought: given a set of patterns, induce the trend. Classical AI is the branch of artificial intelligence ...
Nimra Malik's user avatar
4 votes

What are some examples of Statistical AI applications?

There are many online services that use statistical neural networks for recommendations. For example, we have a well known service here in Russia that could give it's users recommendations for movies ...
Andrei Chevozerov's user avatar
4 votes

How much statistics is involved in AI?

Many people without a formal/solid background in statistics (e.g. without knowing exactly what the central limit theorem (CLT) states) are doing research on machine learning, which is a very big and ...
nbro's user avatar
  • 40.9k
3 votes
Accepted

What makes a machine learning algorithm a low variance one or a high variance one?

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate ...
user's user avatar
  • 146
3 votes

Why do we need Upsampling and Downsampling in Progressive Growing of Gans

Use of Transposed Convolution can lead to checkerboard artifacts. So we prefer to up-sample and then apply convolution. You can check this article for more information https://distill.pub/2016/deconv-...
Giannis Kousis's user avatar
3 votes

What's going on in the equation of the variational lower bound?

From this document, as you found here, $X$ is an observed variable and $Z$ is a hidden variable; $p(X)$ is the density function of $X$. The posterior distribution of the hidden variables can then be ...
OmG's user avatar
  • 1,826
3 votes

Is the target assumed to be a noisy version of the output of the model in machine learning?

Not necessarily. The neural network (or whatever else you use) is a model of what you are trying to do, and usually models are not able to perfectly model reality, as it is too complex. A noise term ...
Oliver Mason's user avatar
  • 5,397
3 votes

Is there any model that is probabilistic but not statistical?

First of all, I don't know of any textbook that clarifies these terms, but, although I am not a statistician, in addition to the other answer, one possible way to look at it is as follows. You use ...
nbro's user avatar
  • 40.9k
3 votes

Is logic AI a complement to learning AI?

What you refer to as logic AI is a subset of what is called symbolic AI, as you manipulate symbols, according to certain rules (which could be rules of logic). These rules are either authored by a ...
Oliver Mason's user avatar
  • 5,397
2 votes

What are some examples of Statistical AI applications?

Not strictly examples of AI, but related to the greater AI project: But us in the psychology / cognitive science side of things sure love our bayesian modelling! In fact there are people who believe ...
k.c. sayz 'k.c sayz''s user avatar
2 votes

How to figure out which words have the same meaning in two different languages?

You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools. We need to write a program that would recognize that a word ...
Mathias Müller's user avatar
2 votes
Accepted

Finding the right questions to increase accuracy in classification

The problem you're trying to address can, in some sense, be viewed as a Feature Selection problem. If you look for literature using only those words, you're not going to find what you're looking for ...
Dennis Soemers's user avatar
  • 10.3k
2 votes

Reinforcement learning objective as conditional expectations

In the YouTube depiction of CS294-112 fall 2017 lecture 3 Reinforcement Learning, Levine, the transition of the finite horizon expected reward to a form where each transition is decoupled from the ...
Douglas Daseeco's user avatar
2 votes

Are neural networks statistical models?

A dataset can be thought of as a set of ordered pairs $\subset R^f \times R^l$, where $f$ is the feature dimensions and $l$ the label dimensions. Ordered pairs give rise to a statistical model (i.e. ...
Tom Huntington's user avatar
2 votes
Accepted

Is explainable AI more feasible through symbolic AI or soft computing?

XAI is relevant to "black box" AI (machine learning methods where the decision making rationale is not apparent, only the structure of the system that led to that decision.) Symbolic AI, GOFAI, and ...
DukeZhou's user avatar
  • 6,237
2 votes
Accepted

Is there any measure of separability of classes?

Are you thinking something like Information Gain? Information Gain basically uses the concept of information entropy to determine if splitting a variable is useful.
user1209675's user avatar
2 votes

What's going on in the equation of the variational lower bound?

The use of KL provides a more intuitive way of what the ELBO is attempting to maximize. Basically, we want to find a posterior approximation such that $p(z\mid x) \approx q(z)\in\mathcal{Q}$ $$KL(q(...
c.uent's user avatar
  • 21
2 votes
Accepted

Why is the equation $\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{Var} (\epsilon)$ true?

Let's say we have $a$ - constant and $\epsilon \sim \mathcal{N}(0,\sigma)$, then: $$\mathbb{E}\left[(a+\epsilon)^2\right] = \mathbb{E}\left[a^2\right] + 2 \mathbb{E}\left[a\right]\mathbb{E}\left[\...
Kostya's user avatar
  • 2,534

Only top scored, non community-wiki answers of a minimum length are eligible