10
$\begingroup$

So Taleb has two heuristics to generally describe data distributions. One is Mediocristan, which basically means things that are on a Gaussian distribution such as height and/or weight of people.

The other is called Extremistan, which describes a more Pareto like or fat-tailed distribution. An example is wealth distribution, 1% of people own 50% of the wealth or something close to that and so predictability from limited data sets is much harder or even impossible. This is because you can add a single sample to your data set and the consequences are so large that it breaks the model, or has an effect so large that it cancels out any of the benefits from prior accurate predictions. In fact this is how he claims to have made money in the stock market, because everyone else was using bad, Gaussian distribution models to predict the market, which actually would work for a short period of time but when things went wrong, they went really wrong which would cause you to have net losses in the market.

I found this video of Taleb being asked about AI. His claim is that A.I. doesn't work (as well) for things that fall into extremistan.

Is he right? Will some things just be inherently unpredictable even with A.I.?

Here is the video I am referring to https://youtu.be/B2-QCv-hChY?t=43m08s

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Well, in Taleb's book "The Black Swan" about the events unknown-unknowns, you realize that these events are inherently unexpected. Therefore, based on that, yes, A.I. in this area won't work. $\endgroup$
    – Manuel Ramos
    Jun 25, 2020 at 4:33
  • $\begingroup$ The video link appears to be broken; do you have a functioning link to the original video? $\endgroup$
    – nutty about natty
    Feb 10, 2021 at 11:08

2 Answers 2

Reset to default
8
$\begingroup$

Yes and no!

There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a Weibull distribution, or another extreme value model, from data.

The bigger issue is with known-unknowns vs. unknown-unknowns. If you know that rare events are possible (as with, say, earthquake prediction), you can incorporate that knowledge into the models you develop, and you'll get something that works as well or better than humans in that domain. If you don't know that rare events are possible (as with, say, a stock market crash produced by correlated housing defaults), then your model will reflect that as well.

I tend to think Taleb is being a bit unfair here: AI can't handle these kinds of events precisely because its creators (us) can't handle them! If we knew they were possible, then we could handle them pretty well, and AI could too.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Isn't the inability of these models (including human ones in the brain) to handle unknown-unknowns kind of his point? Since there is always a limit to the amount of accurate data we can gather in our samples and in instances of fat-tail distributions the effect of an outlier can be huge, whereas in a normal distribution the effect or damage of an extreme outlier will usually be pretty small. So it's as if he is saying this is a fundamental characteristic to knowledge and predictive systems, biological or machine based, hence why A.I. will be limited in certain domains. $\endgroup$
    – Josiah Swaim
    Aug 16, 2018 at 19:22
  • 2
    $\begingroup$ Hmm. I think there are two issues. One is the claim that we can't handle fat-tailed distributions with AI. This is false. The other is that, if you don't know what distributions are suited to the problem you're studying (that is, if you don't really understand your problem), then you'll be surprised by unexpected events. This is true. I think Taleb is conflating the two issues, when really they are separate. $\endgroup$
    – John Doucette
    Aug 16, 2018 at 20:10
0
$\begingroup$

Yes, Taleb is right. There are practical and mathematical limits even an AI, no matter how powefull, cannot overcome. Taleb point is that some events are inherently unpredictable. The amount of data the we would need to correctly guess their distribution is so massive we could never build a machine large enough that would run for long enough to provide accurate predictions:

enter image description here

Quote (and above image) from Taleb book on Statistical Consequences of Fat Tails (emphasis mine)

Once we leave the yellow zone, where the law of large numbers (LLN) largely works, and the central limit theorem (CLT) eventually ends up workin, then, we encounter convergence problems.

Further up, in the top segment, there is no mean. We call it the Fuhgetaboudit. If you see something in that category, you go home and you don’t talk about it.

The traditional statisticians approach to thick tails has been to claim to assume a different distribution but keep doing business as usual, using same metrics, tests, and statements of significance. Once we leave the yellow zone, for which statistical techniques were designed (even then), things no longer work as planned. The next section presents a dozen issues, almost all terminal.

Moreover, consider that not only you need a huge amount of data, but also that the interesting data-points for extremely rare events happen, as their name indicates, extremely rarely. So you might have to wait a literal eternity to ever have enough data to train the AI to make a prediction.

That said, even while AI cannot escape the mathematical boundaries that make prediction of extreme events impossible, it might perhaps be able to learn to do what Taleb recommends us to do: When dealing with extreme events, avoid prediction, and instead come up with approaches that make allow you to survive regardless lack of predictive power.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .