# What are the different possible usages of the word "i.i.d" in machine learning?

The acronym "iid" stands for "independent and identically distributed". It is a property of a sequence of random variables. You can read here for more details. This question is just about the usage of the word "iid" in contemporary machine learning and is not about the feasibility of checking iid based on either associated joint distribution or dataset.

In the formal and strict sense, the word "iid" should be used only as a property for a sequence of random variables based on the underlying joint probability distribution function. But, I noticed that there is another (maybe less-strict) usage for the word 'iid' based on the context.

Consider the following statements compiled from different answers to my questions 1,2

The term i.i.d. is a property of a dataset. A dataset can be created that is i.i.d. with respect to a particular probability distribution. It doesn't matter what that distribution is, it just has to exist, and be relevant to the purpose the ML is being put to.

The point is even you know the distribution, sometimes you can't prove that the sampled data is i.i.d. or not!....

....A table of results of dice throws is likely iid...... (there are some issues with this answer, but the bolded excerpt is true)

So, the usage of the word iid, in this sense, is somewhat different. Although I think, iid is a property of a sequence of random variables in this sense also, it is okay to use the word 'iid' for a dataset (collection of samples) since the dataset represents some underlying probability distribution.

Thus, the two usages I am aware of up to now are

1. iid for a sequence of random variables based on joint distribution.

2. iid for a sequence of random variables based on the collection of samples.

Is my understanding of the two usages of the word "iid" correct? and are there any other usages for the word "iid"?

• @NeilSlater what about the bolded portion? Sep 14, 2021 at 7:43
• "[A] table of results of dice throws is likely iid" seems correct to me (with reasonable assumptions). However, the non-bolded "it is because the dice roll itself is iid" is not correct Sep 14, 2021 at 7:46
• Why do you get hung up on definitions of words so much? The important thing is the concept, not the association between words and concepts. Different people may use the same word with slightly different definitions, and that's fine. Sep 14, 2021 at 8:10
• @user253751 It may be due to self-study. I am trying to understand underlying concepts. Unfortunately, these terminological issues are always popping up for me. Sep 14, 2021 at 8:25
• consider that a sample is a random variable, in a way .When you sample random variable X 5 times it is the same as sampling from 5 random variables whose distribution is identical to the distribution of X and which are independent. Sep 14, 2021 at 8:30