# Why isn't conditional probability sufficient to describe causality?

I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships.

How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?

Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:

$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$

So if you have $$P(B|A)$$, $$P(A)$$, and $$P(B)$$, we can determine $$P(A|B)$$, and similarly you can determine $$P(B|A)$$ from $$P(A|B)$$, $$P(B)$$ and $$P(A)$$. Just by looking at $$P(B|A)$$ and $$P(A|B)$$, it is therefore impossible to tell what the causal direction is (if any).

In fact, probabilistic inference usually works the other way round: When there is a known causal relation, say from diseases $$A$$ to symptoms $$B$$, we usually have $$P(B|A)$$, and are interested in the diagnostic reasoning task of determining $$P(A|B)$$ from that. (The only other thing we need for that is the prior probability $$P(A)$$ since $$P(B)$$ is just a normalization factor.)

But the conditional probability is clearly not symmetric and captures directed relationships.

One needs to consider the kinds of directed relationships that is captured by conditional probability. It surely does capture some kind of association or dependence which could be directed. At the same time, it is not right to say that it surely captures the causal relationships.

Let:

Sun rises = $$A$$, Rooster crows = $$B$$, then, $$P(A |B)$$ is bound to be very high but it does not mean that rooster crowing causes sunrise.

How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?

He will ask him to go back to school.

• I do not think that you actually even try to clarify the doubt of the asker. You're just saying that correlation does not imply causation. – nbro Aug 15 '19 at 22:47
• @nbro I have tried to correct the argument provided by the asker, which I believe has led to the question in the first place. I think the example in my answer will clarify that conditional probability is not sufficient to explore causal relationships. How to explore causal relationships? That is another question. – naive Aug 16 '19 at 2:56

Conditional probability does not describe causality at all, since description is a layer of detail about some state, action, scenario, or phenomenon. Causality is a level of abstraction above description. Conditional probability does not imply causality either, because implication is also an abstraction, above causality and description.

In a boxing match, a punch and a person knocked out may occur in succession indicating a high probability that the punch caused the knock out. It may be reasonable to say that the repetition of the trend of knock outs following punches supports this probability of causality, supporting the correlation between conditional probability and causality, but the correlation coefficient is certainly < 1.0 for the general case.

Most that would be studying conditional probability already realize that the two observations (punch, knock out) represent a minuscule subset of all observations that could be made with the right instrumentation. The person may have fainted from dehydration immediately after being punched and the temporal alignment of the two events leads to an incorrect conclusion about cause among the observers. On a higher level of abstraction, the person may only appear knocked out but is acting in a boxing sequence on a realistically choreographed movie shoot.

If the appearance of impact precedes the appearance of unconsciousness, we can correctly make probabilistic statements about causality within an abstract system we constructed about the class of phenomena about which the observations are a part. The announcer might say, "And a round house from Thompson brings Pauly swirling to the floor." The fight is judged on the basis of conclusions about causality that stem from observing multiple fights. Media channels are filled with such conclusions.

Human beings are swimming in a pool of unproven statements presented as facts, so most humans are unaware of the basis upon which they draw conclusions. Diligent and well educated scientists are more likely to remember these concepts:

• Only in virtual scenarios brought to life in human constructions such as chess games and digital circuits can observation be exhaustive.
• Purely unidirectional causality is rare, such as the moonlight on a camera lens. The sun is unaffected by the reflection of light off of the camera lens but the exposure of the image is the detection of solar nuclear reaction. Within the biosphere, physical phenomena more commonly exhibit bidirectional information flow. A acts on B while B acts on A.
• Most descriptions of an event are of a trend found in a cacophony of tiny interactions, such as a punch and knock out being a large number of electrical events in a set of neurons, the elastic deformation of a large number of molecules within cells and bone material, and a large number of electrical events in a second set of neurons.

Human cognitive abilities embrace all of these abstractions of composition and causality, and many thoughtful humans attach some level of doubt about causality when observations appear to indicate it. We can hear reports like, "I think the shot came from behind me and I saw the President's head jerk back." The causality can nonetheless be debated for decades.

We see a panic and crash in a market, but there is no singular panic and singular crash. No person's trade, public message, personal financial concern, or verbal expression causes the entire trend we examine in a graph part way through or after the sequence of singular events marking such a scenario.

In electronics, we take great pains to permit circuit A's control of circuit B without allowing the back flow of causality from B to A. We like digital electronics because we get a purified universe where causality is unidirectional except where we specify feedback, as in the case of digital learning that depends on successive approximation.

All of these things are part of the framework of concepts that decouple conditional probability from conclusive causality.

A statement concluding that we don't have causality is not particularly compelling as a scientific statement. Causality is not something we can have or not have universally, and what one thinks they have another may think they don't, so the first person plural is presumptuous. A statement concluding that physical equations are symmetric can be correct if the symmetry to which the author is referring is regarding bidirectional impact, not reversibility, and equations are restricted to equalities. However, such a statement is not generally true either.

It may be reasonable to say any of these statements.

• Any consistent temporal pattern of asymmetry gives us information about causality but cannot be conclusive.
• Causal interaction may be bidirectional without being balanced in magnitude such that one side may seem to cause the other when what appears as cause and effect both occur as part of a single unified phenomenon.
• Alleged causality written into historic accounts can be unreliable.
• At a quantum level, causality is not strictly applicable.
• The concept of causality is an abstraction that can be misleading if applied in scenarios that have bidirectional imposition of forces or fields or bidirectional information exchange.

The concluding question is interesting, but the approaches of educators and media figures in answering such questions are usually context dependent.

How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?

It is easy to imagine a defensive retort, a response indicating thoughtful consideration of the proposal, or (in the middle ground) a pedantic one. A private conversation over a coffee or some sushi may result in the more thoughtful conversation.

What Pearl means by being not symmetric is: $$A=B$$ and $$B=A$$ are exactly identical and lead to the same result in a not-causal scientific framework. For example consider very simple set of equations:

\begin{align} Z = & \epsilon_z \\ X = & Z+\epsilon_X \\ Y = & 2X+Z + \epsilon_y \end{align}

From the Algebra point of view, this is a set of 3 equations, and 3 unknowns (consider error terms to be known). You can shuffle the equations, change the LHS and EHS of the equations, add or subtract them. In fact these actions are actually appreciated for solving a linear equation system, righ?! So the system above is identical to this:

\begin{align} \epsilon_z = & Z \\ X = & Y - 2Z - \epsilon_Y - \epsilon_X \\ Y = & 2X+Z + \epsilon_y \\ \end{align}

But I brought to you this example because the first one is the set of equations for a very fundamental causal structure called confounding or common cause structure, Where $$X$$ is the exposure, or treatment, $$Y$$ is the outcome, and $$Z$$ is the parent of both $$X$$ and $$Y$$. The orders of the equations and the RHS/LHS variables in this Structural Equation Models actually mean something. First you calculate $$Z$$, then go for $$X$$, then calculate $$Y$$. In this case, the second system is pointing to a completely different causal structure (or to be honest, it is not even look like a legit structural equation)

## About sufficiency of Probability Theory for Causal Inference

First I would like to say this would be a very ironic question to ask from Pearl, as he also mentioned in one of his interviews, because he has had a significant contribution to the realm of probability theory with bayesian network and bayesian inference framework! And now it is like he is against himself. But he is for a good reason.

why Probability is not enough can be and has been answered with formal proofs, equations and explanations. But There are a lot of examples that will draw your attention to this truth. I recommend you to read about Simpson's Paradox. It will be a great example to see how probability theory is incapable of causal inference.

The fact that probability is not enough, only resembles the idea that correlation is not necessarily causation. and that is true. Again, read about spurious correlations and you will get it. Just think of this funny example: During the year, Crime rate and Ice cream sales amount are highly correlated, THUS we must ban ice cream sales to control the crime rate. and the problem with this dumb inference is that we have not taken into account the common cause which is heat or summer that accounts for the perceived correlation.

Why isn't conditional probability sufficient to describe causality?

Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen

1. the height of the column of mercury in your barometer drops below a certain level

2. a storm occurs

We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm. However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely only correlated with the storm, so it is not the cause of the storm.

How can we see that the drop in barometric pressure is or not the cause of the storm?

We can compare the probabilities $$P(A \mid \text{do}(B))$$ and $$P(A \mid B)$$, where $$A$$ is the event "a storm occurs" and $$B$$ is the event "drop in barometric pressure". What does $$\text{do}(B)$$ mean? It means that we force the event $$B$$ to occur, that is, we force the drop in barometric pressure to occur. Intuitively, what is then the difference between $$P(A \mid \text{do}(B))$$ and $$P(A \mid B)$$? In the case of $$P(A \mid \text{do}(B))$$, we force the event $$B$$ to always occur. In the case of $$P(A \mid B)$$, we only and passively look at the cases of event $$A$$ when event $$B$$ occurs (without thus forcing $$B$$ to occur). We now know the difference between $$P(A \mid \text{do}(B))$$ and $$P(A \mid B)$$. However, how does this help us to understand that $$B$$ is not the cause of $$A$$? If $$B$$ was a cause of $$A$$, then, if we forced $$B$$ always to occur, then the probability of $$A$$ should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of $$A$$ does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.

To conclude, Judea Pearl would say that do-operators (or interventions) are required to analyze causal relationships.

The article Probabilistic Causation by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as forks, chains and colliders), d-separation or Bayesian networks.

• So, basically your example hints at performing controlled experiment to arrive at a conclusion. But, you fail to mention that one needs to control for all other relevant random variables that could possibly affect the occurrence of a storm. – naive Aug 16 '19 at 6:45
• @naive I didn't want to go into the details of how you implement the do operations or randomized experiments. – nbro Aug 16 '19 at 9:24
• Instead of introducing the do-operations, if the answer reflects a bit upon upon controlled experiments it would become more useful for the OP as one can see the OP clearly lacks that understanding. Introducing the do-operation without any background of controlled experiments seems to only provide marginal benefit to the OP. – naive Aug 16 '19 at 9:59
• @naive I don't think so, because I also explain what do operations are. – nbro Aug 16 '19 at 11:08