I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships.
How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?
Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:
$$ P(A|B) = \frac{P(B|A)P(A)}{P(B)} $$
So if you have $P(B|A)$, $P(A)$, and $P(B)$, we can determine $P(A|B)$, and similarly you can determine $P(B|A)$ from $P(A|B)$, $P(B)$ and $P(A)$. Just by looking at $P(B|A)$ and $P(A|B)$, it is therefore impossible to tell what the causal direction is (if any).
In fact, probabilistic inference usually works the other way round: When there is a known causal relation, say from diseases $A$ to symptoms $B$, we usually have $P(B|A)$, and are interested in the diagnostic reasoning task of determining $P(A|B)$ from that. (The only other thing we need for that is the prior probability $P(A)$ since $P(B)$ is just a normalization factor.)
But the conditional probability is clearly not symmetric and captures directed relationships.
One needs to consider the kinds of directed relationships that is captured by conditional probability. It surely does capture some kind of association or dependence which could be directed. At the same time, it is not right to say that it surely captures the causal relationships.
Let:
Sun rises = $A$, Rooster crows = $B$, then, $P(A |B)$ is bound to be very high but it does not mean that rooster crowing causes sunrise.
How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?
He will ask him to go back to school.
Why isn't conditional probability sufficient to describe causality?
Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen
the height of the column of mercury in your barometer drops below a certain level
a storm occurs
We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm.
However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely only correlated with the storm, so it is not the cause of the storm.
How can we see that the drop in barometric pressure is or not the cause of the storm?
We can compare the probabilities $P(A \mid \text{do}(B))$ and $P(A \mid B)$, where $A$ is the event "a storm occurs" and $B$ is the event "drop in barometric pressure". What does $\text{do}(B)$ mean? It means that we force the event $B$ to occur, that is, we force the drop in barometric pressure to occur. Intuitively, what is then the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$? In the case of $P(A \mid \text{do}(B))$, we force the event $B$ to always occur. In the case of $P(A \mid B)$, we only and passively look at the cases of event $A$ when event $B$ occurs (without thus forcing $B$ to occur). We now know the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$. However, how does this help us to understand that $B$ is not the cause of $A$? If $B$ was a cause of $A$, then, if we forced $B$ always to occur, then the probability of $A$ should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of $A$ does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.
To conclude, Judea Pearl would say that do-operators (or interventions) are required to analyze causal relationships.
The article Probabilistic Causation by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as forks, chains and colliders), d-separation or Bayesian networks.
What Pearl means by being not symmetric is: $A=B$ and $B=A$ are exactly identical and lead to the same result in a not-causal scientific framework. For example consider very simple set of equations:
$$ \begin{align} Z = & \epsilon_z \\ X = & Z+\epsilon_X \\ Y = & 2X+Z + \epsilon_y \end{align} $$
From the Algebra point of view, this is a set of 3 equations, and 3 unknowns (consider error terms to be known). You can shuffle the equations, change the LHS and EHS of the equations, add or subtract them. In fact these actions are actually appreciated for solving a linear equation system, righ?! So the system above is identical to this:
$$ \begin{align} \epsilon_z = & Z \\ X = & Y - 2Z - \epsilon_Y - \epsilon_X \\ Y = & 2X+Z + \epsilon_y \\ \end{align} $$
But I brought to you this example because the first one is the set of equations for a very fundamental causal structure called confounding or common cause structure, Where $X$ is the exposure, or treatment, $Y$ is the outcome, and $Z$ is the parent of both $X$ and $Y$. The orders of the equations and the RHS/LHS variables in this Structural Equation Models actually mean something. First you calculate $Z$, then go for $X$, then calculate $Y$. In this case, the second system is pointing to a completely different causal structure (or to be honest, it is not even look like a legit structural equation)
First I would like to say this would be a very ironic question to ask from Pearl, as he also mentioned in one of his interviews, because he has had a significant contribution to the realm of probability theory with bayesian network and bayesian inference framework! And now it is like he is against himself. But he is for a good reason.
why Probability is not enough can be and has been answered with formal proofs, equations and explanations. But There are a lot of examples that will draw your attention to this truth. I recommend you to read about Simpson's Paradox. It will be a great example to see how probability theory is incapable of causal inference.
The fact that probability is not enough, only resembles the idea that correlation is not necessarily causation. and that is true. Again, read about spurious correlations and you will get it. Just think of this funny example: During the year, Crime rate and Ice cream sales amount are highly correlated, THUS we must ban ice cream sales to control the crime rate. and the problem with this dumb inference is that we have not taken into account the common cause which is heat or summer that accounts for the perceived correlation.
