In Probabilistic Graphical Model (written by Daphne Koller), what's the meaning of "parameter" in representation of the distribution?

Question

I just started to read the PGM book written by Daphne Koller.

In the chapter of Bayesian Network Representation(Chapter 3), there are some descriptions about the standard parameterization of the joint distribution corresponding to n-trial coin tosses.

The book also says,

Here I'm very confused about the meaning of $ 2^n parameters $. In terms of random variable or probability distribution, parameter means characteristic of the distribution. But parameter in this paragraph sounds like $O(2^n)$ space complexity. Because it also describes that we can reduce the space of all joint distribution to $n$-dimension by using expression $ \prod_{i} \theta_{x_{i}} $.

So, what's the meaning of parameter in this context? Does it mean space complexity for computation of the joint distribution?

Answer 1

I believe you are correct. When Koller mentions a parameter, it is in reference to the values needed to express the probability distribution. For example, the total number of parameters for a Bernoulli distribution is one, the probability of success. You can watch Nando de Freitas's lecture on undergraduate machine learning 8: Inference in Bayesian networks and dynamic programming for a detailed walkthrough of a binary Bayesian network where Nando also explains how many parameters are needed for each table in the network. In the lecture, he also briefly discussed some implementation details for Bayesian Networks, which also hints toward answering your question.