# What is emperical distribution in MLE?

I was reading the book Deep Learning by Ian Goodfellow. I had a doubt in the Maximum likelihood estimation section (Pg 131). I understand till the Eq 5.58 which describes what is being maximized in the problem.

$$\theta_{\text{ML}} = \text{argmax}_{\theta} \sum_1^m \log(p_{\text{model}}(x^{(i)};\theta))$$

However the next equation 5.59 restates this equation as:

$$\theta_{\text{ML}} = \text{argmax}_{\theta} E_{x \sim \hat{p}_{\text{data}}}(\log(p_{\text{model}}(x;\theta))$$

where $$\hat{p}_{\text{data}}$$ is described as the empirical distribution defined by the training data. Could someone explain what is meant by this empirical distribution? It seems to be different from the distribution parametrized by theta as that is described by $$p_{\text{model}}$$

Let's consider a die. Each number is equally likely to show up if we throw the die, so the true underlying distribution is uniform over the set $$\{1, 2,\dots6\}$$. Now let's say you ask your friend to throw the die 60 times, what you will see is likely something close to uniform over the set $$\{1, 2,\dots6\}$$ (not really uniform though, as that would be highly unlikely). This distribution is the empirical one, and as you collect more and more samples it will converge to the actual underlying distribution.
$$x_1,\dots, x_m$$ is your sample (in the example above, the $$60$$ numbers that you see as your friend throws the die). This sample defines a distribution, $$\hat{p}_{data}$$. In the example above, $$\hat{p}_{data}$$ would likely be close to the uniform distribution over $$\{1, \dots, 6\}$$. Now you can think about the sum over $$x_1\dots, x_m$$ of $$\log(p_{model}(x^{(i)};\theta))$$ as the average of $$\log(p_{model}(x;\theta))$$, where $$x$$ is drawn according to $$\hat{p}_{data}$$.
Let me make another example with some actual numbers. Let's say you toss a fair coin $$7$$ times and you see $$\{H, H, H, T, H, T, H\}.$$ The empirical distribution is $$\mathbb{P}(H)=5/7$$, $$\mathbb{P}(T)=2/7$$. So if you compute the expectation $$\mathbb{E}_{x \sim \hat{p}_{data}}[\log(p_{model}(x;\theta)]$$ you get $$\log(p_{model}(H;\theta)\cdot\mathbb{P}(H) +\log(p_{model}(T;\theta)\cdot\mathbb{P}(T) = \\ \frac{5}{7}\log(p_{model}(H;\theta) +\frac{2}{7}\log(p_{model}(T;\theta)),$$ which is what you will get if you compute the first sum you wrote.