# How can I use the success and failure data to estimate parameters of a Dirichlet distribution?

I have used Beta function to estimate the performance of the agent. I have failure and success data of the task that runs on the agent. The parameter $$\alpha$$ is a number of successful tasks, while $$\beta$$ is the number of failures. Thus, I can estimate the performance by exploiting the expected value of Beta, as $$\mu = \frac{\alpha} {(\alpha+\beta)}$$

So, I am looking for a similar model, such that its parameter can be estimated from the success and failure data. So far I found Dirichlet distribution.

What is the expected value of Dirichlet distribution? How I can use the success and failure data to estimate parameters of this distribution?

Let's check the following example:

Suppose that we use a Dirichlet prior represented by $$Dirichlet(1, 1, 1)$$ and observe $$13$$ results with $$8$$ Successful, $$2$$ Missing, and $$3$$ Failures. Then we get the posterior to be $$Dirichlet(1+8, 1+2, 1+3)$$. Then if you define the performance value $$\alpha$$ to be the expectation of $$P(x=Successful)$$, then $$\alpha$$ will be $$(1+8)/[(1+8)+(1+2)+(1+3)] = 0.56$$

Now Suppose that we use a Beta prior represented by $$Beta(1,1)$$ and observe $$13$$ results with $$8$$ Successful, and $$3$$ Failures. Then we get the posterior to be $$Beta(1+8, 1+3)$$. Then if you define the performance value Pr to be the expectation of $$P(x=Successful)$$, then $$\alpha = (1+8)/[(1+8)+(1+3)] = 0.69$$

Are my calculations and concept right?

• Is this problem in the context of reinforcement learning? So, is your agent an RL agent? – nbro Mar 9 '20 at 3:43
• Actually no, but thanks for this good idea, seems interesting – jou Mar 9 '20 at 20:52
• Your calculations seems correct but I dont know about its effectiveness. – user9947 Mar 11 '20 at 18:00

Dirichlet is the Multi Variate version of the Beta distribution. In general, these distributions can be thought to model the probability of modelling a probability distribution.

The support Dirichlet distribution is defined as follows:

$$S_K = \{ x:0 \leq x_k \leq 1, \sum_{k=1}^K x_k=1 \}$$

and the PDF is defined as:

$$Dir(x|\alpha) = \frac{1}{B(\alpha)} \prod_{k=1}^Kx_k^{\alpha_k-1}$$

where $$B(\alpha)$$ is the beta function of $$K$$ variables:

$$B(\alpha) = \frac{\prod_{k=1}^K \tau(\alpha_k)}{\tau(\sum_{k=1}^K \alpha_k)}$$

and the resultant point estimates are (Define $$\sum_{k=1}^K \alpha_k = \alpha_0)$$:

$$\mu(x_k) = \frac{\alpha_k}{\alpha_0}$$ $$\sigma^2(x_k) = \frac{\alpha_k(\alpha_0-\alpha_k)}{\alpha_0^2(\alpha_0+1)}$$ $$mode[x_k] = \frac{\alpha_k-1}{\alpha_0-1}$$

Beta distribution is the special case where $$k=2$$

Clearly, when you run an experiment a large number of times, the success of each $$k$$ will approach towards its expected value i.e if you define your random variables as $$x_k = \frac{N_{k}}{N}$$ where $$N$$ is the total number of trials and $$N_k$$ is the success of the $$k$$ th term, it clearly satisfies the support of the Dirichlet Distribution and hence you can use

$$\frac{\alpha_{k}}{\alpha_0} = \frac{N_{k}}{N}$$

This is assuming that the experiment follows Dirichlet Distribution.

(Taken in parts from A Probabilistic Approach to ML)

• many thanks, @DuttaA. This is a good explanation. Let me ask this If we assign an agent with 13 jobs to represent as x. Then 10 success, and 3 failure. I Beta: With prior, B(1,1) we will get Performance = E(x) = 10+1/(10+3+2) = 0.73. In Dirichlet we will have Performance = E(x) = 10/13 = 0.76. Is this correct? I mean Numerator and denominator are same in Beta as well – jou Mar 9 '20 at 20:17
• @jou Like I said in my answer beta is a single/special case of Dirichlet distribution (with k=2, just put k=2 in my Dirichlet formula and see, $\alpha_1 = \alpha$ while $\alpha_2 = \beta$), and hence results should remain same for same case. While Beta will deal with a single agent (in your case) Beta deals with multiple agents. In your case you are treating a single agents success and failure, whereas another way to look at it is failure of one agent is success of another. Thus in Dirichlet, success belongs to a single agent while the rest fails. – user9947 Mar 10 '20 at 14:44
• Many thanks @DuttaA. I study your formula, and I write an example to clarify any misusing. Kindly read my updated post again to read my example. – jou Mar 10 '20 at 22:02