1
$\begingroup$

I have used Beta function to estimate the performance of the agent. I have failure and success data of the task that runs on the agent. The parameter $\alpha$ is a number of successful tasks, while $\beta$ is the number of failures. Thus, I can estimate the performance by exploiting the expected value of Beta, as $$\mu = \frac{\alpha} {(\alpha+\beta)}$$

So, I am looking for a similar model, such that its parameter can be estimated from the success and failure data. So far I found Dirichlet distribution.

What is the expected value of Dirichlet distribution? How I can use the success and failure data to estimate parameters of this distribution?

Let's check the following example:

Suppose that we use a Dirichlet prior represented by $Dirichlet(1, 1, 1)$ and observe $13$ results with $8$ Successful, $2$ Missing, and $3$ Failures. Then we get the posterior to be $Dirichlet(1+8, 1+2, 1+3)$. Then if you define the performance value $\alpha$ to be the expectation of $P(x=Successful)$, then $\alpha$ will be $(1+8)/[(1+8)+(1+2)+(1+3)] = 0.56$

Now Suppose that we use a Beta prior represented by $Beta(1,1)$ and observe $13$ results with $8$ Successful, and $3$ Failures. Then we get the posterior to be $Beta(1+8, 1+3)$. Then if you define the performance value Pr to be the expectation of $P(x=Successful)$, then $\alpha = (1+8)/[(1+8)+(1+3)] = 0.69$

Are my calculations and concept right?

$\endgroup$
  • $\begingroup$ Is this problem in the context of reinforcement learning? So, is your agent an RL agent? $\endgroup$ – nbro Mar 9 at 3:43
  • $\begingroup$ Actually no, but thanks for this good idea, seems interesting $\endgroup$ – jou Mar 9 at 20:52
  • 1
    $\begingroup$ Your calculations seems correct but I dont know about its effectiveness. $\endgroup$ – DuttaA Mar 11 at 18:00
1
$\begingroup$

Dirichlet is the Multi Variate version of the Beta distribution. In general, these distributions can be thought to model the probability of modelling a probability distribution.

The support Dirichlet distribution is defined as follows:

$$ S_K = \{ x:0 \leq x_k \leq 1, \sum_{k=1}^K x_k=1 \} $$

and the PDF is defined as:

$$Dir(x|\alpha) = \frac{1}{B(\alpha)} \prod_{k=1}^Kx_k^{\alpha_k-1}$$

where $B(\alpha)$ is the beta function of $K$ variables:

$$B(\alpha) = \frac{\prod_{k=1}^K \tau(\alpha_k)}{\tau(\sum_{k=1}^K \alpha_k)}$$

and the resultant point estimates are (Define $\sum_{k=1}^K \alpha_k = \alpha_0)$:

$$\mu(x_k) = \frac{\alpha_k}{\alpha_0}$$ $$\sigma^2(x_k) = \frac{\alpha_k(\alpha_0-\alpha_k)}{\alpha_0^2(\alpha_0+1)}$$ $$mode[x_k] = \frac{\alpha_k-1}{\alpha_0-1}$$

Beta distribution is the special case where $k=2$

Clearly, when you run an experiment a large number of times, the success of each $k$ will approach towards its expected value i.e if you define your random variables as $x_k = \frac{N_{k}}{N}$ where $N$ is the total number of trials and $N_k$ is the success of the $k$ th term, it clearly satisfies the support of the Dirichlet Distribution and hence you can use

$$\frac{\alpha_{k}}{\alpha_0} = \frac{N_{k}}{N}$$

This is assuming that the experiment follows Dirichlet Distribution.

(Taken in parts from A Probabilistic Approach to ML)

| improve this answer | |
$\endgroup$
  • $\begingroup$ many thanks, @DuttaA. This is a good explanation. Let me ask this If we assign an agent with 13 jobs to represent as x. Then 10 success, and 3 failure. I Beta: With prior, B(1,1) we will get Performance = E(x) = 10+1/(10+3+2) = 0.73. In Dirichlet we will have Performance = E(x) = 10/13 = 0.76. Is this correct? I mean Numerator and denominator are same in Beta as well $\endgroup$ – jou Mar 9 at 20:17
  • 1
    $\begingroup$ @jou Like I said in my answer beta is a single/special case of Dirichlet distribution (with k=2, just put k=2 in my Dirichlet formula and see, $\alpha_1 = \alpha$ while $\alpha_2 = \beta$), and hence results should remain same for same case. While Beta will deal with a single agent (in your case) Beta deals with multiple agents. In your case you are treating a single agents success and failure, whereas another way to look at it is failure of one agent is success of another. Thus in Dirichlet, success belongs to a single agent while the rest fails. $\endgroup$ – DuttaA Mar 10 at 14:44
  • $\begingroup$ Many thanks @DuttaA. I study your formula, and I write an example to clarify any misusing. Kindly read my updated post again to read my example. $\endgroup$ – jou Mar 10 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.