2
$\begingroup$

While analyzing the data for a given problem set, I came across a few distributions which are not Gaussian in nature. They are not even uniform or Gamma distributions(so that I can write a function, plug the parameters and calculate the "Likelihood probability" and solve it using Bayes classification method). I got a set of a few absurd looking PDFs and I am wondering how should I define them mathematically so that I can plug the parameters and calculate the likelihood probability.

The set of PDFs/Distributions that I got are the following and I am including some solutions that I intend to use. Please comment on their validity:

1)enter image description here

The distribution looks like:

$ y = ax +b $ from $ 0.8<x<1.5 $

How to programmatically calculate

1. The value of x where the pdf starts
2. The value of x where the pdf ends
3. The value of y where the pdf starts
4. The value of y where the pdf ends

However, I would have liked it better to have a generic distribution for this form of graphs so that I can plug the parameters to calculate the probability.

2)enter image description here

This PDF looks neither uniform nor Gaussian. What kind of distribution should I consider it roughly?

3)enter image description here

I can divide this graph into three segments. The first segment is from $2<x<3$ with a steep slope, the second segment is from $3<x<6$ with a moderate sope and the third segment is from $6<x<8$ with a high negative slope.

How to programmatically calculate

 1. the values of x where the graph changes its slope.
 2. the values of y where the graph changes its slope.

4)enter image description here

This looks like two Gaussian densities with different mean superimposed together. But then the question arises, how do we find these two individual Gaussian densities?

The following code may help:

variable1=nasa1['PerihelionArg'][nasa1.PerihelionArg>190] 
variable2=nasa1['PerihelionArg'][nasa1.PerihelionArg<190]

Find mean and variance of variable1 and variable2, find the corresponding PDFs. Define the overall PDF with a suitable range of $x$.

5)enter image description here

This can be estimated as a Gamma distribution. We can find the mean and variance, calculate $\alpha$ and $\beta$ and finally calculate the PDF.

It would be very helpful if someone could give their insights on the above analysis, its validity, and correctness and their suggestions regarding how problems such as these should be dealt with.

Improve this question
$\endgroup$
8
  • $\begingroup$ I am trying to train the classifier using Bayes theorem. Suppose for a given set of input, I want to determine if that asteroid is hazardous or not. For that, we need to calculate the probability P(Perihelion Time/Asteroid is Hazardous), that is, what is the probability that the asteroid takes the particular "Perihelion time" (mentioned in the test input) given that the asteroid is "hazardous". $\endgroup$
    – Soumee
    Aug 6, 2019 at 6:06
  • $\begingroup$ So, to calculate P(Perihelion Time/Asteroid is Hazardous) we can segregate those values of Perihelion Time for which the asteroid is hazardous, compute its mean and variance and draw the pdf $$\text{Gamma}(\alpha,\lambda)\implies\frac{{\lambda}^{\alpha}}{\Gamma(\alpha)}\cdot x^{\alpha-1}e^{-\lambda x}$$, plug the value of x(=Perihelion Time) and calculate the probability. $\endgroup$
    – Soumee
    Aug 6, 2019 at 6:06
  • $\begingroup$ I want to know that how should we calculate pdfs of graphs which do not belong to the general category of pdfs such as Gaussian or Gamma distributions. $\endgroup$
    – Soumee
    Aug 6, 2019 at 6:06
  • $\begingroup$ You need to perform proper statistical tests to conclude some reasonable distributional assumptions. $\endgroup$
    – naive
    Aug 7, 2019 at 18:21
  • $\begingroup$ Try a larger data set. If you dont have one use what you have and do a k fold cross validation. If you have outliers you have outliers. NB isn't perfect. $\endgroup$
    – solarflare
    Aug 8, 2019 at 1:49

1 Answer 1

Reset to default
1
$\begingroup$

The relationship between the axes of graph (1) and your variables $x$ and $y$ is not clear, so this generalized answer may be helpful or useless.

From graph (1) it appears that the correlation coefficient $\mathcal{C}$ of a quadratic fit of data set $\mathcal{S}$ would be much better. Consider $y_1$ and $y_2$ approximations of $y$.

$$ \mathcal{C} (y_2, a, b, c, \mathcal{S}) > \mathcal{C} (y_1, a, b, \mathcal{S}) \\ y_2 = ax^2 + bx + c \\ y_1 = ax + b $$

To achieve a more nearly uniform distribution, perform a least squares fit for $y_2$ against $y$ on $\mathcal{S}$ to obtain $(a, b, c)$. Then find a mapping function that produces $y'$ and use it where the uniform distribution is desired. A reasonable approximation is simply this.

$$y' = \frac{y}{y_2(x)}$$

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .