I have encountered this problem on how to predict the probability of a periodically happening event occurring at a given time.

For example, we have an event called being_an_undergrad. There are many data points: bob is an undergrad from (1999 - 2003), Bill is an undergrad from (1900 - 1903), Alice is an undergrad from (1900 - 1905), and there are many other data points such as (2010 - 2015), (2011 - 2013) ....

There are many events(data points) of being_an_undergrad. The lasting interval varies, it might be 1 year, 2 years, 3 years, .... or even 10 years. But the majority is around 4 years.

However, I am wondering given all the data points above. If I now know that Jason starts college in 2021, and how can I calculate/predict the probability that he will still be an undergrad in 2022? and 2023? and 2024 .... 2028, etc.

My current dataset consists of 10000 tuples representing events of different relations. The relations are all continuous relations similar to the example above. There are about 10 continuous relations in total in this dataset, such as isMarriedTo, beingUndergrad, livesIn, etc. For each relation, there are about 1000 data points(1000 durations) about this relation, for example,

<Leo, isUndergrad, Harvard, 2010 - 2011>, <Leo, isUndergrad, Stanford, 2013 - 2016>.....

<Jason, livesIn, US, 1990 - 2021>, <Richard, livesIn, UK, 1899- 1995> ...

My problem now is that I want to get a confidence level(probability) when I want to predict one event happening at a specific time point. For example, I want to predict the probability that event <Jason, livesIn, US, 2068> happens, given:

1.the above datasets which includes info about the relation: livesIn

2.the starting time when Mike lives in US, say he started to live in US since 2030.

I have used normal distribution to simulate, but I am wondering if there are any other better AI / ML / Stats approaches. Thanks a lot!

  • $\begingroup$ If the only data you have is samples from population of period lengths, and you want to predict probability of a new randomly sampled one (from the same population), then this looks like a basic question in statistics, which is not really an AI question. If you still think this is an AI question, could you explain where AI is involved? For instance, if there was a lot more contextual data about the time periods, then this might be a suitable problem for machine learning. If that is the case, please use edit to explain roughly what contextual data you have. $\endgroup$ Mar 27 at 11:06
  • $\begingroup$ Hi @NeilSlater! Thanks a lot for your comment. Yup, initially I thought of using statistics approaches such as survival function or poisson distribution. But their accuracy is not that good. Thus, I am thinking of using AI/ML to do the prediction. Actually from my point of view, AI is also like a way of modelling(just like physics/math). For example, we can also use ML/DL to learn the Newton's rules right haha(by inputting lots of experiment data to train the Newton's rule's model). Thus, I was wondering if there are similar time-series approaches in ML/DL that could tackle this problem. $\endgroup$
    – Leonard
    Mar 27 at 12:05
  • $\begingroup$ Unless there is some context that you have not explained in your data, a simple statistics approach will get the best accuracy, most ML methods should get close to it but will not be better, and can easily be worse. ML cannot improve on basic probabilities when the prediction is essentially a guessing game. $\endgroup$ Mar 27 at 13:45
  • $\begingroup$ Advanced AI might even be able to use information such as the student's name, because that may be correlated with demographics that do affect length of the course and likelihood to complete it. It is still not clear from your question that this is what you are asking though. $\endgroup$ Mar 27 at 13:49
  • $\begingroup$ Hi @NeilSlater! Thank you so much for your very detailed explanation : ) Actually in my context, I have a dataset of continuous relations similar to the example above. There are about 10 relations in total, such as isMarriedTo, beingUndergrad, livesIn, etc. For each relation, there are about 1000 data points about this relation(1000 durations), such as <Leo, isUndergrad, 2010 - 2011> and <Leo, isUndergrad, 2013 - 2016> ........ $\endgroup$
    – Leonard
    Mar 27 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.