I have a big amount of light curves (image below) and I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve). However, the data is not labeled. I tried labeling it by hand and using a bi-directional LSTM succeeds in labeling the data points properly. However, there are thousands of light curves and labeling all of them would take very long. Is there any good unsupervised approach to do this (unsupervised LSTM maybe, but any other method that might work on time series would do just fine)? Thank you! enter image description here

  • $\begingroup$ What do you mean by "light curves"? Do you mean the curves associated with the waves of light? Furthermore, what kind of labels could there be? $\endgroup$ – nbro Apr 24 '19 at 20:13
  • $\begingroup$ @nbro Sorry for the confusion. I added an image of a lightcurve. I want to identify the downward spikes (class label = 1) from the background (class label = 0). But the data is not labeled and labeling it by hand would take a long time. $\endgroup$ – Alex Marshall Apr 24 '19 at 21:31
  • $\begingroup$ It sounds like you’ve labeled some of the data by hand. Can you use a clustering algorithm trained on the small amount of data you’ve already labeled? If the dataset is quite imbalanced, you may be able to separate the data into two clusters and check some of the results for signal by hand? Or maybe you can set it up as an “anomaly detection” network, which you can find many example implementations of online. That way you could separate signal (which would appear as anomaly) from noise (which seemingly dominates the signal). $\endgroup$ – Hanzy Apr 28 '19 at 22:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.