I am using LSTM network for predicting IOT time-series data receiving from un-reliable devices and networks.
This results in several multiple sections [continuous streak of bad data for several days until the problem is fixed].
I need to exclude this bad data section before feeding it to model training.
Since I am using LSTM-RNN network, it requires to do an un-roll data based on the previous records.

How can I properly exclude this bad data?
I thought of an approach as training model separately using each batch of good data, and use subsequent good-data batch for fine-tuning the model.
Please let me know if this is a good approach? or is there a better method?

example data:
"1-04",0  [bad data]
"1-05",0  [bad data]
"1-06",0  [bad data]
"1-07",0  [bad data]
"1-11",0  [bad data]
"1-12",0  [bad data]

1 Answer 1


Your idea is a good one.

Another idea is to upsample or aggregate your data. For example, average by week if you generally have a couple of missing days in every week.

A similar question on Stack Exchange: https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .