I am using LSTM network for predicting IOT time-series data receiving from un-reliable devices and networks.
This results in several multiple sections [continuous streak of bad data for several days until the problem is fixed].
I need to exclude this bad data section before feeding it to model training.
Since I am using LSTM-RNN network, it requires to do an un-roll data based on the previous records.
How can I properly exclude this bad data?
I thought of an approach as training model separately using each batch of good data, and use subsequent good-data batch for fine-tuning the model.
Please let me know if this is a good approach? or is there a better method?
example data:
"1-01",266.0
"1-02",145.9
"1-03",183.1
"1-04",0 [bad data]
"1-05",0 [bad data]
"1-06",0 [bad data]
"1-07",0 [bad data]
"1-08",224.5
"1-09",192.8
"1-10",122.9
"1-11",0 [bad data]
"1-12",0 [bad data]
"2-01",194.3
"2-02",149.5