# Pattern recognition for live stream Time serie

I would like to submit you a problem with which I struggle.

Suppose I have this kind of record over time in a dataframe:

fig.1

If we zoom in a bit we see such shape:

fig.2

We see that the general pattern is a increase with a pick (very high or very thin sometime even flat) follow by an almost flat part with vibration then a decrease, the we go back to zero (almost) for a time (like in the middle of fig.1) or we start an other cycle

Some have very high peak, some are more flat, some have a more longuer part before to decrease.

I have 4 classes :

• increase time - 1
• working time - 2
• decrease time - 3
• rest time (no activity) - 0

Now assume in my dataframe I have columns that tell to what class belong each point in time.

I would like to build a model that can recognize those 4 class when it see it on stream data . Imagine that our stream data is fig.1 and that we read N points (on a sliding windows) over time. What model could allow me to classify correctly each point or subpart data point in this window according to a certain pattern (hope I'm clear) Regarding the fact that in reallity I could be in rest time for a very long time or in working time a very long time also. It may also depend a lot of the sliding window, we for exemple see the beginning of the increase time on the first window then it end on the next window.

I first try to use LSTM or 1D-CNN, problem I have is that it tend to see this general pattern even when it's not present.

--- UPDATE : Chillston

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=777, shuffle=False)

# input N x T x D
X_train.shape, y_train.shape, X_test.shape, y_test.shape,

((17562, 1000, 1), (17562, 1000, 4), (4391, 1000, 1), (4391, 1000, 4))


I then pass it to 1d-CNN model, I have try many Architecture even resnet-cnn version, here's it's just more classical one.

def build_res1dcnn(n_classes):

input_shape = (X_train.shape[1], 1)

inputs = Input(shape = input_shape, name = 'input')

# Stage 1
x = Conv1D(64, kernel_size=3, strides = 2, padding = 'same', activation = 'relu',
kernel_regularizer = 'l2', kernel_initializer = 'he_normal',
bias_regularizer = 'l2')(inputs)
x = BatchNormalization()(x)
x = MaxPooling1D(pool_size=(2))(x)
x = Dropout(0.2)(x)

x = Conv1D(128, kernel_size= 5, strides = 2, padding = 'same', activation = 'relu',
kernel_regularizer = 'l2', kernel_initializer = 'he_normal',
bias_regularizer = 'l2')(inputs)
x = BatchNormalization()(x)
x = MaxPooling1D(pool_size=(2))(x)
x = Dropout(0.2)(x)

x = Conv1D(64, kernel_size=3, strides = 2, padding = 'same', activation = 'relu',
kernel_regularizer = 'l2', kernel_initializer = 'he_normal',
bias_regularizer = 'l2')(inputs)
x = BatchNormalization()(x)
x = MaxPooling1D(pool_size=(2))(x)
x = Dropout(0.2)(x)

x = GlobalMaxPooling1D()(x)

# Here I want T x n_class for a T X 1 input sequence
outputs = []

# https://stackoverflow.com/questions/51397484/appending-tensors-in-keras
Ty = X_train.shape[1]
for i in range(Ty):
out = Dense(n_classes, activation = "softmax")(x)
outputs.append(out)

output = Concatenate()(outputs)
output = Reshape([Ty, n_classes])(output)

model = Model(inputs = inputs, outputs=output)

loss = 'categorical_crossentropy',
metrics = ['categorical_accuracy'])

#model.summary()

return model

• As your data is coming in as a stream, I would try a model that, given some number of historic samples (in your case you already have a sliding window $w_{t-n:t}$), classifies only the last sample of that window. That way the model can base the prediction on the $n$ previous samples. And from the looks a 1D-CNN should work fine with that data. Maybe you can provide a bit more info on what you have tried and what exactly didn't work? May 9 at 21:47
• @Chillston Hello, sure. I have update my post. First I start by cutting 1000 points in the fig.1 with a step of 500 points (before I was cutting sequence in fig.1 so to have many sequence like those on fig.2) - maybe what cause the model to see this general patttern everywhere). I also get the label associated with those point in "y" variable, then I one-hot encode "y". after that I send it to 1d-cnn that predict (N x T X n_class). I guess the real problem is the model architecture+hyperparam combine with windows size from train and test. May 11 at 12:47

There are a number of possibilities you can consider:

## simple classification

Following what you have suggested as your problem definition, you can train a model for those labels, given a hisotry of some window size (your hyper-parameter). In that case, common timeseries models can be utilized (LSTM, TCN, MCNN ). I prefer the MCNN method for it is a nice model especially when fixing a proper window-size is a challenge. It probably outperforms LSTM, so chances are you'll have better results. Moreover, LSTM must be improved with Attention mechanism, as you said the main challenge of your problem is long-term dependencies.

## Kernel-methods classification

Since you already know that meaningful patterns exist in your data, you can base your classification on similarity measures. Note that this is still a classification model like the one above, but now instead of raw timepoints, you provide a similarity measure for signals (or shapelets so to speak) such as Dynamic Time Warping (DTW) and do classification using kernel methods. This still requires the window-size as one of the hyper-parameters)

## Switching models and timeseries segmentation

The way you have associated those patterns to a label, I think you can assume that your problem is an example of switching state space models, where you have $$k$$ states, each defined by a state-space model, and you switch among those states. you can assume the switching mechanism depends on the data, or the duration of the current state, which makes it interesting for your problem, since you have the challenge of long-term dependencies. switching model can learn the expected dwelling time on states, thus it will form a prior belief, not to switch too often, if it repeatedly sees rare switches. Nice thing about this model is that you can do timeseries segmentation, which marks for you your "class" regions. the learning is a bit tricky though. for that purpose, you can checkout this paper.

## conclusion

In timeseries problems, the problem definition is often very important and deciding. Sometimes you aim for an impossible problem to solve, whereas with a change of perspective, you can solve it. As an example, imagine a simple linear trend with an additive noise component. if you define your goal as "predicting the next timepoint" you might not achieve satisfying "regression" results because you're basically trying to learn an independent noise. Whereas you can instead be interested in "if the timeseries will go down or up in the next timepoint" then the expected value your label is simply the slope of the trend, which is way more doable than regression. My point is, maybe you can get better result if you think of your problem as another form of research question such as segmentation, etc.

• Hello, thanks. It's been a little while since I was thinking about DTW or Shapelets. If I remember correctly You compute distance matrix between points of the serie your're studying and one or more series of references in a database for example (I my case it would be the subpart that compose my curve like in fig.2). Then you send the result of the calculation to a model (svm, ...) ? I had also investigated image-transformation to turn time serie into an image - very interesting but until now it gave me only average results. May 10 at 12:40
• yes, a kernel method accepts the pair-wise distance matrix. for a more basic method, you can also do KNN for example, if your sample size allows for that. your "average" results might be due to lack of information in shapelets. maybe you can do some summary statistics to see how diverse your signals are, or calculate TFIDF on your shapelets. I'd start with the most basic models to check for some trivial results May 10 at 15:42
• Hi, I have indeed started by extracted on shapelet statistical and temporal features (haven't try spectral features yet) such as Mode, Skewness, signal_distance, and so on. Then I used features selection to keep only the most relevant one. I then trained a SVM model on it and achieved a pretty good précsion on test set. But once I try on a stream version with different windows size I end up with the same class prediction (class 2). I'm reading for the switching model its look very interesting. May 10 at 15:59