# What kind of neural network should I build to classify each instance of a time series sequence?

Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify each sample with the correct class label.

    Training Sequence:              Test Sequence:

Time,   Bitrate,   Class           Time,   Bitrate    Predicted Class
0,       312,       1              0,       234      -----> 0
0.3,     319,       1              0.2,     261      -----> 0
0.5,     227,       0              0.4,     277      -----> 0
0.6,     229,       0              0.7,     301      -----> 1
0.7,     219,       0              0.8,     305      -----> 1
0.8,     341,       1              0.9,     343      -----> 1
0.9,     281,       0              1.0,     299      -----> 0
...                                ...


So, what kind of neural network should I build to classify each instance of a time series sequence?

A time series, usually, requires regular time intervals, but, from looking at your example, it seems that's not the case. You could try to use a MLP and give it as input the Time and Bitrate pairs and make it output the Class.

The activation function is what makes your neural network produce its output, i.e. activate the neurons. The loss function calculates the error of your model's predictions. This error is used by the backpropagation algorithm to adjust the model when training.

If your data can only be in one of two classes, for example 0 or 1, you have a binary classification problem. I recommend using the Sigmoid as the activation function. If you look at the Sigmoid's plot, you'll notice that, no matter the input, the output will always be within 0 and 1. You can think of this as the probability of the input being in class 0 or 1. So, for example, any input that produces a probability under 0.5 is class 0. Of course, if you want to be more precise you can simply procure the probabilities without the making it fit the labels. For example, an output of 0.7 represents that there is a 70% probability of the input belonging to class 1. Another term for this is Logistic Regression.

The Binary cross-entropy calculates the error between the model's prediction and the real class label. This loss function can be applied to many classes, but the binary version keeps it to 0 and 1.

Choosing the right activation functions and loss functions will depend on the problem you are trying to solve. Here are some useful guides on loss functions and activation functions.

Additional to the above, try to use scaling(normalize) the inputs before feeding to the network, so that the network may generalize, as it looks like bitrate value above 300 is likely to be associated with class 1. It may improve performance.

• Thanks Marcus! Do you have any useful tutorial etc. advice to understand what kind of loss functions should be used with which activations or vice-versa? Apr 29 at 16:10
• @bbasaran I updated the answer. If you are interested in a deep learning course, I would recommend Coursera's Deep Learning course. Apr 29 at 20:05
• Thanks a lot for your help, Marcus! Apr 29 at 20:22