0
$\begingroup$

I am using LSTM to do time-series anomaly detection. The data is an hourly sensor input across multiple years (i.e. the global_active_energy attribute of the dataset from https://www.kaggle.com/uciml/electric-power-consumption-data-set). Data is cut into one-week sequences to predict the next 24 hours i.e. from the sequence of 01-01-2009 00:00 to 07-01-2009 23:00, predict the values of 08-01-2009 00:00 to 23:00.

The training data is one-week sequences preceding each day in 2008 (25-31 Dec 2007 to 24-30 Dec 2008), and the training labels are 24-hour sequences for each day in 2008.

The test data is one-week sequences from 25-31 Dec 2008 to 24-30 Dec 2009, and the test labels are 24-hour sequences for each day in 2009.

Here's the relevant code:

# tensorflow-keras
def create_model(seq_shape: Tuple, lbl_len: int) -> Model:
    inputs = Input(shape=seq_shape)
    lstm = LSTM(units=32, recurrent_initializer="glorot_uniform", 
                bias_initializer="glorot_uniform", dropout=0.2, 
                return_sequences=True)(inputs)
    lstm = LSTM(units=32, recurrent_initializer="glorot_uniform", 
                bias_initializer="glorot_uniform", dropout=0.2, 
                return_sequences=True)(lstm)
    lstm = LSTM(units=32, recurrent_initializer="glorot_uniform", 
                bias_initializer="glorot_uniform", dropout=0.2)(lstm)
    dense = Dense(units=32, activation="relu",
                  kernel_initializer="he_uniform", 
                  bias_initializer="he_uniform")(lstm)
    dropout = Dropout(rate=0.2)(dense)
    dense = Dense(units=32, activation="relu",
                  kernel_initializer="he_uniform", 
                  bias_initializer="he_uniform")(dropout)
    dropout = Dropout(rate=0.2)(dense)
    output = Dense(units=lbl_len, activation="linear", 
                   kernel_initializer="he_uniform", 
                   bias_initializer="he_uniform")(dropout)

    model = Model(inputs=inputs, outputs=output)
    model.compile(optimizer="adam", loss="mse")

    return model

# Create model
model = create_model(seq_shape=(tr_data.shape[1], tr_data.shape[2]), 
                     lbl_len=tr_lbls.shape[1])
print(model.summary())

# Start training
csv_logger = CSVLogger("./global_active_energy_lstm_log.csv")
history = model.fit(tr_data, tr_lbls, epochs=50, verbose=2, shuffle=True, 
                    callbacks=[csv_logger])

# Get predictions
tr_preds = model.predict(tr_data)
ts_preds = model.predict(ts_data)
# Sequences are rejoined into a single time-series via a function

Here's the training output:

Model: "model_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         [(None, 168, 1)]          0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 168, 32)           4352      
_________________________________________________________________
lstm_4 (LSTM)                (None, 168, 32)           8320      
_________________________________________________________________
lstm_5 (LSTM)                (None, 32)                8320      
_________________________________________________________________
dense_3 (Dense)              (None, 32)                1056      
_________________________________________________________________
dropout_2 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense_4 (Dense)              (None, 32)                1056      
_________________________________________________________________
dropout_3 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense_5 (Dense)              (None, 24)                792       
=================================================================
Total params: 23,896
Trainable params: 23,896
Non-trainable params: 0
_________________________________________________________________
None
Train on 366 samples
Epoch 1/50
366/366 - 19s - loss: 2.1201
Epoch 2/50
366/366 - 8s - loss: 1.6635
Epoch 3/50
366/366 - 9s - loss: 1.3817
Epoch 4/50
366/366 - 9s - loss: 1.1956
Epoch 5/50
366/366 - 9s - loss: 1.0298
Epoch 6/50
366/366 - 10s - loss: 0.9564
Epoch 7/50
366/366 - 9s - loss: 0.9121
Epoch 8/50
366/366 - 9s - loss: 0.8559
Epoch 9/50
366/366 - 9s - loss: 0.8743
Epoch 10/50
366/366 - 9s - loss: 0.8407
Epoch 11/50
366/366 - 9s - loss: 0.8338
Epoch 12/50
366/366 - 9s - loss: 0.8095
Epoch 13/50
366/366 - 9s - loss: 0.8032
Epoch 14/50
366/366 - 9s - loss: 0.8103
Epoch 15/50
366/366 - 9s - loss: 0.7780
Epoch 16/50
366/366 - 9s - loss: 0.7772
Epoch 17/50
366/366 - 9s - loss: 0.7489
Epoch 18/50
366/366 - 9s - loss: 0.7512
Epoch 19/50
366/366 - 9s - loss: 0.7213
Epoch 20/50
366/366 - 9s - loss: 0.7153
Epoch 21/50
366/366 - 9s - loss: 0.7198
Epoch 22/50
366/366 - 9s - loss: 0.7389
Epoch 23/50
366/366 - 9s - loss: 0.7389
Epoch 24/50
366/366 - 9s - loss: 0.7047
Epoch 25/50
366/366 - 9s - loss: 0.6984
Epoch 26/50
366/366 - 9s - loss: 0.6968
Epoch 27/50
366/366 - 10s - loss: 0.6704
Epoch 28/50
366/366 - 9s - loss: 0.7039
Epoch 29/50
366/366 - 10s - loss: 0.6944
Epoch 30/50
366/366 - 9s - loss: 0.6930
Epoch 31/50
366/366 - 9s - loss: 0.6862
Epoch 32/50
366/366 - 9s - loss: 0.6750
Epoch 33/50
366/366 - 9s - loss: 0.6669
Epoch 34/50
366/366 - 9s - loss: 0.6771
Epoch 35/50
366/366 - 9s - loss: 0.6747
Epoch 36/50
366/366 - 9s - loss: 0.6562
Epoch 37/50
366/366 - 9s - loss: 0.6503
Epoch 38/50
366/366 - 9s - loss: 0.6734
Epoch 39/50
366/366 - 9s - loss: 0.6607
Epoch 40/50
366/366 - 9s - loss: 0.6518
Epoch 41/50
366/366 - 9s - loss: 0.6567
Epoch 42/50
366/366 - 9s - loss: 0.6720
Epoch 43/50
366/366 - 10s - loss: 0.6696
Epoch 44/50
366/366 - 10s - loss: 0.6489
Epoch 45/50
366/366 - 10s - loss: 0.6449
Epoch 46/50
366/366 - 10s - loss: 0.6536
Epoch 47/50
366/366 - 9s - loss: 0.6453
Epoch 48/50
366/366 - 10s - loss: 0.6451
Epoch 49/50
366/366 - 9s - loss: 0.6537
Epoch 50/50
366/366 - 10s - loss: 0.6617

Something is obviously wrong, however, based on the charts:

enter image description here

As you can see, the predictions captured the general pattern in the actual data (e.g. training predictions for Aug 2008 flatlined like in the actual data). However, while the actual data ranged between approx. 0-6, the predictions ranged between 0-2+ in both cases.

What am I doing wrong here? How can I get the model to predict data in the correct value range of approx. 0-6?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.