0
$\begingroup$

I have the following code snippet which takes in a single column of value i.e. 1 feature. How do I modify the LSTM model such that it accepts 3 features?

import numpy as np
from keras.models import Sequential
from keras.layers import LSTM, Input, Dropout
from keras.layers import Dense
from keras.layers import RepeatVector
from keras.layers import TimeDistributed
import pandas as pd
from matplotlib import pyplot as plt
from sklearn.preprocessing import MinMaxScaler, StandardScaler
from keras.models import Model
import seaborn as sns    

dataframe = pd.read_csv('GE.csv')
dataframe.head()

df = dataframe[['Date', 'EnergyInWatts']]
df['Date'] = pd.to_datetime(df['Date'])
sns.lineplot(x=df['Date'], y=df['EnergyInWatts'])

#train, test = df.loc[df['Date'] <= '2003-12-31'], df.loc[df['Date'] > '2003-12-31']
train = df.loc[df['Date'] <= '2003-12-31']
test = df.loc[df['Date'] > '2003-12-31']

scaler = StandardScaler()

scaler = scaler.fit(train[['EnergyInWatts']])

train['EnergyInWatts'] = scaler.transform(train[['EnergyInWatts']])
test['EnergyInWatts'] = scaler.transform(test[['EnergyInWatts']])

seq_size = 30 


def to_sequences(x, y, seq_size=1):
    x_values = []
    y_values = []

    for i in range(len(x)-seq_size):
        #print(i)
        x_values.append(x.iloc[i:(i+seq_size)].values)
        y_values.append(y.iloc[i+seq_size])
        
    return np.array(x_values), np.array(y_values)

trainX, trainY = to_sequences(train[['EnergyInWatts']], train['EnergyInWatts'], seq_size)
testX, testY = to_sequences(test[['EnergyInWatts']], test['EnergyInWatts'], seq_size)


model = Sequential()
model.add(LSTM(128, input_shape=(trainX.shape[1], trainX.shape[2])))
model.add(Dropout(rate=0.2))
model.add(RepeatVector(trainX.shape[1]))
model.add(LSTM(128, return_sequences=True))
model.add(Dropout(rate=0.2))
model.add(TimeDistributed(Dense(trainX.shape[2])))
model.compile(optimizer='adam', loss='mae')
model.summary()

history = model.fit(trainX, trainY, epochs=10, batch_size=32, validation_split=0.1, verbose=1)
```
$\endgroup$
0
$\begingroup$

If each of your three features is a scalar then my first attempt would be to combine them into a vector for each step in the sequence. So instead of LSTM(128, input_shape=(30,1)) for a length-30 univariate sequence you would say LSTM(128, input_shape=(30,3)) for a multivariate (3) sequence. Similarly your output would become TimeDistributed(Dense(3, activation='linear')).

If your input features are each a vector then you should consider switching from the sequential API to the functional API. You would need three separate Input objects followed by a Concatenate layer before the LSTM layer. You would have three Dense layers to be your corresponding outputs. I tried adapting your code:

in0 = Input(shape=(trainX0.shape[1], trainX0.shape[2]))
in1 = Input(shape=(trainX1.shape[1], trainX1.shape[2]))
in2 = Input(shape=(trainX2.shape[1], trainX2.shape[2]))
concat = Concatenate()([in0, in1, in2])
lstm_enc = LSTM(128)(concat)
lstm_enc_drop = Dropout(rate=0.2)(lstm_enc)
rep = RepeatVector(trainX0.shape[1])(lstm_enc_drop)
lstm_dec = LSTM(128, return_sequences=True)(rep)
lstm_dec_drop = Dropout(rate=0.2)(lstm_dec)
out0 = TimeDistributed(Dense(trainX0.shape[2], activation='linear'))(lstm_dec_drop)
out1 = TimeDistributed(Dense(trainX1.shape[2], activation='linear'))(lstm_dec_drop)
out2 = TimeDistributed(Dense(trainX2.shape[2], activation='linear'))(lstm_dec_drop)
model = Model(inputs=[in0, in1, in2], outputs=[out0, out1, out2])
model.compile(optimizer='adam', loss='mae')
model.summary()
$\endgroup$
2
  • $\begingroup$ My 3 inputs would be 3 columns of time series values e.g. 0.31314 0.15891 0.41713. Would the output of concatenate be a single vector i.e. [0.31314 0.15891 0.41713] or a single value? $\endgroup$
    – Angelina
    Mar 18 at 13:51
  • $\begingroup$ Concatenate would produce a 3-dimension vector. The output tensor from the Concatenate layer will be of form (batch, steps, features) so your model summary should report an output tensor shape of (None, 30, 3) given your 30 time steps and 3 features. I think that using multiple Inputs with Concatenate would be functionally equivalent to changing your input shape to (30, 3) and providing it with the 30x3 input data array. That way you wouldn't need to worry about changing from Sequential to Functional. $\endgroup$ Mar 18 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.