Time Series Prediction with LSTM and Keras for Multiple Steps Ahead

In this post I will share experiment with Time Series Prediction with LSTM and Keras. LSTM neural network is used in this experiment for multiple steps ahead for stock prices data. The experiment is based on the paper [1]. The authors of this paper examine independent value prediction approach. In this approach a separate model is built for each prediction step. This approach helps to avoid error accumulation problem that we have when we use multi-stage step prediction.

LSTM Implementation

Following this approach I decided to use Long Short-Term Memory network or LSTM network. This is a type of recurrent neural network used in deep learning. LSTMs have been used to advance the state-of the-art
for many difficult problems. [2]

In this experiment I selected the number of steps to predict ahead = 5 and built 5 LSTM models with Keras in python. It does not mean that the code got 5 times more. I used the same code for all models but only recalculated changing parameters for input data to neural network within for loop.

The data were obtained from stock prices from Internet. LSTM network was constructed as following:

fit1 = Sequential ()
fit1.add (LSTM (output_dim =len(cols), activation = 'tanh', inner_activation = 'hard_sigmoid' , input_shape =(len(cols), 1), W_regularizer=l2(0.01) ))
fit1.add(Dropout(0.2))
fit1.add (Dense (output_dim =1, activation = 'linear', W_regularizer=l1(0.01)))
fit1.compile (loss ="mean_squared_error" , optimizer = "adam")   
fit1.fit (x_train, y_train, batch_size =16, nb_epoch =25, shuffle = False)

The full python source code for time series prediction with LSTM in python is shown here

Experiment Results

The LSTM neural network was running with :
Training MSE 0.026
Testing MSE 0.066
Accuracy of prediction 87.2%
The results are showing that there are still some place for improvements. This however will serve as baseline for further improved models.

References
1. Multistep-ahead Time Series Prediction
2. LSTM: A Search Space Odyssey

Leave a Comment