{"id":1783,"date":"2018-01-19T01:41:50","date_gmt":"2018-01-19T01:41:50","guid":{"rendered":"http:\/\/intelligentonlinetools.com\/blog\/?p=1783"},"modified":"2018-03-02T01:36:43","modified_gmt":"2018-03-02T01:36:43","slug":"machine-learning-stock-prediction-lstm-keras","status":"publish","type":"post","link":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","title":{"rendered":"Machine Learning Stock Prediction with LSTM and Keras"},"content":{"rendered":"

In this post I will share experiments on machine learning stock prediction with LSTM<\/b> and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest model – prediction for one step ahead. So I put here the example of prediction of stock price data time series for one next step.<\/p>\n

Preparing Data for Neural Network Prediction<\/h3>\n

Our first task is feed the data into LSTM. Our data is stock price<\/b> data time series that were downloaded from the web.
\nOur interest is closed price for the next day so target variable will be closed price<\/b> but shifted to left (back) by one step.<\/p>\n

Figure below is showing how do we prepare X, Y data.
\nPlease note that I put X Y horizontally for convenience but still calling X, Y as columns.
\nAlso on this figure only one variable (closed price) is showed for X but in the code it is actually more than one variable (it is closed price, open price, volume and high price). <\/p>\n

\"LSTM
LSTM data input<\/figcaption><\/figure>\n

Now we have the data that we can feed for LSTM neural network prediction. Before doing this we need also do the following things:<\/p>\n

1. Decide how many data we want to use. For example if we have data for 10 years but want use only last 200 rows of data we would need specify start point because the most recent data will be in the end.<\/p>\n

\r\n#how many data we will use \r\n# (should not be more than dataset length )\r\ndata_to_use= 100\r\n\r\n# number of training data\r\n# should be less than data_to_use\r\ntrain_end =70\r\n\r\n\r\ntotal_data=len(data_csv)\r\n\r\n#most recent data is in the end \r\n#so need offset\r\nstart=total_data - data_to_use\r\n<\/pre>\n
2. Rescale (normalize) data as below. Here feature_range  is tuple (min, max), default=(0, 1)  is desired range of transformed data. [1]<\/p>\n
\r\nscaler_x = preprocessing.MinMaxScaler ( feature_range =( -1, 1))\r\nx = np. array (x).reshape ((len( x) ,len(cols)))\r\nx = scaler_x.fit_transform (x)\r\n\r\n\r\nscaler_y = preprocessing. MinMaxScaler ( feature_range =( -1, 1))\r\ny = np.array (y).reshape ((len( y), 1))\r\ny = scaler_y.fit_transform (y)\r\n<\/pre>\n
3. Divide data into training and testing set. <\/p>\n
\r\nx_train = x [0: train_end,]\r\nx_test = x[ train_end +1:len(x),]    \r\ny_train = y [0: train_end] \r\ny_test = y[ train_end +1:len(y)]  \r\n<\/pre>\n
Building and Running LSTM<\/h3>\n
Now we need to define our LSTM neural network layers ,  parameters and run neural network to see how it works.<\/p>\n
\r\nfit1 = Sequential ()\r\nfit1.add (LSTM (  1000 , activation = 'tanh', inner_activation = 'hard_sigmoid' , input_shape =(len(cols), 1) ))\r\nfit1.add(Dropout(0.2))\r\nfit1.add (Dense (output_dim =1, activation = 'linear'))\r\n\r\nfit1.compile (loss ="mean_squared_error" , optimizer = "adam")   \r\nfit1.fit (x_train, y_train, batch_size =16, nb_epoch =25, shuffle = False)\r\n<\/pre>\n
The first run was not good – prediction was way below actual.<\/p>\n
However by changing the number of neurons in LSTM neural network prediction code, it was possible to improve MSE and get predictions much closer to actual data.  Below are screenshots of plots of prediction and actual data for LSTM with 5, 60 and 1000 neurons. Note that the plot is showing only testing data. Training data is not shown on the plots. Also you can find below the data for MSE and number of neurons. As the number neorons in LSTM layer is changing there were 2 minimums (one around 100 neurons and another around 1000 neurons)<\/p>\n
\"Results
Results of prediction on LSTM with 5 neurons<\/figcaption><\/figure>\n
\"Forecasting
Results of prediction on LSTM with 60 neurons<\/figcaption><\/figure>\n
\"Forecasting
Results of prediction on LSTM with 1000 neurons<\/figcaption><\/figure>\n
LSTM 5
\n in train MSE =  0.0475
\n in test MSE =  0.2714831660102521<\/p>\n
LSTM 60
\n in train MSE =  0.0127
\n in test MSE =  0.02227417068206705 <\/p>\n
LSTM 100
\n in train MSE =  0.0126
\n in test MSE =  0.018672733913263073 <\/p>\n
LSTM 200
\n in train MSE =  0.0133
\n in test MSE =  0.020082660237026824<\/p>\n
LSTM 900
\n in train MSE =  0.0103
\n in test MSE =  0.015546535929778267 <\/p>\n
LSTM 1000
\n in train MSE =  0.0104
\n in test MSE =  0.015037054958075455  <\/p>\n
LSTM 1100
\n in train MSE =  0.0113
\n in test MSE =  0.016363980411369994 <\/p>\n
Conclusion<\/h3>\n
The final LSTM was running with testing MSE 0.015 and accuracy 97%. This was obtained by changing number of neurons in LSTM. This example will serve as baseline for further possible improvements on machine learning stock prediction with LSTM. If you have any tips or anything else to add, please leave a comment the comment box. The full source code for LSTM neural network prediction can be found here<\/a><\/p>\n
References<\/strong>
\n1. sklearn.preprocessing.MinMaxScaler<\/a>
\n2. Keras documentation – RNN Layers<\/a>
\n3. How to Reshape Input Data for Long Short-Term Memory Networks in Keras<\/a>
\n4. Keras FAQ: Frequently Asked Keras Questions<\/a>
\n5. Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras<\/a>
\n6.Deep Time Series Forecasting with Python: An Intuitive Introduction to Deep Learning for Applied Time Series Modeling<\/a>
\n7. Machine Learning Stock Prediction with LSTM and Keras – Python Source Code<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
In this post I will share experiments on machine learning stock prediction with LSTM and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest model – prediction for one … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[9,10,3],"tags":[42,49,48,18,51,52,16],"jetpack_publicize_connections":[],"yoast_head":"\nMachine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications\" \/>\n<meta property=\"og:description\" content=\"In this post I will share experiments on machine learning stock prediction with LSTM and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest model – prediction for one ... Read more\" \/>\n<meta property=\"og:url\" content=\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/\" \/>\n<meta property=\"og:site_name\" content=\"Machine Learning Applications\" \/>\n<meta property=\"article:published_time\" content=\"2018-01-19T01:41:50+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-03-02T01:36:43+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/01\/lstm-data-input-2.png\" \/>\n<meta name=\"author\" content=\"owygs156\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"owygs156\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/\",\"name\":\"Machine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications\",\"isPartOf\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\"},\"datePublished\":\"2018-01-19T01:41:50+00:00\",\"dateModified\":\"2018-03-02T01:36:43+00:00\",\"author\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\"},\"breadcrumb\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/intelligentonlinetools.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning Stock Prediction with LSTM and Keras\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/\",\"name\":\"Machine Learning Applications\",\"description\":\"Artificial intelligence, data mining and machine learning for building web based tools and services.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\",\"name\":\"owygs156\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"caption\":\"owygs156\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications","og_description":"In this post I will share experiments on machine learning stock prediction with LSTM and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest model – prediction for one ... Read more","og_url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","og_site_name":"Machine Learning Applications","article_published_time":"2018-01-19T01:41:50+00:00","article_modified_time":"2018-03-02T01:36:43+00:00","og_image":[{"url":"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/01\/lstm-data-input-2.png"}],"author":"owygs156","twitter_card":"summary_large_image","twitter_misc":{"Written by":"owygs156","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","name":"Machine Learning Stock Prediction with LSTM and Keras - Machine Learning Applications","isPartOf":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#website"},"datePublished":"2018-01-19T01:41:50+00:00","dateModified":"2018-03-02T01:36:43+00:00","author":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478"},"breadcrumb":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/intelligentonlinetools.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Machine Learning Stock Prediction with LSTM and Keras"}]},{"@type":"WebSite","@id":"http:\/\/intelligentonlinetools.com\/blog\/#website","url":"http:\/\/intelligentonlinetools.com\/blog\/","name":"Machine Learning Applications","description":"Artificial intelligence, data mining and machine learning for building web based tools and services.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478","name":"owygs156","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","caption":"owygs156"}}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7h1IJ-sL","jetpack-related-posts":[{"id":1754,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/02\/27\/time-series-prediction-lstm-keras\/","url_meta":{"origin":1783,"position":0},"title":"Time Series Prediction with LSTM and Keras for Multiple Steps Ahead","date":"February 27, 2018","format":false,"excerpt":"In this post I will share experiment with Time Series Prediction with LSTM and Keras. LSTM neural network is used in this experiment for multiple steps ahead for stock prices data. The experiment is based on the paper [1]. The authors of the paper examine independent value prediction approach. With\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"Multiple step prediction with separate neural networks","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/02\/multiple-step-prediction.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1798,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/20\/machine-learning-stock-prediction-lstm-keras-python-source-code\/","url_meta":{"origin":1783,"position":1},"title":"Machine Learning Stock Prediction with LSTM and Keras – Python Source Code","date":"January 20, 2018","format":false,"excerpt":"Python Source Code for Machine Learning Stock Prediction with LSTM and Keras - Python Source Code with LSTM and Keras Below is the code for machine learning stock prediction with LSTM neural network. References 1. Machine Learning Stock Prediction with LSTM and Keras - Python Source Code with LSTM and\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1974,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/04\/03\/machine-learning-stock-market-prediction-lstm-keras\/","url_meta":{"origin":1783,"position":2},"title":"Machine Learning Stock Market Prediction with LSTM Keras","date":"April 3, 2018","format":false,"excerpt":"In the previous posts [1,2] I created script for machine learning stock market price on next day prediction. But it was pointed by readers that in stock market prediction, it is more important to know the trend: will the stock go up or down. So I updated the script to\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"Stock Data Prices Prediction with LSTM","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/04\/timeseries_differenced_and_inverted_back.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1995,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/04\/17\/lstm-neural-network-training-techniques-tuning-hyperparameters\/","url_meta":{"origin":1783,"position":3},"title":"LSTM Neural Network Training – Few Useful Techniques for Tuning Hyperparameters and Saving Time","date":"April 17, 2018","format":false,"excerpt":"Neural networks are among the most widely used machine learning techniques.[1] But neural network training and tuning multiple hyper-parameters takes time. I was recently building LSTM neural network for prediction for this post Machine Learning Stock Market Prediction with LSTM Keras and I learned some tricks that can save time.\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"LSTM NN Training Value Loss Charts with High Number and Adjusted","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/04\/LSTM-NN-Training-Value-Loss-with-High-Number-and-adjusted-e1524097124417.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1178,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/05\/14\/time-series-prediction-with-convolutional-neural-networks\/","url_meta":{"origin":1783,"position":4},"title":"Forecasting Time Series Data with Convolutional Neural Networks","date":"May 14, 2017","format":false,"excerpt":"Convolutional neural networks(CNN) is increasingly important concept in computer science and finds more and more applications in different fields. Many posts on the web are about applying convolutional neural networks for image classification as CNN is very useful type of neural networks for image classification. But convolutional neural networks can\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/05\/time-series-LSTM-300x164.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1628,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/12\/17\/time-series-analysis-python-prophet\/","url_meta":{"origin":1783,"position":5},"title":"Time Series Analysis with Python and Prophet","date":"December 17, 2017","format":false,"excerpt":"Recently Facebook released Prophet - open source software tool for forecasting time series data. Facebook team have implemented in Prophet two trend models that can cover many applications: a saturating growth model, and a piecewise linear model. [4] With growth model Prophet can be used for prediction growth\/decay - for\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/12\/time-series-analysis-python-300x180.png?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1783"}],"collection":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/comments?post=1783"}],"version-history":[{"count":24,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1783\/revisions"}],"predecessor-version":[{"id":1959,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1783\/revisions\/1959"}],"wp:attachment":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/media?parent=1783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/categories?post=1783"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/tags?post=1783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}