Fibonacci Stock Trading – Using Fibonacci Retracement for Stock Market Prediction

As stated on allstarcharts.com by expert with more than 10 years, Fibonacci Analysis is one of the most valuable and easy to use tools for stock market technical analysis. And Fibonacci tools can be applied to longer-term as well as to short-term. [3]

In this post we will take a look how Fibonacci numbers can help to stock market analysis. For this we will use different daily stock prices data charts with added Fibonacci lines.

Fibonacci numbers
Just for the references – The Fibonacci numbers (or Fibonacci sequence), are numbers that after the first two are the sum of the two preceding ones[1] : 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55 …

According to “Magic of Fibonacci Sequence in Prediction of Stock Behavior” [7] Fibonacci series are widely used in financial market to predict the resistance and support levels through Fibonacci retracement. In this method major peak and trough are identified, then it is followed by dividing the vertical distance into 23.6%, 38.2%, 50%, 61.8% and 100%. These percentage numbers (except 50%) are obtained by dividing element in Fibonacci sequence by its successors for example 13/55=0.236. [2]
Ratio of two successive numbers of Fibonacci sequence is approximately 1.618034, so if we multiply 23.6 by 1.618034 we will get next level number 38.2.

5 Fibonacci Chart Examples

Now let’s look at charts with added Fibonacci lines (23.6%, 38.2%, 50%, 61.8% and 100%). The charts are created using our online tool: Stock Data Prediction with Fibonacci Sequence [10] In addition to data, the required input to the tool is the lowest and highest the stock market price values (which will correspond to 0% and 100% levels)

Below are 5 daily the stock market price charts for different stock tickers. Fibonacci line numbers are shown on the right side, and stock ticker is on the top of chart.

Botz stock data chart with Fibonacci lines
Botz stock data chart with Fibonacci lines

Here we can see Fibonacci line at 61.8 is a support line

Botz stock data chart with Fibonacci lines
Botz stock data chart with Fibonacci lines

This chart is for the same symbol but for smaller time frame. 23.6 line is support and then resistance line

GE  stock data chart with Fibonacci lines
GE stock data chart with Fibonacci lines

Here are Fibonacci Retracement lines at 23.6 and 61.8 line up well with support and resistance areas

GE stock data chart with Fibonacci lines
GE stock data chart with Fibonacci lines

Fibonacci Retracement 61.8 is support line

T stock data chart with Fibonacci lines
T stock data chart with Fibonacci lines

Fibonacci Retracement lines such as 23.6, 38.2, 61.8 line up well with support and resistance areas

Conclusion

After reviewing above charts we can say that Fibonacci Retracement can indicate potential support and resistance levels. The trend often is changing in such areas. So Fibonacci retracement can be used for stock market prediction.

Do you use Fibonacci tools? Feel free to put in the comments how do you apply Fibonacci retracements in stock trading. Do you find Fibonacci tools helpful or not? As always any feedback is welcome.

References
1.Fibonacci number
2.How do I use the Fibonacci series in analysis of stocks? What are some examples with Buy and Sell signals?
3.How I Use Fibonacci Analysis To Make Money In The Market
4.Technicals with ETMarkets: How to use Fibonacci to identify buying levels
5.Fibonacci Retracements
6.What is the Fibonacci sequence and where does it derive from? Why do we find it everywhere around us, from nature to art?
7.Magic of Fibonacci Sequence in Prediction of Stock Behavior
8. How to Use Fibonacci Retracement Levels in Day Trading
9. Fibonacci Retracement Trading Strategy In Python
10. Online ML algorithms – this was used to get charts for this post



Time Series Prediction with LSTM and Keras for Multiple Steps Ahead

In this post I will share experiment with Time Series Prediction with LSTM and Keras. LSTM neural network is used in this experiment for multiple steps ahead for stock prices data. The experiment is based on the paper [1]. The authors of the paper examine independent value prediction approach. With this approach a separate model is built for each prediction step. This approach helps to avoid error accumulation problem that we have when we use multi-stage step prediction.

LSTM Implementation

Following this approach I decided to use Long Short-Term Memory network or LSTM network for daily data stock price prediction. LSTM is a type of recurrent neural network used in deep learning. LSTMs have been used to advance the state-of the-art for many difficult problems. [2]

For this time series prediction I selected the number of steps to predict ahead = 3 and built 3 LSTM models with Keras in python. For each model I used different variable (fit0, fit1, fit2) to avoid any “memory leakage” between models.
The model initialization code is the same for all 3 models except changing parameters (number of neurons in LSTM layer)
The architecture of the system is shown on the fig below.

Multiple step prediction with separate neural networks
Multiple step prediction with separate neural networks

Here we have 3 LSTM models that are getting same X input data but different target Y data. The target data is shifted by number of steps. If model is forecasting the data stock price for day 2 then Y is shifted by 2 elements.
This happens in the following line when i=1:

yt_ = yt.shift (-i - 1  ) 

The data were obtained from stock prices from Internet.

The number of unit was obtained by running several variations and chosen based on MSE as following:

   
    if i==0:
        units=20
        batch_size=1
    if i==1:
        units=15
        batch_size=1
    if i==2:
         units=80
         batch_size=1

If you want run more than 3 steps / models you will need to add parameters to the above code. Additionally you will need add model initialization code shown below.

Each LSTM network was constructed as following:


 if i == 0 :
          fit0 = Sequential ()
          fit0.add (LSTM (  units , activation = 'tanh', inner_activation = 'hard_sigmoid' , input_shape =(len(cols), 1) ))
          fit0.add(Dropout(0.2))
          fit0.add (Dense (output_dim =1, activation = 'linear'))
          fit0.compile (loss ="mean_squared_error" , optimizer = "adam")  
   
          fit0.fit (x_train, y_train, batch_size =batch_size, nb_epoch =25, shuffle = False)
          train_mse[i] = fit0.evaluate (x_train, y_train, batch_size =batch_size)
          test_mse[i] = fit0.evaluate (x_test, y_test, batch_size =batch_size)
          pred = fit0.predict (x_test) 
          pred = scaler_y.inverse_transform (np. array (pred). reshape ((len( pred), 1)))
             # below is just fo i == 0
          for j in range (len(pred)) :
                   prediction_data[j] = pred[j] 

For each model the code is saving last forecasted number.
Additionally at step i=0 predicted data is saved for comparison with actual data:

prediction_data = np.asarray(prediction_data)
prediction_data = prediction_data.ravel()

# shift back by one step
for j in range (len(prediction_data) - 1 ):
    prediction_data[len(prediction_data) - j - 1  ] =  prediction_data[len(prediction_data) - 1 - j - 1]

# combine prediction data from first model and last predicted data from each model
prediction_data = np.append(prediction_data, forecast)

The full python source code for time series prediction with LSTM in python is shown here

Data can be found here

Experiment Results

The LSTM neural network was running with the following performance:

train_mse
[0.01846262458218137, 0.009637593373373323, 0.0018845983509225203]
test_mse
[0.01648362025879952, 0.026161141224167357, 0.01774421124347165]

Below is the graph of actual data vs data testing data, including last 3 stock data prices from each model.

Multiple step prediction actual data vs predictions
Multiple step prediction – actual data vs predictions

Accuracy of prediction 98% calculated for last 3 data stock prices (one from each model).

The experiment confirmed that using models (one model for each step) in multistep-ahead time series prediction has advantages. With this method we can adjust parameters of needed LSTM for each step. For example, number of neurons for i=2 was modified to decrease prediction error for this step. And it did not affect predictions for other steps. This is one of machine learning techniques for stock prediction that is described in [1]

References
1. Multistep-ahead Time Series Prediction
2. LSTM: A Search Space Odyssey
3. Deep Time Series Forecasting with Python: An Intuitive Introduction to Deep Learning for Applied Time Series Modeling



Prediction on Next Stock Market Correction

On Feb. 6, 2018, the stock market officially entered “correction” territory. A stock market correction is defined as a drop of at least 10% or more for an index or stock from its recent high. [1] During one week the stock data prices (closed price) were decreasing for many stocks. Are there any signals that can be used to predict next stock market correction?

I pulled historical data from 20 stocks selected randomly and then created python program that counts how many stocks (closed price) were decreased, increased or did not change for each day (comparing with previous day). The numbers then converted into percentage. So if all 20 stock closed prices decreased at some day it would be 100%. For now I was just looking at % of decreased stocks per day. Below is the graph for decreasing stocks. Highlighted zone A is when we many decreasing stocks during the correction.

Number of decreasing stocks per day in %
Number of decreasing stocks per day in %

Observations

I did not find good strong signal to predict market correction but probably more analysis needed. However before this correction there was some increasing trend for number of stocks that close at lower prices. This is shown below. On this graph the trend line can be viewed as indicator of stock market direction.

Number-of-decreasing-stocks-per-day-before-correction
Number of decreasing stocks per day before correction in %

Python Source Code to download Stock Data

Here is the script that was used to download data:

from pandas_datareader import data as pdr 
import time   

# put below actual symbols as many as you need
symbols=['XXX','XXX', 'XXX', ...... 'XXX']
 

def get_data (symbol):
    
    data = pdr.get_data_google(symbol,'1970-01-01','2018-02-19')
    path="C:\\Users\\stocks\\"
    data.to_csv( path + symbol+".csv")
 
    return data


    
for symbol in symbols:
        get_data(symbol)    
        time.sleep(7)

Script for Stock Data Analysis

Here is the program that takes downloaded data and counts the number of decreased/increased/same stocks per day. The results are saved in the file and also plotted. Plots are shown after source code below.

And here is the link to the data output from the below program.

# -*- coding: utf-8 -*-

import os

path="C:\\Users\\stocks\\"
from datetime import datetime
import pandas as pd
import numpy as np

def days_between(d1, d2):
    d1 = datetime.strptime(d1, "%Y-%m-%d")
    d2 = datetime.strptime(d2, "%Y-%m-%d")
    print (d1)
    print (d2)
    return abs((d2 - d1).days)


i=10000   # index to replace date
j=20      # index for stock symbols
k=5       # other attributes
data = np.zeros((i,j,k))           
symbols=[]           

count=0        

# get index of previous trade day
# because there is no trades on weekend or holidays
# need to calculate prvious trade day index instead
# of just subracting 1
def get_previous_ind(row_ind, col_count ):
    
    k=1
    print (str(row_ind) + "   " + str(col_count))
    while True:
        if  data[row_ind-k][col_count][0] == 1:
            return row_ind-k
        else:
            k=k+1
    
        if k > 1000 :
            print ("ERROR: PREVIOUS ROW IS NOT FOUND")
            return -1

dates=["" for i in range(10000) ]          
# read the entries
listOfEntries = os.scandir(path)
for entry in  listOfEntries: 
        
     if entry.is_file():
            print(entry.name)
            stock_data = pd.read_csv (str(path) + str(entry.name))
            symbols.append (entry.name)

                     
            for index, row in stock_data.iterrows():
                 ind=days_between(row['Date'], "2002-01-01") 
                
                 dates[ind] = row['Date']
                 data[ind][count][0] = 1
                 data[ind][count][1] = row['Close']
                 
                 if (index > 1):
                     print(entry.name)
                     prev_ind=get_previous_ind(ind, count)
                     delta= 1000*(row['Close'] - data[prev_ind][count][1])
                     change=0
                     if (delta > 0) :
                          change = 1
                     if (delta < 0) :
                          change = -1
                     data[ind][count][3] = change  
                     data[ind][count][4] = 1   
                
                 
            count=count+1                      

    
upchange=[0 for i in range(10000)]
downchange=[0 for i in range(10000)]
zerochange=[0 for i in range(10000)]
datesnew = ["" for i in range(10000) ]
icount=0
for i in range(10000):
       total=0 
       for j in range (count):
           
           if data[i][j][4] == 1 :
               datesnew[icount]=dates[i]
               total=total+1
               if (data[i][j][3] ==0):
                       zerochange[icount]=zerochange[icount]+1
               if (data[i][j][3] ==1):
                       upchange[icount]=upchange[icount] + 1
               if (data[i][j][3] == - 1):
                       downchange[icount]=downchange[icount] + 1
         
           
       if (total != 0) :
               upchange[icount]=100* upchange[icount] / total
               downchange[icount]=100* downchange[icount] / total
               zerochange[icount]=100* zerochange[icount] / total    
               print (str(upchange[icount]) + "  " +  str(downchange[icount]) + "  " + str(zerochange[icount]))
               icount=icount+1

            

df=pd.DataFrame({'Date':datesnew, 'upchange':upchange, 'downchange':downchange, 'zerochange':zerochange })
print (df)
df.to_csv("changes.csv", encoding='utf-8', index=False)               
            

import matplotlib.pyplot as plt

downchange=downchange[icount-200:icount]
upchange=upchange[icount-200:icount]
zerochange=zerochange[icount-200:icount]


# Two subplots, the axes array is 1-d
f, axarr = plt.subplots(3, sharex=True)
axarr[0].plot(downchange)
axarr[0].set_title('downchange')
axarr[1].plot(upchange)
axarr[1].set_title('upchange')
axarr[2].plot(zerochange)
axarr[2].set_title('zerochange')
plt.show()
Number of stocks increasing decreasing same in %
Number of stocks increasing decreasing same in %

References
1. 6 Things You Should Know About a Stock Market Correction
2. How to Predict the Eventual Stock Market Correction Before Anyone Else
3. 4 Ways To Predict Market Performance



Machine Learning Stock Prediction with LSTM and Keras – Python Source Code

Python Source Code for Machine Learning Stock Prediction with LSTM and Keras – Python Source Code with LSTM and Keras Below is the code for machine learning stock prediction with LSTM neural network.

# -*- coding: utf-8 -*-

import numpy as np
import pandas as pd
from sklearn import preprocessing
import matplotlib.pyplot as plt


fname="C:\\Users\\stock_data.csv"
data_csv = pd.read_csv (fname)

#how many data we will use 
# (should not be more than dataset length )
data_to_use= 100

# number of training data
# should be less than data_to_use
train_end =70


total_data=len(data_csv)

#most recent data is in the end 
#so need offset
start=total_data - data_to_use


#currently doing prediction only for 1 step ahead
steps_to_predict =1

 
yt = data_csv.iloc [start:total_data ,4]    #Close price
yt1 = data_csv.iloc [start:total_data ,1]   #Open
yt2 = data_csv.iloc [start:total_data ,2]   #High
yt3 = data_csv.iloc [start:total_data ,3]   #Low
vt = data_csv.iloc [start:total_data ,6]    # volume


print ("yt head :")
print (yt.head())

yt_ = yt.shift (-1)
    
data = pd.concat ([yt, yt_, vt, yt1, yt2, yt3], axis =1)
data. columns = ['yt', 'yt_', 'vt', 'yt1', 'yt2', 'yt3']
    
data = data.dropna()
    
print (data)
    
# target variable - closed price
# after shifting
y = data ['yt_']

       
#       closed,  volume,   open,  high,   low    
cols =['yt',    'vt',  'yt1', 'yt2', 'yt3']
x = data [cols]

  
   
scaler_x = preprocessing.MinMaxScaler ( feature_range =( -1, 1))
x = np. array (x).reshape ((len( x) ,len(cols)))
x = scaler_x.fit_transform (x)

   
scaler_y = preprocessing. MinMaxScaler ( feature_range =( -1, 1))
y = np.array (y).reshape ((len( y), 1))
y = scaler_y.fit_transform (y)

    
x_train = x [0: train_end,]
x_test = x[ train_end +1:len(x),]    
y_train = y [0: train_end] 
y_test = y[ train_end +1:len(y)]  
x_train = x_train.reshape (x_train. shape + (1,)) 
x_test = x_test.reshape (x_test. shape + (1,))

    
    

from keras.models import Sequential
from keras.layers.core import Dense
from keras.layers.recurrent import LSTM
from keras.layers import  Dropout


seed =2016 
np.random.seed (seed)
fit1 = Sequential ()
fit1.add (LSTM (  1000 , activation = 'tanh', inner_activation = 'hard_sigmoid' , input_shape =(len(cols), 1) ))
fit1.add(Dropout(0.2))
fit1.add (Dense (output_dim =1, activation = 'linear'))

fit1.compile (loss ="mean_squared_error" , optimizer = "adam")   
fit1.fit (x_train, y_train, batch_size =16, nb_epoch =25, shuffle = False)

print (fit1.summary())

score_train = fit1.evaluate (x_train, y_train, batch_size =1)
score_test = fit1.evaluate (x_test, y_test, batch_size =1)
print (" in train MSE = ", round( score_train ,4)) 
print (" in test MSE = ", score_test )

   
pred1 = fit1.predict (x_test) 
pred1 = scaler_y.inverse_transform (np. array (pred1). reshape ((len( pred1), 1)))
    
 

 
prediction_data = pred1[-1]     
   

fit1.summary()
print ("Inputs: {}".format(fit1.input_shape))
print ("Outputs: {}".format(fit1.output_shape))
print ("Actual input: {}".format(x_test.shape))
print ("Actual output: {}".format(y_test.shape))
  

print ("prediction data:")
print (prediction_data)


print ("actual data")
x_test = scaler_x.inverse_transform (np. array (x_test). reshape ((len( x_test), len(cols))))
print (x_test)


plt.plot(pred1, label="predictions")


y_test = scaler_y.inverse_transform (np. array (y_test). reshape ((len( y_test), 1)))
plt.plot( [row[0] for row in y_test], label="actual")

plt.legend(loc='upper center', bbox_to_anchor=(0.5, -0.05),
          fancybox=True, shadow=True, ncol=2)

import matplotlib.ticker as mtick
fmt = '$%.0f'
tick = mtick.FormatStrFormatter(fmt)

ax = plt.axes()
ax.yaxis.set_major_formatter(tick)


plt.show()

References
1. Machine Learning Stock Prediction with LSTM and Keras – Python Source Code with LSTM and Keras



Machine Learning Stock Prediction with LSTM and Keras

In this post I will share experiments on machine learning stock prediction with LSTM and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest model – prediction for one step ahead. So I put here the example of prediction of stock price data time series for one next step.

Preparing Data for Neural Network Prediction

Our first task is feed the data into LSTM. Our data is stock price data time series that were downloaded from the web.
Our interest is closed price for the next day so target variable will be closed price but shifted to left (back) by one step.

Figure below is showing how do we prepare X, Y data.
Please note that I put X Y horizontally for convenience but still calling X, Y as columns.
Also on this figure only one variable (closed price) is showed for X but in the code it is actually more than one variable (it is closed price, open price, volume and high price).

LSTM data input
LSTM data input

Now we have the data that we can feed for LSTM neural network prediction. Before doing this we need also do the following things:

1. Decide how many data we want to use. For example if we have data for 10 years but want use only last 200 rows of data we would need specify start point because the most recent data will be in the end.

#how many data we will use 
# (should not be more than dataset length )
data_to_use= 100

# number of training data
# should be less than data_to_use
train_end =70


total_data=len(data_csv)

#most recent data is in the end 
#so need offset
start=total_data - data_to_use

2. Rescale (normalize) data as below. Here feature_range is tuple (min, max), default=(0, 1) is desired range of transformed data. [1]

scaler_x = preprocessing.MinMaxScaler ( feature_range =( -1, 1))
x = np. array (x).reshape ((len( x) ,len(cols)))
x = scaler_x.fit_transform (x)


scaler_y = preprocessing. MinMaxScaler ( feature_range =( -1, 1))
y = np.array (y).reshape ((len( y), 1))
y = scaler_y.fit_transform (y)

3. Divide data into training and testing set.

x_train = x [0: train_end,]
x_test = x[ train_end +1:len(x),]    
y_train = y [0: train_end] 
y_test = y[ train_end +1:len(y)]  

Building and Running LSTM

Now we need to define our LSTM neural network layers , parameters and run neural network to see how it works.

fit1 = Sequential ()
fit1.add (LSTM (  1000 , activation = 'tanh', inner_activation = 'hard_sigmoid' , input_shape =(len(cols), 1) ))
fit1.add(Dropout(0.2))
fit1.add (Dense (output_dim =1, activation = 'linear'))

fit1.compile (loss ="mean_squared_error" , optimizer = "adam")   
fit1.fit (x_train, y_train, batch_size =16, nb_epoch =25, shuffle = False)

The first run was not good – prediction was way below actual.

However by changing the number of neurons in LSTM neural network prediction code, it was possible to improve MSE and get predictions much closer to actual data. Below are screenshots of plots of prediction and actual data for LSTM with 5, 60 and 1000 neurons. Note that the plot is showing only testing data. Training data is not shown on the plots. Also you can find below the data for MSE and number of neurons. As the number neorons in LSTM layer is changing there were 2 minimums (one around 100 neurons and another around 1000 neurons)

Results of prediction on LSTM with 5 neurons
Results of prediction on LSTM with 5 neurons
Forecasting one step ahead LSTM 60
Results of prediction on LSTM with 60 neurons
Forecasting one step ahead
Results of prediction on LSTM with 1000 neurons

LSTM 5
in train MSE = 0.0475
in test MSE = 0.2714831660102521

LSTM 60
in train MSE = 0.0127
in test MSE = 0.02227417068206705

LSTM 100
in train MSE = 0.0126
in test MSE = 0.018672733913263073

LSTM 200
in train MSE = 0.0133
in test MSE = 0.020082660237026824

LSTM 900
in train MSE = 0.0103
in test MSE = 0.015546535929778267

LSTM 1000
in train MSE = 0.0104
in test MSE = 0.015037054958075455

LSTM 1100
in train MSE = 0.0113
in test MSE = 0.016363980411369994

Conclusion

The final LSTM was running with testing MSE 0.015 and accuracy 97%. This was obtained by changing number of neurons in LSTM. This example will serve as baseline for further possible improvements on machine learning stock prediction with LSTM. If you have any tips or anything else to add, please leave a comment the comment box. The full source code for LSTM neural network prediction can be found here

References
1. sklearn.preprocessing.MinMaxScaler
2. Keras documentation – RNN Layers
3. How to Reshape Input Data for Long Short-Term Memory Networks in Keras
4. Keras FAQ: Frequently Asked Keras Questions
5. Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras
6.Deep Time Series Forecasting with Python: An Intuitive Introduction to Deep Learning for Applied Time Series Modeling
7. Machine Learning Stock Prediction with LSTM and Keras – Python Source Code