Summary

Get a list of ASX stocks that have a market capitlisation greater than $100 million from Eoddata.com, then use this list to download as much daily data as possible for each stock in the list from Yahoo Finance.

Next, run a backtest for a momentum trading strategy with trading signals generated by Doncian Channels, which is one of the signals that Richard Dennis' Turtle Traders used.

Note: This analysis of the momentum strategy assumes/suffers from the following:

  • Survivorship Bias, which will inflate returns, due to stocks that went bankrupt in the past not being included in the analysis.
  • Overfitting, as it uses all data available to determine performance metrics, and does not perform any out-of-sample testing.
  • Efficient Market Hypothesis is rejected due to the use of technical analysis.

Change Log

Date Created: 2017-01-19

Date of Change    Change Notes
--------------    ----------------------------------------------------------------
2017-01-19        Initial draft

3. Get data to run our tests with


3.1 Setup

  • Run our imports and setup the constants.
  • Setup tje instance variables that can be amended later if we so choose.
  • Set the credentials for the Eoddata.com web service.
  • Log into the service and acquire a token for accessing the web service.

In [1]:
# %matplotlib inline

from getpass import getpass
from pandas_datareader import data
from datetime import datetime, timedelta
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
import requests as r
import pandas as pd
import xml.etree.cElementTree as etree

# allow plotly to embed plots in notebook
init_notebook_mode(connected=True)

# constants
web_service = 'http://ws.eoddata.com/data.asmx'
name_space ='http://ws.eoddata.com/Data'
exchange = 'ASX'

# ivars
min_mcap = 100000000
min_days_traded = 250
start = '1999-01-01'
end = str(datetime.now())[:10]



In [2]:
eoddata_username = getpass()


········

In [3]:
eoddata_password = getpass()


········

In [8]:
# set params to make the login call
call = 'Login'
url = '/'.join((web_service, call))
params = {'Username': eoddata_username, 'Password': eoddata_password}

# make login call
s = r.Session()
response = s.get(url, params=params, stream=True)

# set token if login was successful
if response.status_code == 200:
    xml_doc_root = etree.parse(response.raw).getroot()
    
    if xml_doc_root.get('Message') == 'Login Successful':
        token = xml_doc_root.get('Token')

3.2 Get list of stocks

Here we are going to make a call to get a list of fundamentals for each stock from the ASX. This uses a web service call named FundamentalList.


In [9]:
# set params to make the fundamental list call
call = 'FundamentalList'
url = '/'.join((web_service, call))
params = {'Token': token, 'Exchange': exchange}

# make fundamental list call
response = s.get(url, params=params, stream=True)
xml_doc_root = etree.parse(response.raw).getroot()

# get a list of ticker symbols for all stocks that have a market cap greater than
# the threshold established by min_mcap
symbol_list = []
for fundamental in xml_doc_root.findall(".//{%s}FUNDAMENTAL" % (name_space)):
    if int(fundamental.get('MarketCap')) >= min_mcap:
        symbol_list.append([
                fundamental.get('Symbol'),
                fundamental.get('Name'),
                int(fundamental.get('MarketCap')),])

# create a dataframe from the data returned to use as a reference/lookup table later
df = pd.DataFrame(symbol_list, columns=['yahoo_ticker', 'name', 'mcap'])
df = df.sort_values(by='mcap', ascending=False).reset_index(drop=True)
symbol_list = None

# create a column that will hold the ticker symbols Yahoo Finance expects
df['yahoo_ticker'] = df['yahoo_ticker'] + '.AX'

# place the ticker symbols into a separate list to keep our later code tidier,
# and determine how many stocks we have in the list
stocks = df['yahoo_ticker'].values.tolist()
len(stocks)

3.3 Download stock data

Get the end of day adjusted closing price for each stock in our list. Errors maybe returned as we haven't cleaned the stock list to ensure exact matches with Yahoo Finance's expected ticker symbol formatting.

Also time the operation purely out of curiosity.


In [13]:
%%time
prices = data.DataReader(stocks, 'yahoo', start, end)['Adj Close']


/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'USP.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'JIP.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'AYH.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'ING.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'ATT.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'WGX.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'SM1.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'FIG.AX', replacing with NaN.

/home/adrian/miniconda3/envs/ds_ml_portfolio/lib/python3.5/site-packages/pandas_datareader/base.py:192: SymbolWarning:

Failed to read symbol: 'MRG.AX', replacing with NaN.

CPU times: user 15 s, sys: 1.55 s, total: 16.6 s
Wall time: 9min 9s

In [14]:
stats = prices.describe().transpose()
cols_to_drop = list(stats[stats['count'] < min_days_traded].index)
len(cols_to_drop)


Out[14]:
29

In [15]:
prices.drop(cols_to_drop, axis=1, inplace=True)
prices.iloc[:, 0:10].head()


Out[15]:
1AL.AX 3PL.AX 88E.AX 8IH.AX A2M.AX AAC.AX AAD.AX ABA.AX ABC.AX ABP.AX
Date
1999-01-04 NaN NaN NaN NaN NaN NaN NaN 0.75078 0.21648 NaN
1999-01-05 NaN NaN NaN NaN NaN NaN NaN NaN 0.21380 NaN
1999-01-06 NaN NaN NaN NaN NaN NaN NaN 0.76647 0.20846 NaN
1999-01-07 NaN NaN NaN NaN NaN NaN NaN 0.78440 0.21380 NaN
1999-01-08 NaN NaN NaN NaN NaN NaN NaN 0.78440 0.21648 NaN

In [16]:
prices.fillna(method='ffill', inplace=True)
prices.iloc[:, 0:10].head()


Out[16]:
1AL.AX 3PL.AX 88E.AX 8IH.AX A2M.AX AAC.AX AAD.AX ABA.AX ABC.AX ABP.AX
Date
1999-01-04 NaN NaN NaN NaN NaN NaN NaN 0.75078 0.21648 NaN
1999-01-05 NaN NaN NaN NaN NaN NaN NaN 0.75078 0.21380 NaN
1999-01-06 NaN NaN NaN NaN NaN NaN NaN 0.76647 0.20846 NaN
1999-01-07 NaN NaN NaN NaN NaN NaN NaN 0.78440 0.21380 NaN
1999-01-08 NaN NaN NaN NaN NaN NaN NaN 0.78440 0.21648 NaN

In [19]:
iplot([{'x': prices.index, 'y': prices['CBA.AX']}])