If you were to pick the three most ridiculous fads of 2017, they would definitely be fidget spinners (are they still cool? Do kids still use that word "cool"?), artificial intelligence and, yes, cryptocurrencies. Joking aside, I'm actually impressed by the underlying concept and I'm quite bullish on the long term prospects of this disruptive technology. But enough about fidget spinners!!! I'm actually not a hodler of any cryptos. So, while I may not have a ticket to the moon, I can at least get on board the hype train by successfully predicting the price of cryptos by harnessing deep learning, machine learning and artificial intelligence (yes, all of them!).

I thought this was a completely unique concept to combine deep learning and cryptos (blog-wise at least), but in researching this post (i.e. looking for code to copy+paste), I came across something quite similar. That post only touched on Bitcoin (the most famous crypto of them all), but I'll also discuss Ethereum (commonly known as ether, eth or lambo-money). And since Ether is clearly superior to Bitcoin (have you not heard of Metropolis?), this post will definitely be better than that other one (though this project includes Ether).

We're going to employ a Long Short Term Memory (LSTM) model; it's a particular type of deep learning model that is well suited to time series data (or any data with temporal/spatial/structural order e.g. movies, sentences, etc.). If you wish to truly understand the underlying theory (what kind of crypto enthusiast are you?), then I'd recommend this blog or this blog or the original (white)paper. As I'm shamelessly trying to appeal to a wider non-machine learning audience, I'll keep the code to a minimum. There's a Jupyter (Python) notebook available here, if you want to play around with the data or build your own models. Let's get started!

### Data

Before we build the model, we need to obtain some data for it. There's a dataset on Kaggle that details minute by minute Bitcoin prices (plus some other factors) for the last few years (featured on that other blog post). Over this timescale, noise could overwhelm the signal, so we'll opt for daily prices. The issue here is that we may have not sufficient data (we'll have hundreds of rows rather than thousands or millions). In deep learning, no model can overcome a severe lack of data. I also don't want to rely on static files, as that'll complicate the process of updating the model in the future with new data. Instead, we'll aim to pull data from websites and APIs.

As we'll be combining multiple cryptos in one model, it's probably a good idea to pull the data from one source. We'll use coinmarketcap.com. For now, we'll only consider Bitcoin and Ether, but it wouldn't be hard to add the latest overhyped altcoin using this approach. Before we import the data, we must load some python packages that will make our lives so much easier.

In [3]:
import pandas as pd
import time
import seaborn as sns
import matplotlib.pyplot as plt
import datetime
import numpy as np

# get market info for bitcoin from the start of 2016 to the current day
# convert the date string to the correct date format
bitcoin_market_info = bitcoin_market_info.assign(Date=pd.to_datetime(bitcoin_market_info['Date']))
# when Volume is equal to '-' convert it to 0
bitcoin_market_info.loc[bitcoin_market_info['Volume']=="-",'Volume']=0
# convert to int
bitcoin_market_info['Volume'] = bitcoin_market_info['Volume'].astype('int64')
# sometime after publication of the blog, coinmarketcap starting returning asterisks in the column names
# this will remove those asterisks
bitcoin_market_info.columns = bitcoin_market_info.columns.str.replace("*", "")
# look at the first few rows

Out[3]:
Date Open High Low Close Volume Market Cap
0 2017-11-21 8205.74 8348.66 7762.71 8071.26 4277610000 136967000000
1 2017-11-20 8039.07 8336.86 7949.36 8200.64 3488450000 134167000000
2 2017-11-19 7766.03 8101.91 7694.10 8036.49 3149320000 129595000000
3 2017-11-18 7697.21 7884.99 7463.44 7790.15 3667190000 128425000000
4 2017-11-17 7853.57 8004.59 7561.09 7708.99 4651670000 131026000000

In [4]:
# get market info for ethereum from the start of 2016 to the current day
# convert the date string to the correct date format
eth_market_info = eth_market_info.assign(Date=pd.to_datetime(eth_market_info['Date']))
# sometime after publication of the blog, coinmarketcap starting returning asterisks in the column names
# this will remove those asterisks
eth_market_info.columns = eth_market_info.columns.str.replace("*", "")
# look at the first few rows

Out[4]:
Date Open High Low Close Volume Market Cap
0 2017-11-21 367.44 372.47 350.69 360.40 949912000 35220200000
1 2017-11-20 354.09 372.14 353.29 366.73 807027000 33933400000
2 2017-11-19 347.40 371.29 344.74 354.39 1181530000 33284900000
3 2017-11-18 331.98 349.62 327.69 347.61 649639000 31800700000
4 2017-11-17 330.17 334.96 327.52 332.39 621733000 31620300000

To explain what's just happened, we've loaded some python packages and then imported the table that you see on this site. With a little bit of data cleaning, we arrive at the above table. We also do the same thing for ether by simply replacing 'bitcoin' with 'ethereum' in the url (code omitted).

To prove that the data is accurate, we can plot the price and volume of both cryptos over time.

In [5]:
# getting the Bitcoin and Eth logos
import sys
from PIL import Image
import io

if sys.version_info[0] < 3:
import urllib2 as urllib
else:
import urllib

bitcoin_im = Image.open(image_file)

eth_im = Image.open(image_file)
width_eth_im , height_eth_im  = eth_im.size
eth_im = eth_im.resize((int(eth_im.size[0]*0.8), int(eth_im.size[1]*0.8)), Image.ANTIALIAS)

In [6]:
bitcoin_market_info.columns =[bitcoin_market_info.columns[0]]+['bt_'+i for i in bitcoin_market_info.columns[1:]]
eth_market_info.columns =[eth_market_info.columns[0]]+['eth_'+i for i in eth_market_info.columns[1:]]

In [7]:
fig, (ax1, ax2) = plt.subplots(2,1, gridspec_kw = {'height_ratios':[3, 1]})
ax1.set_ylabel('Closing Price ($)',fontsize=12) ax2.set_ylabel('Volume ($ bn)',fontsize=12)
ax2.set_yticks([int('%d000000000'%i) for i in range(10)])
ax2.set_yticklabels(range(10))
ax1.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax2.set_xticklabels([datetime.date(i,j,1).strftime('%b %Y')  for i in range(2013,2019) for j in [1,7]])
ax1.plot(bitcoin_market_info['Date'].astype(datetime.datetime),bitcoin_market_info['bt_Open'])
ax2.bar(bitcoin_market_info['Date'].astype(datetime.datetime).values, bitcoin_market_info['bt_Volume'].values)
fig.tight_layout()
fig.figimage(bitcoin_im, 100, 120, zorder=3,alpha=.5)
plt.show()

In [8]:
fig, (ax1, ax2) = plt.subplots(2,1, gridspec_kw = {'height_ratios':[3, 1]})
#ax1.set_yscale('log')
ax1.set_ylabel('Closing Price ($)',fontsize=12) ax2.set_ylabel('Volume ($ bn)',fontsize=12)
ax2.set_yticks([int('%d000000000'%i) for i in range(10)])
ax2.set_yticklabels(range(10))
ax1.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax2.set_xticklabels([datetime.date(i,j,1).strftime('%b %Y')  for i in range(2013,2019) for j in [1,7]])
ax1.plot(eth_market_info['Date'].astype(datetime.datetime),eth_market_info['eth_Open'])
ax2.bar(eth_market_info['Date'].astype(datetime.datetime).values, eth_market_info['eth_Volume'].values)
fig.tight_layout()
fig.figimage(eth_im, 300, 180, zorder=3, alpha=.6)
plt.show()

In [9]:
market_info = pd.merge(bitcoin_market_info,eth_market_info, on=['Date'])
market_info = market_info[market_info['Date']>='2016-01-01']
for coins in ['bt_', 'eth_']:
kwargs = { coins+'day_diff': lambda x: (x[coins+'Close']-x[coins+'Open'])/x[coins+'Open']}
market_info = market_info.assign(**kwargs)

Out[9]:
Date bt_Open bt_High bt_Low bt_Close bt_Volume bt_Market Cap eth_Open eth_High eth_Low eth_Close eth_Volume eth_Market Cap bt_day_diff eth_day_diff
0 2017-11-21 8205.74 8348.66 7762.71 8071.26 4277610000 136967000000 367.44 372.47 350.69 360.40 949912000 35220200000 -0.016389 -0.019160
1 2017-11-20 8039.07 8336.86 7949.36 8200.64 3488450000 134167000000 354.09 372.14 353.29 366.73 807027000 33933400000 0.020098 0.035697
2 2017-11-19 7766.03 8101.91 7694.10 8036.49 3149320000 129595000000 347.40 371.29 344.74 354.39 1181530000 33284900000 0.034826 0.020121
3 2017-11-18 7697.21 7884.99 7463.44 7790.15 3667190000 128425000000 331.98 349.62 327.69 347.61 649639000 31800700000 0.012075 0.047081
4 2017-11-17 7853.57 8004.59 7561.09 7708.99 4651670000 131026000000 330.17 334.96 327.52 332.39 621733000 31620300000 -0.018409 0.006724

### Training, Test & Random Walks

We have some data, so now we need to build a model. In deep learning, the data is typically split into training and test sets. The model is built on the training set and subsequently evaluated on the unseen test set. In time series models, we generally train on one period of time and then test on another separate period. Rather arbitrarily, I'll set the cut-off date to June 1st 2017 (i.e. model will be trained on data before that date and assessed on data after it).

In [10]:
split_date = '2017-06-01'
fig, (ax1, ax2) = plt.subplots(2,1)
ax1.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,7]])
ax2.set_xticklabels([datetime.date(i,j,1).strftime('%b %Y')  for i in range(2013,2019) for j in [1,7]])
ax1.plot(market_info[market_info['Date'] < split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date'] < split_date]['bt_Close'],
color='#B08FC7', label='Training')
ax1.plot(market_info[market_info['Date'] >= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date'] >= split_date]['bt_Close'],
color='#8FBAC8', label='Test')
ax2.plot(market_info[market_info['Date'] < split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date'] < split_date]['eth_Close'],
color='#B08FC7')
ax2.plot(market_info[market_info['Date'] >= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date'] >= split_date]['eth_Close'], color='#8FBAC8')
ax1.set_xticklabels('')
ax1.set_ylabel('Bitcoin Price ($)',fontsize=12) ax2.set_ylabel('Ethereum Price ($)',fontsize=12)
plt.tight_layout()
ax1.legend(bbox_to_anchor=(0.03, 1), loc=2, borderaxespad=0., prop={'size': 14})
fig.figimage(bitcoin_im.resize((int(bitcoin_im.size[0]*0.65), int(bitcoin_im.size[1]*0.65)), Image.ANTIALIAS),
200, 260, zorder=3,alpha=.5)
fig.figimage(eth_im.resize((int(eth_im.size[0]*0.65), int(eth_im.size[1]*0.65)), Image.ANTIALIAS),
350, 40, zorder=3,alpha=.5)
plt.show()

You can see that the training period mostly consists of periods when cryptos were relatively cheaper. As such, the training data may not be representative of the test data, undermining the model's ability to generalise to unseen data (you could try to make your data stationary- discussed here). But why let negative realities get in the way of baseless optimism? Before we take our deep artificially intelligent machine learning model to the moon, it's worth discussing a simpler model. The most basic model is to set tomorrow's price equal to today's price (which we'll crudely call a lag model). This is how we'd define such a model in mathematical terms:

\begin{align} PredPrice_{t} & = ActualPrice_{t-1} \end{align}

In [11]:
# trivial lag model: P_t = P_(t-1)
fig, (ax1, ax2) = plt.subplots(2,1)
ax1.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax2.set_xticklabels([datetime.date(2017,i+1,1).strftime('%b %d %Y')  for i in range(12)])
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['bt_Close'].values, label='Actual')
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= datetime.datetime.strptime(split_date, '%Y-%m-%d') -
datetime.timedelta(days=1)]['bt_Close'][1:].values, label='Predicted')
ax1.set_ylabel('Bitcoin Price ($)',fontsize=12) ax1.legend(bbox_to_anchor=(0.1, 1), loc=2, borderaxespad=0., prop={'size': 14}) ax1.set_title('Simple Lag Model (Test Set)') ax2.set_ylabel('Etherum Price ($)',fontsize=12)
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['eth_Close'].values, label='Actual')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= datetime.datetime.strptime(split_date, '%Y-%m-%d') -
datetime.timedelta(days=1)]['eth_Close'][1:].values, label='Predicted')
fig.tight_layout()
plt.show()

Extending this trivial lag model, stock prices are commonly treated as random walks, which can be defined in these mathematical terms:

\begin{align} PredPrice_{t} & = ActualPrice_{t-1} * \epsilon, \epsilon \sim N(\mu, \sigma) \end{align}

We'll determine μ and σ from the training sets and apply the random walk model to the Bitcoin and Ethereum test sets.

First, we may want to make sure the daily change in price follows a normal distribution. We'll plot the histogram of values.

In [12]:
fig, (ax1, ax2) = plt.subplots(1,2)
ax1.hist(market_info[market_info['Date']< split_date]['bt_day_diff'].values, bins=100)
ax2.hist(market_info[market_info['Date']< split_date]['eth_day_diff'].values, bins=100)
ax1.set_title('Bitcoin Daily Price Changes')
ax2.set_title('Ethereum Daily Price Changes')
plt.show()

In [13]:
np.random.seed(202)
bt_r_walk_mean, bt_r_walk_sd = np.mean(market_info[market_info['Date']< split_date]['bt_day_diff'].values), \
np.std(market_info[market_info['Date']< split_date]['bt_day_diff'].values)
bt_random_steps = np.random.normal(bt_r_walk_mean, bt_r_walk_sd,
(max(market_info['Date']).to_pydatetime() - datetime.datetime.strptime(split_date, '%Y-%m-%d')).days + 1)
eth_r_walk_mean, eth_r_walk_sd = np.mean(market_info[market_info['Date']< split_date]['eth_day_diff'].values), \
np.std(market_info[market_info['Date']< split_date]['eth_day_diff'].values)
eth_random_steps = np.random.normal(eth_r_walk_mean, eth_r_walk_sd,
(max(market_info['Date']).to_pydatetime() - datetime.datetime.strptime(split_date, '%Y-%m-%d')).days + 1)
fig, (ax1, ax2) = plt.subplots(2,1)
ax1.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax2.set_xticklabels([datetime.date(2017,i+1,1).strftime('%b %d %Y')  for i in range(12)])
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['bt_Close'].values, label='Actual')
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[(market_info['Date']+ datetime.timedelta(days=1))>= split_date]['bt_Close'].values[1:] *
(1+bt_random_steps), label='Predicted')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['eth_Close'].values, label='Actual')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[(market_info['Date']+ datetime.timedelta(days=1))>= split_date]['eth_Close'].values[1:] *
(1+eth_random_steps), label='Predicted')
ax1.set_title('Single Point Random Walk (Test Set)')
ax1.set_ylabel('Bitcoin Price ($)',fontsize=12) ax2.set_ylabel('Ethereum Price ($)',fontsize=12)
ax1.legend(bbox_to_anchor=(0.1, 1), loc=2, borderaxespad=0., prop={'size': 14})
plt.tight_layout()
plt.show()

Wow! Look at those prediction lines. Apart from a few kinks, it broadly tracks the actual closing price for each coin. It even captures the eth rises (and subsequent falls) in mid-June and late August. At this stage, if I was to announce the launch of sheehanCoin, I'm sure that ICO would stupidly over-subscribed. As pointed out on that other blog, models that only make predictions one point into the future are often misleadingly accurate, as errors aren't carried over to subsequent predictions. No matter how large the error, it's essentially reset at each time point, as the model is fed the true price. The Bitcoin random walk is particularly deceptive, as the scale of the y-axis is quite wide, making the prediction line appear quite smooth.

Single point predictions are unfortunately quite common when evaluating time series models (e.g.here and here). A better idea could be to measure its accuracy on multi-point predictions. That way, errors from previous predictions aren't reset but rather are compounded by subsequent predictions. Thus, poor models are penalised more heavily. In mathematical terms:

\begin{align} PredPrice_{t} & = PredPrice_{t-1} * \epsilon, \epsilon \sim N(\mu, \sigma)\ \& \ PredPrice_0 = Price_0 \end{align}

Let's get our random walk model to predict the closing prices over the total test set.

In [14]:
bt_random_walk = []
eth_random_walk = []
for n_step, (bt_step, eth_step) in enumerate(zip(bt_random_steps, eth_random_steps)):
if n_step==0:
bt_random_walk.append(market_info[market_info['Date']< split_date]['bt_Close'].values[0] * (bt_step+1))
eth_random_walk.append(market_info[market_info['Date']< split_date]['eth_Close'].values[0] * (eth_step+1))
else:
bt_random_walk.append(bt_random_walk[n_step-1] * (bt_step+1))
eth_random_walk.append(eth_random_walk[n_step-1] * (eth_step+1))
fig, (ax1, ax2) = plt.subplots(2, 1)
ax1.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax2.set_xticklabels([datetime.date(2017,i+1,1).strftime('%b %d %Y')  for i in range(12)])
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['bt_Close'].values, label='Actual')
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
bt_random_walk[::-1], label='Predicted')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['eth_Close'].values, label='Actual')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
eth_random_walk[::-1], label='Predicted')

ax1.set_title('Full Interval Random Walk')
ax1.set_ylabel('Bitcoin Price ($)',fontsize=12) ax2.set_ylabel('Ethereum Price ($)',fontsize=12)
ax1.legend(bbox_to_anchor=(0.1, 1), loc=2, borderaxespad=0., prop={'size': 14})
plt.tight_layout()
plt.show()

The model predictions are extremely sensitive to the random seed. I've selected one where the full interval random walk looks almost decent for Ethereum. In the accompanying Jupyter notebook, you can interactively play around with the seed value below to see how badly it can perform.

In [15]:
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets

def plot_func(freq):
np.random.seed(freq)
random_steps = np.random.normal(eth_r_walk_mean, eth_r_walk_sd,
(max(market_info['Date']).to_pydatetime() - datetime.datetime.strptime(split_date, '%Y-%m-%d')).days + 1)
random_walk = []
for n_step,i in enumerate(random_steps):
if n_step==0:
random_walk.append(market_info[market_info['Date']< split_date]['eth_Close'].values[0] * (i+1))
else:
random_walk.append(random_walk[n_step-1] * (i+1))
fig, (ax1, ax2) = plt.subplots(2,1)
ax1.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax1.set_xticklabels('')
ax2.set_xticks([datetime.date(2017,i+1,1) for i in range(12)])
ax2.set_xticklabels([datetime.date(2017,i+1,1).strftime('%b %d %Y')  for i in range(12)])
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[market_info['Date']>= split_date]['eth_Close'].values, label='Actual')
ax1.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[(market_info['Date']+ datetime.timedelta(days=1))>= split_date]['eth_Close'].values[1:] *
(1+random_steps), label='Predicted')
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
market_info[(market_info['Date']+ datetime.timedelta(days=1))>= split_date]['eth_Close'].values[1:] *
(1+random_steps))
ax2.plot(market_info[market_info['Date']>= split_date]['Date'].astype(datetime.datetime),
random_walk[::-1])
ax1.set_title('Single Point Random Walk')
ax1.set_ylabel('')
# for static figures, you may wish to insert the random seed value
#    ax1.annotate('Random Seed: %d'%freq, xy=(0.75, 0.2),  xycoords='axes fraction',
#            xytext=(0.75, 0.2), textcoords='axes fraction')
ax1.legend(bbox_to_anchor=(0.1, 1), loc=2, borderaxespad=0., prop={'size': 14})
ax2.set_title('Full Interval Random Walk')
fig.text(0.0, 0.5, 'Ethereum Price ($)', va='center', rotation='vertical',fontsize=12) plt.tight_layout() # plt.savefig('image%d.png'%freq, bbox_inches='tight') plt.show() interact(plot_func, freq =widgets.IntSlider(min=200,max=210,step=1,value=205, description='Random Seed:')) Out[15]: <function __main__.plot_func> Notice how the single point random walk always looks quite accurate, even though there's no real substance behind it. Hopefully, you'll be more suspicious of any blog that claims to accurately predict prices. I probably shouldn't worry; it's not like crypto fans to be seduced by slick marketing claims. ## Long Short Term Memory (LSTM) Like I said, if you're interested in the theory behind LSTMs, then I'll refer you to this, this and this. Luckily, we don't need to build the network from scratch (or even understand it), there exists packages that include standard implementations of various deep learning algorithms (e.g. TensorFlow, Keras, PyTorch, etc.). I'll opt for Keras, as I find it the most intuitive for non-experts. If you're not familiar with Keras, then check out my previous tutorial. In [16]: for coins in ['bt_', 'eth_']: kwargs = { coins+'close_off_high': lambda x: 2*(x[coins+'High']- x[coins+'Close'])/(x[coins+'High']-x[coins+'Low'])-1, coins+'volatility': lambda x: (x[coins+'High']- x[coins+'Low'])/(x[coins+'Open'])} market_info = market_info.assign(**kwargs) In [17]: model_data = market_info[['Date']+[coin+metric for coin in ['bt_', 'eth_'] for metric in ['Close','Volume','close_off_high','volatility']]] # need to reverse the data frame so that subsequent rows represent later timepoints model_data = model_data.sort_values(by='Date') model_data.head() Out[17]: Date bt_Close bt_Volume bt_close_off_high bt_volatility eth_Close eth_Volume eth_close_off_high eth_volatility 690 2016-01-01 434.33 36278900 -0.560641 0.020292 0.948024 206062 -0.418477 0.025040 689 2016-01-02 433.44 30096600 0.250597 0.009641 0.937124 255504 0.965898 0.034913 688 2016-01-03 430.01 39633800 -0.173865 0.020827 0.971905 407632 -0.317885 0.060792 687 2016-01-04 433.09 38477500 -0.474265 0.012649 0.954480 346245 -0.057657 0.047943 686 2016-01-05 431.96 34522600 -0.013333 0.010391 0.950176 219833 0.697930 0.025236 I've created a new data frame called model_data. I've removed some of the previous columns (open price, daily highs and lows) and reformulated some new ones. close_off_high represents the gap between the closing price and price high for that day, where values of -1 and 1 mean the closing price was equal to the daily low or daily high, respectively. The volatility columns are simply the difference between high and low price divided by the opening price. You may also notice that model_data is arranged in order of earliest to latest. We don't actually need the date column anymore, as that information won't be fed into the model. Our LSTM model will use previous data (both bitcoin and eth) to predict the next day's closing price of a specific coin. We must decide how many previous days it will have access to. Again, it's rather arbitrary, but I'll opt for 10 days, as it's a nice round number. We build little data frames consisting of 10 consecutive days of data (called windows), so the first window will consist of the 0-9th rows of the training set (Python is zero-indexed), the second will be the rows 1-10, etc. Picking a small window size means we can feed more windows into our model; the downside is that the model may not have sufficient information to detect complex long term behaviours (if such things exist). Deep learning models don't like inputs that vary wildly. Looking at those columns, some values range between -1 and 1, while others are on the scale of millions. We need to normalise the data, so that our inputs are somewhat consistent. Typically, you want values between -1 and 1. The off_high and volatility columns are fine as they are. For the remaining columns, like that other blog post, we'll normalise the inputs to the first value in the window. In [18]: # we don't need the date columns anymore training_set, test_set = model_data[model_data['Date']<split_date], model_data[model_data['Date']>=split_date] training_set = training_set.drop('Date', 1) test_set = test_set.drop('Date', 1) In [19]: window_len = 10 norm_cols = [coin+metric for coin in ['bt_', 'eth_'] for metric in ['Close','Volume']] In [20]: LSTM_training_inputs = [] for i in range(len(training_set)-window_len): temp_set = training_set[i:(i+window_len)].copy() for col in norm_cols: temp_set.loc[:, col] = temp_set[col]/temp_set[col].iloc[0] - 1 LSTM_training_inputs.append(temp_set) LSTM_training_outputs = (training_set['eth_Close'][window_len:].values/training_set['eth_Close'][:-window_len].values)-1 In [21]: LSTM_test_inputs = [] for i in range(len(test_set)-window_len): temp_set = test_set[i:(i+window_len)].copy() for col in norm_cols: temp_set.loc[:, col] = temp_set[col]/temp_set[col].iloc[0] - 1 LSTM_test_inputs.append(temp_set) LSTM_test_outputs = (test_set['eth_Close'][window_len:].values/test_set['eth_Close'][:-window_len].values)-1 In [22]: LSTM_training_inputs[0] Out[22]: bt_Close bt_Volume bt_close_off_high bt_volatility eth_Close eth_Volume eth_close_off_high eth_volatility 690 0.000000 0.000000 -0.560641 0.020292 0.000000 0.000000 -0.418477 0.025040 689 -0.002049 -0.170410 0.250597 0.009641 -0.011498 0.239937 0.965898 0.034913 688 -0.009946 0.092475 -0.173865 0.020827 0.025190 0.978201 -0.317885 0.060792 687 -0.002855 0.060603 -0.474265 0.012649 0.006810 0.680295 -0.057657 0.047943 686 -0.005457 -0.048411 -0.013333 0.010391 0.002270 0.066829 0.697930 0.025236 685 -0.012019 -0.061645 -0.003623 0.012782 0.002991 0.498534 -0.214540 0.026263 684 0.054613 1.413585 -0.951499 0.069045 -0.006349 2.142074 0.681644 0.040587 683 0.043515 0.570968 0.294196 0.032762 0.040890 1.647747 -0.806717 0.055274 682 0.030576 -0.110282 0.814194 0.017094 0.040937 0.098121 -0.411897 0.019021 681 0.031451 -0.007801 -0.919598 0.017758 0.054014 0.896944 -0.938235 0.025266 This table represents an example of our LSTM model input (we'll actually have hundreds of similar tables). We've normalised some columns so that their values are equal to 0 in the first time point, so we're aiming to predict changes in price relative to this timepoint. We're now ready to build the LSTM model. This is actually quite straightforward with Keras, you simply stack componenets on top of each other (better explained here). In [23]: # I find it easier to work with numpy arrays rather than pandas dataframes # especially as we now only have numerical data LSTM_training_inputs = [np.array(LSTM_training_input) for LSTM_training_input in LSTM_training_inputs] LSTM_training_inputs = np.array(LSTM_training_inputs) LSTM_test_inputs = [np.array(LSTM_test_inputs) for LSTM_test_inputs in LSTM_test_inputs] LSTM_test_inputs = np.array(LSTM_test_inputs) In [24]: # import the relevant Keras modules from keras.models import Sequential from keras.layers import Activation, Dense from keras.layers import LSTM from keras.layers import Dropout def build_model(inputs, output_size, neurons, activ_func="linear", dropout=0.25, loss="mae", optimizer="adam"): model = Sequential() model.add(LSTM(neurons, input_shape=(inputs.shape[1], inputs.shape[2]))) model.add(Dropout(dropout)) model.add(Dense(units=output_size)) model.add(Activation(activ_func)) model.compile(loss=loss, optimizer=optimizer) return model Using TensorFlow backend. So, the build_model functions constructs an empty model unimaginatively called model (model = Sequential), to which an LSTM layer is added. That layer has been shaped to fit our inputs (n x m tables, where n and m represent the number of timepoints/rows and columns, respectively). The function also includes more generic neural network features, like dropout and activation functions. Now, we just need to specify the number of neurons to place in the LSTM layer (I've opted for 20 to keep runtime reasonable), as well as the data on which the model will be trained. In [25]: # random seed for reproducibility np.random.seed(202) # initialise model architecture eth_model = build_model(LSTM_training_inputs, output_size=1, neurons = 20) # model output is next price normalised to 10th previous closing price LSTM_training_outputs = (training_set['eth_Close'][window_len:].values/training_set['eth_Close'][:-window_len].values)-1 # train model on data # note: eth_history contains information on the training error per epoch eth_history = eth_model.fit(LSTM_training_inputs, LSTM_training_outputs, epochs=50, batch_size=1, verbose=2, shuffle=True) Epoch 1/50 6s - loss: 0.1648 Epoch 2/50 6s - loss: 0.1117 Epoch 3/50 6s - loss: 0.1064 Epoch 4/50 4s - loss: 0.0952 Epoch 5/50 4s - loss: 0.0870 Epoch 6/50 4s - loss: 0.0848 Epoch 7/50 5s - loss: 0.0835 Epoch 8/50 4s - loss: 0.0801 Epoch 9/50 4s - loss: 0.0788 Epoch 10/50 4s - loss: 0.0784 Epoch 11/50 4s - loss: 0.0752 Epoch 12/50 4s - loss: 0.0743 Epoch 13/50 4s - loss: 0.0740 Epoch 14/50 4s - loss: 0.0690 Epoch 15/50 4s - loss: 0.0675 Epoch 16/50 4s - loss: 0.0691 Epoch 17/50 4s - loss: 0.0746 Epoch 18/50 4s - loss: 0.0682 Epoch 19/50 4s - loss: 0.0705 Epoch 20/50 4s - loss: 0.0710 Epoch 21/50 4s - loss: 0.0680 Epoch 22/50 4s - loss: 0.0714 Epoch 23/50 4s - loss: 0.0676 Epoch 24/50 4s - loss: 0.0665 Epoch 25/50 4s - loss: 0.0692 Epoch 26/50 4s - loss: 0.0658 Epoch 27/50 4s - loss: 0.0654 Epoch 28/50 4s - loss: 0.0652 Epoch 29/50 4s - loss: 0.0685 Epoch 30/50 4s - loss: 0.0686 Epoch 31/50 4s - loss: 0.0666 Epoch 32/50 5s - loss: 0.0689 Epoch 33/50 4s - loss: 0.0695 Epoch 34/50 4s - loss: 0.0688 Epoch 35/50 4s - loss: 0.0655 Epoch 36/50 4s - loss: 0.0676 Epoch 37/50 4s - loss: 0.0655 Epoch 38/50 4s - loss: 0.0660 Epoch 39/50 4s - loss: 0.0676 Epoch 40/50 4s - loss: 0.0658 Epoch 41/50 4s - loss: 0.0668 Epoch 42/50 4s - loss: 0.0663 Epoch 43/50 4s - loss: 0.0646 Epoch 44/50 4s - loss: 0.0657 Epoch 45/50 4s - loss: 0.0651 Epoch 46/50 4s - loss: 0.0661 Epoch 47/50 4s - loss: 0.0666 Epoch 48/50 4s - loss: 0.0633 Epoch 49/50 4s - loss: 0.0663 Epoch 50/50 4s - loss: 0.0625 If everything went to plan, then we'd expect the training error to have gradually decreased over time. In [26]: fig, ax1 = plt.subplots(1,1) ax1.plot(eth_history.epoch, eth_history.history['loss']) ax1.set_title('Training Error') if eth_model.loss == 'mae': ax1.set_ylabel('Mean Absolute Error (MAE)',fontsize=12) # just in case you decided to change the model loss calculation else: ax1.set_ylabel('Model Loss',fontsize=12) ax1.set_xlabel('# Epochs',fontsize=12) plt.show() We've just built an LSTM model to predict tomorrow's Ethereum closing price. Let's see how well it performs. We start by examining its performance on the training set (data before June 2017). That number below the code represents the model's mean absolute error (mae) on the training set after the 50th training iteration (or epoch). Instead of relative changes, we can view the model output as daily closing prices. In [27]: from mpl_toolkits.axes_grid1.inset_locator import zoomed_inset_axes from mpl_toolkits.axes_grid1.inset_locator import mark_inset fig, ax1 = plt.subplots(1,1) ax1.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,5,9]]) ax1.set_xticklabels([datetime.date(i,j,1).strftime('%b %Y') for i in range(2013,2019) for j in [1,5,9]]) ax1.plot(model_data[model_data['Date']< split_date]['Date'][window_len:].astype(datetime.datetime), training_set['eth_Close'][window_len:], label='Actual') ax1.plot(model_data[model_data['Date']< split_date]['Date'][window_len:].astype(datetime.datetime), ((np.transpose(eth_model.predict(LSTM_training_inputs))+1) * training_set['eth_Close'].values[:-window_len])[0], label='Predicted') ax1.set_title('Training Set: Single Timepoint Prediction') ax1.set_ylabel('Ethereum Price ($)',fontsize=12)
ax1.legend(bbox_to_anchor=(0.15, 1), loc=2, borderaxespad=0., prop={'size': 14})
ax1.annotate('MAE: %.4f'%np.mean(np.abs((np.transpose(eth_model.predict(LSTM_training_inputs))+1)-\
(training_set['eth_Close'].values[window_len:])/(training_set['eth_Close'].values[:-window_len]))),
xy=(0.75, 0.9),  xycoords='axes fraction',
xytext=(0.75, 0.9), textcoords='axes fraction')
# figure inset code taken from http://akuederle.com/matplotlib-zoomed-up-inset
axins = zoomed_inset_axes(ax1, 3.35, loc=10) # zoom-factor: 3.35, location: centre
axins.set_xticks([datetime.date(i,j,1) for i in range(2013,2019) for j in [1,5,9]])
axins.plot(model_data[model_data['Date']< split_date]['Date'][window_len:].astype(datetime.datetime),
training_set['eth_Close'][window_len:], label='Actual')
axins.plot(model_data[model_data['Date']< split_date]['Date'][window_len:].astype(datetime.datetime),
((np.transpose(eth_model.predict(LSTM_training_inputs))+1) * training_set['eth_Close'].values[:-window_len])[0],
label='Predicted')
axins.set_xlim([datetime.date(2017, 3, 1), datetime.date(2017, 5, 1)])
axins.set_ylim([10,60])
axins.set_xticklabels('')
mark_inset(ax1, axins, loc1=1, loc2=3, fc="none", ec="0.5")
plt.show()