ntfdl - intraday example

This example shows how to get all trades (ticks) from the last trading day for Statoil ASA (STL). Ntfdl uses free and accessible data from the norwegian broker http://www.netfonds.no


In [1]:
%matplotlib inline
%pylab inline --no-import-all
pylab.rcParams['figure.figsize'] = (18, 10)


Populating the interactive namespace from numpy and matplotlib

Import the downloader dl from ntfdl


In [2]:
from ntfdl import Dl
import pandas as pd
import talib
from matplotlib.finance import candlestick_ohlc
import matplotlib.dates as mdates

We instantiate the dl class with:

  • instrument (ticker used by Netfonds)
  • exchange="OSE" (Exchange abbr used by Netfonds)
  • day="today" (or string 'YYYYMMDD' or datetime/date object)
  • download=True (download data on init True or False)
  • exclude_derivate=True (Removes derivate trades which can be off the chart in pricing)

In [3]:
stl = Dl('STL', exchange='OSE', day='today', download=True)

In [4]:
# If download=False then you need to call dl.get_trades()
#trades = stl.get_trades()

In [5]:
# Calculate vwap
stl.vwap()

Accessing intraday trades

Since download=True all trades for the day is downloaded and accessable with dl.trades. dl.get_trades() returns a panda dataframe available both from dl.get_trades() and dl.trades.

Since this is a pandas dataframe, all pandas functionality is exposed through dl.get_trades()


In [6]:
stl.trades.tail(5)


Out[6]:
time price volume source buyer seller initiator vwap
time
2017-11-20 16:25:09 2017-11-20 16:25:09 164.2 218 Auto trade SGP MSI NaN 164.212393
2017-11-20 16:25:09 2017-11-20 16:25:09 164.2 20 Auto trade NON MSI NaN 164.212393
2017-11-20 16:25:09 2017-11-20 16:25:09 164.2 540 Auto trade MSI MSI NaN 164.212389
2017-11-20 16:25:09 2017-11-20 16:25:09 164.2 158 Auto trade MSI MSI NaN 164.212388
2017-11-20 16:25:09 2017-11-20 16:25:09 164.2 92 Auto trade MSI MLI B 164.212388

In [7]:
stl.trades.price.max()


Out[7]:
165.0

In [8]:
stl.trades.price.std()


Out[8]:
0.3082697954727128

Plotting pandas is also easy


In [9]:
fig, ax = plt.subplots()

# Plot price
stl.trades.price.plot(drawstyle='steps')

# Plot vwap
#stl.trades.price.plot(color='r')

# Adding axis right hand side 
ax.tick_params(labeltop=False, labelright=True)

# Annotate last quote
xmin, xmax = ax.get_xlim()
plt.annotate(stl.trades.iloc[-1].price, xy=(1, stl.trades.iloc[-1].price), xytext=(0, 0), \
                 xycoords=('axes fraction', 'data'), textcoords='offset points', backgroundcolor='k', color='w')
ax.plot(xmax, stl.trades.iloc[-1].price, '<k', markersize=18, markeredgecolor='k')

plt.grid()
plt.title("Statoil ASA (STL.OSE)")


Out[9]:
<matplotlib.text.Text at 0x7ff4a4f894e0>

Resampling trade data

You can easily resample the trade data yourself using pandas resample, but dl.get_ohlcv() will make it easier. dl.get_ohlcv() takes two parameters:

  • interval is the pandas resampling window ('30s', '2min', '1h' etc)
  • vwap which will also resample the volume weighted average price from the trade data dataframe.

Note that volume always is included.


In [10]:
ohlcv = stl.get_ohlcv(interval='5min')

dl.get_ohlcv() also returns a Pandas dataframe, hence all Pandas functionality is available as well.


In [11]:
ohlcv.tail()


Out[11]:
time open high low close volume
time
2017-11-20 16:05:00 2017-11-20 16:05:00 163.7 164.0 163.7 164.0 29869.0
2017-11-20 16:10:00 2017-11-20 16:10:00 164.0 164.2 164.0 164.2 11678.0
2017-11-20 16:15:00 2017-11-20 16:15:00 164.2 164.3 164.0 164.1 32335.0
2017-11-20 16:20:00 2017-11-20 16:20:00 164.1 164.1 164.1 164.1 0.0
2017-11-20 16:25:00 2017-11-20 16:25:00 164.2 164.2 164.2 164.2 573296.0

Plotting OHLC resampeled data

Matplotlib is not the very best plotting library for financial plots, so using a custom wrapper function borrowed from quantopian (not sure where). Here we also utilize Ta-lib to add some technical overlays.


In [12]:
def plot_candles(pricing, title=None,
                 volume_bars=False,
                 color_function=None,
                 overlays=None,
                 technicals=None,
                 technicals_titles=None):

    def default_color(index, open, close, low, high):
        return 'r' if open[index] > close[index] else 'g'
    color_function = color_function or default_color
    overlays = overlays or []
    technicals = technicals or []
    technicals_titles = technicals_titles or []
    open = pricing['open']
    close = pricing['close']
    low = pricing['low']
    high = pricing['high']
    oc_min = pd.concat([open, close], axis=1).min(axis=1)
    oc_max = pd.concat([open, close], axis=1).max(axis=1)
    
    subplot_count = 1
    if volume_bars:
        subplot_count = 2
    if technicals:
        subplot_count += len(technicals)
    
    if subplot_count == 1:
        fig, ax1 = plt.subplots(1, 1)
    else:
        ratios = np.insert(np.full(subplot_count - 1, 1), 0, 3)
        fig, subplots = plt.subplots(subplot_count, 1, sharex=True, gridspec_kw={'height_ratios': ratios})
        ax1 = subplots[0]
        
    if title:
        ax1.set_title(title)
    x = np.arange(len(pricing))
    candle_colors = [color_function(i, open, close, low, high) for i in x]
    candles = ax1.bar(x, oc_max-oc_min, bottom=oc_min, color=candle_colors, linewidth=0)
    lines = ax1.vlines(x, low, high, color=candle_colors, linewidth=1)
    ax1.xaxis.grid(False)
    ax1.xaxis.set_tick_params(which='major', length=3.0, direction='in', top='off')
    # Assume minute frequency if first two bars are in the same day.
    frequency = 'minute' if (pricing.index[1] - pricing.index[0]).days == 0 else 'day'
    time_format = '%d-%m-%Y'
    if frequency == 'minute':
        time_format = '%H:%M'
    # Set X axis tick labels.
    plt.xticks(x, [date.strftime(time_format) for date in pricing.index], rotation='vertical')
    for overlay in overlays:
        ax1.plot(x, overlay)
    # Plot volume bars if needed
    if volume_bars:
        ax2 = subplots[1]
        volume = pricing['volume']
        volume_scale = None
        scaled_volume = volume
        if volume.max() > 1000000:
            volume_scale = 'M'
            scaled_volume = volume / 1000000
        elif volume.max() > 1000:
            volume_scale = 'K'
            scaled_volume = volume / 1000
        ax2.bar(x, scaled_volume, color=candle_colors)
        volume_title = 'Volume'
        if volume_scale:
            volume_title = 'Volume (%s)' % volume_scale
        ax2.set_title(volume_title)
        ax2.xaxis.grid(False)
    # Plot additional technical indicators
    for (i, technical) in enumerate(technicals):
        ax = subplots[i - len(technicals)] # Technical indicator plots are shown last
        ax.plot(x, technical)
        if i < len(technicals_titles):
            ax.set_title(technicals_titles[i])

In [13]:
upper, middle, lower = talib.BBANDS(ohlcv['close'].as_matrix())
plot_candles(ohlcv, title='Statoil ASA 2017-10-23 [STL.OSE]',volume_bars=True, overlays=[upper, middle, lower])


Plotting using Matplotlib's candlestick_ohlc


In [14]:
import matplotlib.ticker as mplticker
from matplotlib.dates import date2num
from matplotlib.finance import candlestick_ohlc

fig, ax = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(18,10))

ax.xaxis.set_major_locator(mplticker.MaxNLocator(6))

#Convert to integer, not date...
ohlcv['time'] = ohlcv.index.astype(np.int64) // 10**9
xdate = ohlcv['time'] #[datetime.fromtimestamp(i) for i in ohlc['time']]

def mydate(x,pos):
    try:
        return xdate[int(x)]
    except IndexError:
        return ''

ax.xaxis.set_major_formatter(mplticker.FuncFormatter(mydate))
ax.tick_params(labeltop=False, labelright=True)

fig.autofmt_xdate()
fig.tight_layout()

tuples = [tuple(x) for x in ohlcv[['time','open','high','low','close']].values]
candlestick_ohlc(ax, tuples, colorup='g')

plt.show()


Accessing intraday positions (orderbook)

There is also a orderbook called positions available from Netfonds. It contains best bid and ask and a summary of total bid and ask.


In [15]:
positions = stl.get_positions()

In [16]:
positions.tail(5)


Out[16]:
time bid bid_depth bid_depth_total ask ask_depth ask_depth_total
time
2017-11-20 16:25:06 2017-11-20 16:25:06 164.2 573473 1268845 164.2 573473 1531450
2017-11-20 16:25:07 2017-11-20 16:25:07 164.2 573296 1268668 164.2 573296 1531450
2017-11-20 16:25:09 2017-11-20 16:25:09 164.1 29140 686572 164.2 12526 1067775
2017-11-20 16:25:09 2017-11-20 16:25:09 164.1 29140 686572 164.2 12434 1067683
2017-11-20 16:25:09 2017-11-20 16:25:09 164.1 29140 548414 164.2 921 1056170

In [17]:
fig, ax = plt.subplots()
ax.tick_params(labeltop=False, labelright=True)
positions.bid.plot(drawstyle='steps', color='b')
positions.ask.plot(drawstyle='steps', color='g')
plt.title("Intraday best bid vs best ask")
plt.grid()


Getting broker statistics


In [18]:
brokers = stl.get_broker_stats()

In [19]:
brokers.tail()


Out[19]:
index broker sold sold_trades bought bought_trades total positive
31 31 CGML 16385.0 21.0 8201.0 26.0 -8184.0 False
32 32 INT 109650.0 75.0 41698.0 80.0 -67952.0 False
33 33 DBL 88668.0 39.0 15890.0 24.0 -72778.0 False
34 34 MSI 360661.0 511.0 219501.0 416.0 -141160.0 False
35 35 UBS 297021.0 503.0 16762.0 47.0 -280259.0 False

Plotting brokers net sold/bought


In [20]:
brokers['sold'] = -1*brokers['sold']
brokers[['total']].plot(kind='barh', color=brokers['positive'].map({True: 'g', False: 'r'}))
plt.yticks( range(brokers.total.count()), brokers.broker)
plt.show()


Plotting sold and bought per broker


In [21]:
brokers[['sold', 'bought']].plot(kind='barh', color=['r', 'g'])
plt.yticks( range(brokers.total.count()), brokers.broker)
#ax1.text(0, 0, stock_sell_buy['broker'], horizontalalignment=align, verticalalignment='center', color=clr, weight='bold') 
plt.show()