Backtesting with TWP backtesting module

Strategy simulation often includes going through same steps : determining entry and exit moments, calculating number of shares capital and pnl. To limit the number of lines of code needed to perform a backtest, the twp library includes a simple backtesting module.

The backtest module is a very simple version of a vectorized backtester. It can be used as a stand-alone module without the rest of the tradingWithPython library.

All of the functionality is accessible through the Backtest class, which will be demonstrated here.

The backtester needs an instrument price and entry/exit signals to do its job. Let's start by creating simple data series to test it.


In [2]:
import pandas as pd
import tradingWithPython as twp
assert twp.__version__ >= '0.0.12' , 'Please update your twp module '

# create example signals for demonstrating backterster functionality

price = pd.Series(arange(10)) # price, just a linear incrementing series
signal = pd.Series(index=price.index) # strategy signal, in $

signal[2] = 100 # go long 100$ 
signal[3] = 0  # exit

signal[6] = -100 # go short 100$
signal[8] = 0 # exit


pd.DataFrame({'price':price,'signal':signal}) # show price and signal as a table.


Out[2]:
price signal
0 0 NaN
1 1 NaN
2 2 100
3 3 0
4 4 NaN
5 5 NaN
6 6 -100
7 7 NaN
8 8 0
9 9 NaN

The Backtest class is created with these arguments:

  • price Series with instrument price.
  • signal Series with capital to invest (long+,short-) or number of shares.
  • sitnalType capital to bet or number of shares 'capital' mode is default.
  • initialCash starting cash. 0 is default

The signal can be either capital to trade or number of shares to buy/sell. 'capital' is the default mode, to use number of shares, use signalType='shares'

All calculations are perfromed at class creation.

Results of the calculations can be acessed as follows:

  • Backtest.data full data table, excel-like. pnl column is cumulative
  • Backtest.pnl easy access to pnl column of the data table
  • Backtest.sharpe Sharpe of the pnl

In [5]:
# use price and signal to perform a backtest
       
bt = twp.Backtest(price,signal) # create class, simulate
bt.plotTrades() # plot price and trades (short=red markers, long=green markers)

figure()
bt.pnl.plot(style='x-') # plot pnl
title('pnl')
print 'Sharpe: ', bt.sharpe

# normally, daily change in shares (delta) is not included in the Backtest.data .
# Sometimes it is handy to have it, for adding transaction cost for example. 
# You can easily add it :
bt.data['delta'] = bt.data['shares'].diff().fillna(0)

bt.data # output strategy data


Sharpe:  2.17999027863
Out[5]:
price shares value cash pnl delta
0 0 0 0 0 0 0
1 1 0 0 0 0 0
2 2 50 100 -100 0 50
3 3 0 0 50 50 -50
4 4 0 0 50 50 0
5 5 0 0 50 50 0
6 6 -17 -102 152 50 -17
7 7 -17 -119 152 33 0
8 8 0 0 16 16 17
9 9 0 0 16 16 0

Test on real price data

Now let's use the backtester on some real price


In [249]:
# start with getting some data to test on
import tradingWithPython.lib.yahooFinance as yahoo # yahoo finance module
stockname = 'TSLA'
n = 30*12
price = yahoo.getHistoricData(stockname)#['adj_close'][-n:]
#price.plot()


Got 1196 days of data

normally a buy/sell signal would be generated by a strategy. For simplicity I'll just simulate holding a position between two dates


In [3]:
import pandas as pd
import tradingWithPython as twp
import tradingWithPython.lib.yahooFinance as yahoo # yahoo finance module
stockname = 'SCTY' #'SCTY' # 'TSLA' #'SCTY' #'TSLA''GS''NFLX' 'AMZN'
n = (5*4)*12
# adj_close   close    high     low    open    volume
price = yahoo.getHistoricData(stockname)['open'][-n:]


Got 579 days of data

In [4]:
figsize(15,9)
initialCash=25000
min_stock = 30
# sumulate two trades
strategy = pd.Series(index=price.index) #init signal series

# while setting values this way, make sure that the dates are actually in the index
#strategy[pd.datetime(2008,1,3)] = -10000 # go short 
#strategy[pd.datetime(2009,1,5)] = 10000 # go long 10k$ on this day
#strategy[pd.datetime(2013,1,8)] = 0 # close position

# manual Strategy
if False:
    strategy[price.index[50]] = -10000 # go short 
    strategy[price.index[100]] = 10000 # go long 10k$ on this day
    strategy[price.index[150]] = 0 # close position
# automatic strategy
else:
    # find when to buy and sell
    import trendy
    reload(trendy)
    import tradingWithPython as twp
    #order=trendy.iterlines(price, window = 30, charts = True)
    a,b,c,d, order = trendy.segtrends(price, segments = n/5, charts = True, momentum=True); 
    #print order
     # force sell at the end
    order[-1] = 0
    title('automatic strategy')
    order=[el*min_stock for el in order]
    # create a stratgy from order
    for i, idx in enumerate(price.index):
        if order[i]!=0:
            strategy[idx] = order[i]
   
# do the backtest
bt = twp.Backtest(price, strategy, initialCash=initialCash, signalType='shares')
bt.plotTrades()

figure()
bt.pnl.plot()
title('pnl')

figure()
bt.data.plot()
title('all strategy data')
#bt.data


Out[4]:
<matplotlib.text.Text at 0x7fbf9d759890>
<matplotlib.figure.Figure at 0x7fbf9d7aaf10>

In [267]:
print order


[0.0, -30.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, 60.0, -30.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, 60.0, -30.0, -60.0, 30.0, 60.0, -30.0, 30.0, 60.0, -30.0, -60.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, -30.0, -60.0, 30.0, 60.0, -30.0, 30.0, 60.0, -30.0, -60.0, -60.0, 30.0, 60.0, -30.0, 30.0, -30.0, 30.0, -30.0, -60.0, -60.0, 30.0, 60.0, 60.0, 60.0, 60.0, -30.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, -30.0, 30.0, -30.0, 30.0, -30.0, -60.0, 30.0, -30.0, 30.0, -30.0, 30.0, 60.0, -30.0, -60.0, -60.0, -60.0, -60.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, 60.0, -30.0, 30.0, 60.0, 60.0, -30.0, -60.0, -60.0, -60.0, 30.0, -30.0, 30.0, -30.0, -60.0, 30.0, 60.0, -30.0, -60.0, -60.0, 30.0, 60.0, -30.0, -60.0, -60.0, 30.0, -30.0, 30.0, -30.0, -60.0, -60.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, 60.0, -30.0, -60.0, -60.0, 30.0, -30.0, -60.0, 30.0, 60.0, -30.0, 30.0, -30.0, -60.0, -60.0, -60.0, 30.0, -30.0, 30.0, 60.0, -30.0, -60.0, 30.0, -30.0, -60.0, 30.0, 60.0, -30.0, 30.0, -30.0, -60.0, -60.0, 30.0, 60.0, 60.0, -30.0, -60.0, 30.0, 60.0, -30.0, -60.0, 30.0, -30.0, 30.0, 60.0, -30.0, -60.0, -60.0, -60.0, 30.0, 60.0, -30.0, -60.0, -60.0, -60.0, 30.0, 60.0, -30.0, -60.0, 30.0, 60.0, -30.0, 30.0, -30.0, -60.0, -60.0, 30.0, -30.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 60.0, -30.0, -60.0, -60.0, -60.0, -60.0, -60.0, -60.0, 30.0, -30.0, -60.0, 30.0, 60.0, -30.0, -60.0, 30.0, 60.0, 60.0, -30.0, 30.0, 60.0, -30.0, -60.0, -60.0, 30.0, 60.0, -30.0, -60.0, -60.0, -60.0, -60.0, 30.0, -30.0, -60.0, -60.0, -60.0, -60.0, 30.0, 60.0, 0.0]

In [222]:
bt.data


Out[222]:
price shares value cash pnl
2014-04-16 323.68 0 0.0 25000.0 0.0
2014-04-17 324.91 -30 -9747.3 34747.3 0.0
2014-04-21 330.87 -60 -19852.2 44673.4 -178.8
2014-04-22 329.32 -60 -19759.2 44673.4 -85.8
2014-04-23 324.58 -60 -19474.8 44673.4 198.6
2014-04-24 337.15 -60 -20229.0 44673.4 -555.6
2014-04-25 303.83 30 9114.9 17328.7 1443.6
2014-04-28 296.58 60 17794.8 8431.3 1226.1
2014-04-29 300.38 -30 -9011.4 35465.5 1454.1
2014-04-30 304.13 -60 -18247.8 44589.4 1341.6
2014-05-01 307.89 -60 -18473.4 44589.4 1116.0
2014-05-02 308.01 -60 -18480.6 44589.4 1108.8
2014-05-05 310.05 -60 -18603.0 44589.4 986.4
2014-05-06 297.38 -60 -17842.8 44589.4 1746.6
2014-05-07 292.71 30 8781.3 18245.5 2026.8
2014-05-08 288.32 60 17299.2 9595.9 1895.1
2014-05-09 292.24 60 17534.4 9595.9 2130.3
2014-05-12 302.86 -30 -9085.8 36853.3 2767.5
2014-05-13 304.64 30 9139.2 18574.9 2714.1
2014-05-14 297.62 60 17857.2 9646.3 2503.5
2014-05-15 295.19 60 17711.4 9646.3 2357.7
2014-05-16 297.70 60 17862.0 9646.3 2508.3
2014-05-19 296.76 60 17805.6 9646.3 2451.9
2014-05-20 301.19 60 18071.4 9646.3 2717.7
2014-05-21 305.01 60 18300.6 9646.3 2946.9
2014-05-22 304.91 60 18294.6 9646.3 2940.9
2014-05-23 312.24 -30 -9367.2 37747.9 3380.7
2014-05-27 310.82 -60 -18649.2 47072.5 3423.3
2014-05-28 310.16 30 9304.8 19158.1 3462.9
2014-05-29 313.78 -30 -9413.4 37984.9 3571.5
... ... ... ... ... ...
2015-02-17 375.43 -60 -22525.8 68432.5 20906.7
2015-02-18 373.37 -60 -22402.2 68432.5 21030.3
2015-02-19 379.00 -60 -22740.0 68432.5 20692.5
2015-02-20 383.66 -60 -23019.6 68432.5 20412.9
2015-02-23 380.14 -60 -22808.4 68432.5 20624.1
2015-02-24 378.59 -60 -22715.4 68432.5 20717.1
2015-02-25 385.37 -60 -23122.2 68432.5 20310.3
2015-02-26 384.80 -60 -23088.0 68432.5 20344.5
2015-02-27 380.16 -60 -22809.6 68432.5 20622.9
2015-03-02 385.66 -60 -23139.6 68432.5 20292.9
2015-03-03 384.61 -60 -23076.6 68432.5 20355.9
2015-03-04 382.72 -60 -22963.2 68432.5 20469.3
2015-03-05 387.83 -60 -23269.8 68432.5 20162.7
2015-03-06 380.09 -60 -22805.4 68432.5 20627.1
2015-03-09 378.56 -60 -22713.6 68432.5 20718.9
2015-03-10 369.51 30 11085.3 35176.6 21261.9
2015-03-11 366.37 60 21982.2 24185.5 21167.7
2015-03-12 374.24 -30 -11227.2 57867.1 21639.9
2015-03-13 370.58 -60 -22234.8 68984.5 21749.7
2015-03-16 373.35 -60 -22401.0 68984.5 21583.5
2015-03-17 371.92 -60 -22315.2 68984.5 21669.3
2015-03-18 375.14 -60 -22508.4 68984.5 21476.1
2015-03-19 373.24 -60 -22394.4 68984.5 21590.1
2015-03-20 378.49 -60 -22709.4 68984.5 21275.1
2015-03-23 375.11 -60 -22506.6 68984.5 21477.9
2015-03-24 374.09 -60 -22445.4 68984.5 21539.1
2015-03-25 370.96 -60 -22257.6 68984.5 21726.9
2015-03-26 367.35 -60 -22041.0 68984.5 21943.5
2015-03-27 370.56 -60 -22233.6 68984.5 21750.9
2015-03-30 374.59 -60 -22475.4 68984.5 21509.1

240 rows × 5 columns


In [94]:
reload(trendy)
price = price[:]
# Generate general support/resistance trendlines and show the chart
# winow < 1 is considered a fraction of the length of the data set
#trendy.gentrends(price, window = 1.0/3, charts = True);

# Generate a series of support/resistance lines by segmenting the price history
a,b,c,d, order = trendy.segtrends(price, segments = n/10, charts = True);  # equivalent to gentrends with window of 1/2
#print order
#trendy.segtrends(price, segments = 5, charts = True);  # plots several S/R lines

# Generate smaller support/resistance trendlines to frame price over smaller periods
#trendy.minitrends(price, window = 10, charts = True)

# Iteratively generate trading signals based on maxima/minima in given window
#trendy.iterlines(price, window = 60, charts = True);  # buy at green dots, sell at red dots


[ 0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0. -1. -1. -1.
 -1. -1.  1.  1.  0.  1.  0.  0.  0.  0.  0.  0.  0.  1.  1.  1.  0.  0.
  1.  1.  1.  1.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.
  1.  1.  1.  1.  1.  1.  0.  1.  1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
 -1.  1.  1.  0.  1.  0.  0.  1.  0.  0.  0.  0.  0.  0.  1.  1.  1.  1.
  1.  1.  1.  1.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  1.  1.  0.
  1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
 -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.  1.  1.  1.
  1.  0.  0.  0.  0.  0.  0.  1.  0.  1.  0.  0.  0.  0.  1.  1.  1.  0.
  1.  1.  1.  1.  1.  1.  1. -1.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.
  0.  1.  1.  1.  0.  0.  1.  0.  1. -1. -1.  1.  1.  0.  0.  0.  0.  1.
  1.  1.  0.  0.  0.  0.  0.  0.  0.  0.  1.  1.  1.  1.  0.  0.  0.  1.
  1.  0.  0. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
 -1. -1. -1. -1. -1. -1.]

In [9]:
print pd.__version__
print np.__version__
print matplotlib.__version__


0.16.0
1.9.2
1.4.0

In [7]:



Out[7]:
'1.9.2'

In [ ]: