Pipeline Example


In [1]:
from quantopian.pipeline import Pipeline
from quantopian.research import run_pipeline
from quantopian.pipeline.data.builtin import USEquityPricing

Getting the Securities we want.

The Q500US and Q1500US

These gropus of tradeable stocks are refered to as "universes", because all your trades will use these stocks as their "Universe" of available stock, they won't be trading with anything outside these groups.


In [16]:
from quantopian.pipeline.filters import Q1500US

There are two main benefits of the Q500US and Q1500US. Firstly, they greatly reduce the risk of an order not being filled. Secondly, they allow for more meaningful comparisons between strategies as now they will be used as the standard universes for algorithms.


In [17]:
universe = Q1500US()

Filtering the universe further with Classifiers

Let's only grab stocks in the energy sector: https://www.quantopian.com/help/fundamentals#industry-sector


In [25]:
from quantopian.pipeline.data import morningstar

In [29]:
sector = morningstar.asset_classification.morningstar_sector_code.latest

Alternative:


In [30]:
#from quantopian.pipeline.classifiers.morningstar import Sector
#morningstar_sector = Sector()

In [36]:
energy_sector = sector.eq(309)

Masking Filters

Masks can be also be applied to methods that return filters like top, bottom, and percentile_between.

Masks are most useful when we want to apply a filter in the earlier steps of a combined computation. For example, suppose we want to get the 50 securities with the highest open price that are also in the top 10% of dollar volume.

Suppose that we then want the 90th-100th percentile of these securities by close price. We can do this with the following:


In [38]:
from quantopian.pipeline.factors import SimpleMovingAverage, AverageDollarVolume

In [37]:
# Dollar volume factor
dollar_volume = AverageDollarVolume(window_length = 30)

# High dollar volume filter
high_dollar_volume = dollar_volume.percentile_between(90, 100)

# Top open price filter (high dollar volume securities)
top_open_price = USEquityPricing.open.latest.top(50, 
                                                 mask = high_dollar_volume)

# Top percentile close price filter (high dollar volume, top 50 open price)
high_close_price = USEquityPricing.close.latest.percentile_between(90, 100, 
                                                                   mask = top_open_price)

Applying Filters and Factors

Let's apply our own filters, following along with some of the examples above. Let's select the following securities:

  • Stocks in Q1500US
  • Stocks that are in the energy Sector
  • They must be relatively highly traded stocks in the market (by dollar volume traded, need to be in the top 5% traded)

Then we'll calculate the percent difference as we've done previously. Using this percent difference we'll create an unsophisticated strategy that shorts anything with negative percent difference (the difference between the 10 day mean and the 30 day mean).


In [73]:
def make_pipeline():
    
    # Base universe filter.
    base_universe = Q1500US()
    
    # Sector Classifier as Filter
    energy_sector = sector.eq(309)
    
    # Masking Base Energy Stocks
    base_energy = base_universe & energy_sector
    
    # Dollar volume factor
    dollar_volume = AverageDollarVolume(window_length = 30)

    # Top half of dollar volume filter
    high_dollar_volume = dollar_volume.percentile_between(95, 100)
    
    # Final Filter Mask
    top_half_base_energy = base_energy & high_dollar_volume
    
    # 10-day close price average.
    mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], 
                                  window_length = 10, 
                                  mask = top_half_base_energy)

    # 30-day close price average.
    mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], 
                                  window_length = 30, 
                                  mask = top_half_base_energy)

    # Percent difference factor.
    percent_difference = (mean_10 - mean_30) / mean_30
    
    # Create a filter to select securities to short.
    shorts = percent_difference < 0
    
    # Create a filter to select securities to long.
    longs = percent_difference > 0
    
    # Filter for the securities that we want to trade.
    securities_to_trade = (shorts | longs)
    
    return Pipeline(
        columns = {
            'longs': longs,
            'shorts': shorts,
            'percent_diff':percent_difference
        },
        screen=securities_to_trade
    )

In [74]:
result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')
result


Out[74]:
longs percent_diff shorts
2015-05-05 00:00:00+00:00 Equity(216 [HES]) True 0.036528 False
Equity(448 [APA]) True 0.035926 False
Equity(455 [APC]) True 0.049153 False
Equity(858 [BHI]) True 0.033807 False
Equity(1746 [COG]) True 0.058078 False
Equity(2368 [DVN]) True 0.046264 False
Equity(2564 [EOG]) True 0.032102 False
Equity(2621 [ESV]) True 0.060197 False
Equity(3443 [HAL]) True 0.049257 False
Equity(3647 [HP]) True 0.040991 False
Equity(5035 [MRO]) True 0.061598 False
Equity(5213 [NBL]) True 0.010443 False
Equity(5214 [NBR]) True 0.064133 False
Equity(5249 [NE]) True 0.037559 False
Equity(5729 [OXY]) True 0.029776 False
Equity(6928 [SLB]) True 0.046555 False
Equity(7244 [SWN]) True 0.070788 False
Equity(7612 [ANDV]) True 0.005997 False
Equity(7990 [VLO]) False -0.017145 True
Equity(8214 [WMB]) True 0.018876 False
Equity(8347 [XOM]) True 0.017343 False
Equity(8461 [CHK]) True 0.014265 False
Equity(9038 [RIG]) True 0.048180 False
Equity(13176 [CAM]) True 0.082110 False
Equity(17436 [PXD]) True 0.010248 False
Equity(19249 [RRC]) True 0.087062 False
Equity(19336 [WFT]) True 0.049141 False
Equity(22784 [FTI]) True 0.054529 False
Equity(23112 [CVX]) True 0.018972 False
Equity(23998 [COP]) True 0.023902 False
Equity(24809 [NOV]) True 0.024940 False
Equity(25707 [WLL]) True 0.048205 False
Equity(33856 [CLR]) True 0.064304 False
Equity(34440 [CXO]) True 0.042184 False
Equity(39797 [OAS]) True 0.042388 False
Equity(40852 [KMI]) True 0.023016 False
Equity(41636 [MPC]) True 0.011952 False
Equity(42788 [PSX]) True 0.020911 False

In [75]:
result.info()


<class 'pandas.core.frame.DataFrame'>
MultiIndex: 38 entries, (2015-05-05 00:00:00+00:00, Equity(216 [HES])) to (2015-05-05 00:00:00+00:00, Equity(42788 [PSX]))
Data columns (total 3 columns):
longs           38 non-null bool
percent_diff    38 non-null float64
shorts          38 non-null bool
dtypes: bool(2), float64(1)
memory usage: 684.0+ bytes

Executing this Strategy in the IDE


In [ ]:
from quantopian.algorithm import attach_pipeline,pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.factors import AverageDollarVolume,SimpleMovingAverage
from quantopian.pipeline.filters.morningstar import Q1500US
from quantopian.pipeline.data import morningstar

def initialize(context):
    
    schedule_function(my_rebalance,date_rules.week_start(),time_rules.market_open(hours = 1))
    
    my_pipe = make_pipeline()
    attach_pipeline(my_pipe, 'my_pipeline')
    
def my_rebalance(context,data):
    for security in context.portfolio.positions:
        if security not in context.longs and security not in context.shorts and data.can_trade(security):
            order_target_percent(security,0)
            
    for security in context.longs:
        if data.can_trade(security):
            order_target_percent(security,context.long_weight)

    for security in context.shorts:
        if data.can_trade(security):
            order_target_percent(security,context.short_weight)




def my_compute_weights(context):
    
    if len(context.longs) == 0:
        long_weight = 0
    else:
        long_weight = 0.5 / len(context.longs)
  
    if len(context.shorts) == 0:
        short_weight = 0
    else:
        short_weight = 0.5 / len(context.shorts)
    
    return (long_weight,short_weight)






def before_trading_start(context,data):
    context.output = pipeline_output('my_pipeline')
    
    # LONG
    context.longs = context.output[context.output['longs']].index.tolist()
    
    # SHORT
    context.shorts = context.output[context.output['shorts']].index.tolist()


    context.long_weight,context.short_weight = my_compute_weights(context)



def make_pipeline():
    
    # Universe Q1500US
    base_universe = Q1500US()
    
    # Energy Sector
    sector = morningstar.asset_classification.morningstar_sector_code.latest
    energy_sector = sector.eq(309)
    
    # Make Mask of 1500US and Energy
    base_energy = base_universe & energy_sector
    
    # Dollar Volume (30 Days) Grab the Info
    dollar_volume = AverageDollarVolume(window_length = 30)
    
    # Grab the top 5% in avg dollar volume
    high_dollar_volume = dollar_volume.percentile_between(95, 100)
     
    # Combine the filters
    top_five_base_energy = base_energy & high_dollar_volume
    
    # 10 day mean close
    mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length = 10, mask = top_five_base_energy)
    
    # 30 day mean close
    mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length = 30, mask = top_five_base_energy)
    
    # Percent Difference
    percent_difference = (mean_10-mean_30)/mean_30
    
    # List of Shorts
    shorts = percent_difference < 0
    
    # List of Longs
    longs = percent_difference > 0
    
    # Final Mask/Filter for anything in shorts or longs
    securities_to_trade = (shorts | longs)
    
    # Return Pipeline
    return Pipeline(columns={
        'longs':longs,
        'shorts':shorts,
        'perc_diff':percent_difference
    },screen=securities_to_trade)