In this notebook, we'll take a look at Estimizes's Analyst-by-Analyst Estimates dataset, available on the Quantopian. This dataset spans January, 2010 through the current day.
This data contains a record for every estimate made by an individual on the Estimize product. By comparison, the Estimize Revisions product provides rolled-up consensus numbers for each possible earnings announcement.
In this notebook, we'll examine these detailed estimates and pull in that consensus data as well.
Before we dig into the data, we want to tell you about how you generally access Quantopian Store data sets. These datasets are available using the Blaze library. Blaze provides the Quantopian user with a convenient interface to access very large datasets.
Blaze provides an important function for accessing these datasets. Some of these sets are many millions of records. Bringing that data directly into Quantopian Research directly just is not viable. So Blaze allows us to provide a simple querying interface and shift the burden over to the server side.
To learn more about using Blaze and generally accessing Quantopian Store data, clone this tutorial notebook.
A few key caveats:
1) We limit the number of results returned from any given expression to 10,000 to protect against runaway memory usage. To be clear, you have access to all the data server side. We are limiting the size of the responses back from Blaze.
2) There is a free version of this dataset as well as a paid one. The free one includes about three years of historical data, though not up to the current day.
With preamble in place, let's get started:
In [1]:
# import the free sample of the dataset
from quantopian.interactive.data.estimize import estimates_free
# or if you want to import the full dataset, use:
# from quantopian.interactive.data.estimize import estimates
# import data operations
from odo import odo
# import other libraries we will use
import pandas as pd
import matplotlib.pyplot as plt
In [2]:
# Let's use blaze to understand the data a bit using Blaze dshape()
estimates_free.dshape
Out[2]:
In [3]:
# And how many rows are in this free sample?
# N.B. we're using a Blaze function to do this, not len()
estimates_free.count()
Out[3]:
In [4]:
# Let's see what the data looks like. We'll grab the first three rows.
estimates_free.head(3)
Out[4]:
Let's go over the columns:
asof_datefiscal_yearfiscal_quarterasof_dateWe've done much of the data processing for you. Fields like asof_date and sid are standardized across all our Store Datasets, so the datasets are easy to combine. We have standardized the sid across all our equity databases.
We can select columns and rows with ease. Below, let's just look at the estimates made for TSLA for a particular quarter. Also, we're filtering out some spurious data:
In [5]:
stocks = symbols('TSLA')
one_quarter = estimates_free[(estimates_free.sid == stocks.sid) &
(estimates_free.fiscal_year == '2014') &
(estimates_free.fiscal_quarter == '1') &
(estimates_free.eps < 100)
]
one_quarter.head(5)
Out[5]:
How many records do we have now?
In [6]:
one_quarter.count()
Out[6]:
Let's break it down by user:
In [7]:
one_quarter.username.count_values()
Out[7]:
Let's convert it over to a Pandas DataFrame so we can chart it and examine it closer
In [31]:
one_q_df = odo(one_quarter.sort('asof_date'), pd.DataFrame)
plt.plot(one_q_df.asof_date, one_q_df.eps, marker='.', linestyle='None', color='r')
plt.xlabel("As Of Date (asof_date)")
plt.ylabel("EPS Estimate")
plt.title("Analyst by Analyst EPS Estimates for TSLA")
plt.legend(["Individual Estimate"], loc=2)
Out[31]:
That's neat. But let's add in some data from another dataset -- the Estimize Revisions data. For the same timeframe, the revisions data provides each revision to the overall consensus estimates. So where estimates_free data provides every single estimate made by an individual on the Estimize site, revisions_free provides rolled up summaries of the estimates.
In [32]:
from quantopian.interactive.data.estimize import revisions_free
consensus = revisions_free[(revisions_free.sid == stocks.sid) &
(revisions_free.fiscal_year == '2014') &
(revisions_free.fiscal_quarter == '1') &
(revisions_free.source == 'estimize') &
(revisions_free.metric == 'eps')
]
In [33]:
consensus.head(3)
Out[33]:
For this quick demonstration, let's just grab the consensus mean from the revisions_free data set and convert it over to Pandas. Note, we need to rename the mean column name because it causes problems otherwise:
In [13]:
consensus_df = odo(consensus[['asof_date', 'mean']].sort('asof_date'), pd.DataFrame)
consensus_df.rename(columns={'mean':'eps_mean'}, inplace=True)
Let's chart that in the same chart again so we get a trend of the mean over time, overlayed on a chart of each individual analyst estimate:
In [30]:
plt.plot(consensus_df.asof_date, consensus_df.eps_mean)
plt.plot(one_q_df.asof_date, one_q_df.eps, marker='.', linestyle='None', color='r')
plt.xlabel("As Of Date (asof_date)")
plt.ylabel("EPS Estimate")
plt.title("EPS Estimates for TSLA")
plt.legend(["Mean Estimate", "Individual Estimates"], loc=2)
Out[30]: