twitter: @adamhajari

github: github.com/adamhajari/spyre

this notebook: http://bit.ly/pydata2015_spyre

Before we start

make sure you have the latest version of spyre

pip install --upgrade dataspyre

there have been recent changes to spyre, so if you installed more than a day ago, go ahead and upgrade

Who Am I?

Adam Hajari
Data Scientist on the Next Big Sound team at Pandora
adam@nextbigsound.com
@adamhajari

Simple Interactive Web Applications with Spyre

Spyre is a web application framework for turning static data tables and plots into interactive web apps. Spyre was motivated by Shiny, a similar framework for R created by the developers of Rstudio.

Where does Spyre Live?

Installing Spyre

Spyre depends on:

  • cherrypy (server and backend)
  • jinja2 (html and javascript templating)
  • matplotlib (displaying plots and images)
  • pandas (for working within tabular data)

Assuming you don't have any issues with the above dependencies, you can install spyre via pip:

$ pip install dataspyre

Launching a Spyre App

Spyre's server module has a App class that every Spyre app will needs to inherit. Use the app's launch() method to deploy your app.


In [ ]:
from spyre import server

class SimpleApp(server.App):
    title = "Simple App"

app = SimpleApp()
app.launch()  # launching from ipython notebook is not recommended

If you put the above code in a file called simple_app.py you can launch the app from the command line with

$ python simple_app.py

Make sure you uncomment the last line first.

A Very Simple Example

There are two variables of the App class that need to be overridden to create the UI for a Spyre app: inputs and outputs (a third optional type called controls that we'll get to later). All three variables are lists of dictionaries which specify each component's properties. For instance, to create a text box input, overide the App's inputs variable:


In [ ]:
from spyre import server

class SimpleApp(server.App):
    inputs = [{ "type":"text",
                "key":"words",
                "label": "write here",
                "value":"hello world"}]

app = SimpleApp()
app.launch()

Now let's add an output. We first need to list all our out outputs and their attributes in the outputs dictionary.


In [ ]:
from spyre import server

class SimpleApp(server.App):
    inputs = [{ "type":"text",
                "key":"words",
                "label": "write here",
                "value":"hello world"}]
    
    outputs = [{"type":"html",
                "id":"some_html"}]

app = SimpleApp()
app.launch()

To generate the output, we can override a server.App method specific to that output type. In the case of html output, we overide the getHTML method. Each output method should return an object specific to that output type. In the case of html output, we just return a string.


In [ ]:
from spyre import server

class SimpleApp(server.App):
    title = "Simple App"
    
    inputs = [{ "type":"text",
                "key":"words",
                "label": "write here",
                "value":"hello world"}]
    
    outputs = [{"type":"html",
                "id":"some_html"}]

    def getHTML(self, params):
        words = params['words']
        return "here are the words you wrote: <b>%s</b>"%words

app = SimpleApp()
app.launch()

Great. We've got inputs and outputs, but we're not quite finished. As it is, the content of our output is static. That's because the output doesn't know when it needs to get updated. We can fix this in one of two ways:

  1. We can add a button to our app and tell our output to update whenever the button is pressed.
  2. We can add an action_id to our input that references the output that we want refreshed when the input value changes.

Let's see what the first approach looks like.


In [ ]:
from spyre import server

class SimpleApp(server.App):
    title = "Simple App"
    
    inputs = [{ "type":"text",
                "key":"words",
                "label": "write here",
                "value":"hello world"}]
    
    outputs = [{"type":"html",
                "id":"some_html",
                "control_id":"button1"}]
    
    controls = [{"type":"button",
                 "label":"press to update",
                 "id":"button1"}]

    def getHTML(self, params):
        words = params['words']
        return "here are the words you wrote: <b>%s</b>"%words

app = SimpleApp()
app.launch()

Our app now has a button with id "button1", and our output references our control's id, so that when we press the button we update the output with the most current input values.

Is a button a little overkill for this simple app? Yeah, probably. Let's get rid of it and have the output update just by changing the value in the text box. To do this we'll add an action_id attribute to our input dictionary that references the output's id.


In [ ]:
from spyre import server

class SimpleApp(server.App):
    title = "Simple App"
    
    inputs = [{ "type":"text",
                "key":"words",
                "label": "write here",
                "value":"look ma, no buttons",
                "action_id":"some_html"}]
    
    outputs = [{"type":"html",
                "id":"some_html"}]
    
    def getHTML(self, params):
        words = params['words']
        return "here are the words you wrote: <b>%s</b>"%words

app = SimpleApp()
app.launch()

Now the output gets updated with a change to the input.

Another Example

Let's suppose you've written a function to grab historical stock price data from the web. Your function returns a pandas dataframe.


In [1]:
%pylab inline
from googlefinance.client import get_price_data

def getData(params):
    ticker = params['ticker']
    if ticker == 'empty':
        ticker = params['custom_ticker'].upper()

    xchng = "NASD"
    param = {
        'q': ticker,  # Stock symbol (ex: "AAPL")
        'i': "86400",  # Interval size in seconds ("86400" = 1 day intervals)
        'x': xchng,  # Stock exchange symbol on which stock is traded (ex: "NASD")
        'p': "3M"  # Period (Ex: "1Y" = 1 year)
    }
    df = get_price_data(param)
    return df.drop('Volume', axis=1)

params = {'ticker':'GOOG'}
df = getData(params)
df.head()


Populating the interactive namespace from numpy and matplotlib
Out[1]:
Open High Low Close
2018-01-29 16:00:00 1176.48 1186.89 1171.9800 1175.58
2018-01-30 16:00:00 1167.83 1176.52 1163.5200 1163.69
2018-01-31 16:00:00 1170.57 1173.00 1159.1300 1169.94
2018-02-01 16:00:00 1162.61 1174.00 1157.5200 1167.70
2018-02-02 16:00:00 1122.00 1123.07 1107.2779 1111.90

Let's turn this into a spyre app. We'll use a dropdown menu input this time and start by displaying the data in a table. In the previous example we overrode the getHTML method and had it return a string to generate HTML output. To get a table output we need to override the getData method and have it return a pandas dataframe (conveniently, we've already done that!)


In [ ]:
from spyre import server
from googlefinance.client import get_price_data

server.include_df_index = True


class StockExample(server.App):
    title = "Historical Stock Prices"

    inputs = [{
        "type": 'dropdown',
        "label": 'Company',
        "options": [
            {"label": "Google", "value": "GOOG"},
            {"label": "Amazon", "value": "AMZN"},
            {"label": "Apple", "value": "AAPL"}
        ],
        "key": 'ticker',
        "action_id": "table_id"
    }]

    outputs = [{
        "type": "table",
        "id": "table_id"
    }]

    def getData(self, params):
        ticker = params['ticker']
        xchng = "NASD"
        param = {
            'q': ticker,  # Stock symbol (ex: "AAPL")
            'i': "86400",  # Interval size in seconds ("86400" = 1 day intervals)
            'x': xchng,  # Stock exchange symbol on which stock is traded (ex: "NASD")
            'p': "3M"  # Period (Ex: "1Y" = 1 year)
        }
        df = get_price_data(param)
        return df.drop('Volume', axis=1)


app = StockExample()
app.launch()

One really convenient feature of pandas is that you can plot directly from a dataframe using the plot method.


In [ ]:
df.plot()

Let's take advantage of this convenience and add a plot to our app. To generate a plot output, we need to add another dictionary to our list of outputs.


In [ ]:
from spyre import server
from googlefinance.client import get_price_data

server.include_df_index = True


class StockExample(server.App):
    title = "Historical Stock Prices"

    inputs = [{
        "type": 'dropdown',
        "label": 'Company',
        "options": [
            {"label": "Google", "value": "GOOG"},
            {"label": "Amazon", "value": "AMZN"},
            {"label": "Apple", "value": "AAPL"}
        ],
        "key": 'ticker',
    }]

    outputs = [{
        "type": "plot",
        "id": "plot",
        "control_id": "update_data"
    }, {
        "type": "table",
        "id": "table_id",
        "control_id": "update_data"
    }]

    controls = [{
        "type": "button",
        "label": "get stock data",
        "id": "update_data"
    }]

    def getData(self, params):
        ticker = params['ticker']
        xchng = "NASD"
        param = {
            'q': ticker,  # Stock symbol (ex: "AAPL")
            'i': "86400",  # Interval size in seconds ("86400" = 1 day intervals)
            'x': xchng,  # Stock exchange symbol on which stock is traded (ex: "NASD")
            'p': "3M"  # Period (Ex: "1Y" = 1 year)
        }
        df = get_price_data(param)
        return df.drop('Volume', axis=1)


app = StockExample()
app.launch()

Notice that we didn't have to add a new method for our plot output. getData is pulling double duty here serving the data for our table and our plot. If you wanted to alter the data or the plot object, you could do that by overriding the getPlot method. Under the hood, if you don't specify a getPlot method for your plot output, server.App's built-in getPlot method will look for a getData method, and just return the result of calling the plot() method on its dataframe.


In [ ]:
from spyre import server
from googlefinance.client import get_price_data

server.include_df_index = True


class StockExample(server.App):
    title = "Historical Stock Prices"

    inputs = [{
        "type": 'dropdown',
        "label": 'Company',
        "options": [
            {"label": "Google", "value": "GOOG"},
            {"label": "Amazon", "value": "AMZN"},
            {"label": "Apple", "value": "AAPL"}
        ],
        "key": 'ticker',
    }]

    outputs = [{
        "type": "plot",
        "id": "plot",
        "control_id": "update_data"
    }, {
        "type": "table",
        "id": "table_id",
        "control_id": "update_data"
    }]

    controls = [{
        "type": "button",
        "label": "get stock data",
        "id": "update_data"
    }]

    def getData(self, params):
        ticker = params['ticker']
        xchng = "NASD"
        param = {
            'q': ticker,  # Stock symbol (ex: "AAPL")
            'i': "86400",  # Interval size in seconds ("86400" = 1 day intervals)
            'x': xchng,  # Stock exchange symbol on which stock is traded (ex: "NASD")
            'p': "3M"  # Period (Ex: "1Y" = 1 year)
        }
        df = get_price_data(param)
        return df.drop('Volume', axis=1)

    def getPlot(self, params):
        df = self.getData(params)
        plt_obj = df.plot()
        plt_obj.set_ylabel("Price")
        plt_obj.set_xlabel("Date")
        plt_obj.set_title(params['ticker'])
        return plt_obj


app = StockExample()
app.launch()

Finally we'll put each of the outputs in separate tabs and add an action_id to the dropdown input that references the "update_data" control. Now, a change to the input state triggers the button to be "clicked". This makes the existence of a "button" supurfluous, so we'll change the control type to "hidden"


In [ ]:
from spyre import server
from googlefinance.client import get_price_data

server.include_df_index = True


class StockExample(server.App):
    title = "Historical Stock Prices"

    inputs = [{
        "type": 'dropdown',
        "label": 'Company',
        "options": [
            {"label": "Google", "value": "GOOG"},
            {"label": "Amazon", "value": "AMZN"},
            {"label": "Apple", "value": "AAPL"}
        ],
        "key": 'ticker',
        "action_id": "update_data"
    }]

    tabs = ["Plot", "Table"]

    outputs = [{
        "type": "plot",
        "id": "plot",
        "control_id": "update_data",
        "tab": "Plot"
    }, {
        "type": "table",
        "id": "table_id",
        "control_id": "update_data",
        "tab": "Table"
    }]

    controls = [{
        "type": "hidden",
        "label": "get stock data",
        "id": "update_data"
    }]

    def getData(self, params):
        ticker = params['ticker']
        xchng = "NASD"
        param = {
            'q': ticker,  # Stock symbol (ex: "AAPL")
            'i': "86400",  # Interval size in seconds ("86400" = 1 day intervals)
            'x': xchng,  # Stock exchange symbol on which stock is traded (ex: "NASD")
            'p': "3M"  # Period (Ex: "1Y" = 1 year)
        }
        df = get_price_data(param)
        return df.drop('Volume', axis=1)

    def getPlot(self, params):
        df = self.getData(params)
        plt_obj = df.plot()
        plt_obj.set_ylabel("Price")
        plt_obj.set_xlabel("Date")
        plt_obj.set_title(params['ticker'])
        return plt_obj


app = StockExample()
app.launch()

A few more things you can try

  • there's a "download" output type that uses either the getData method or a getDownload method
  • tables can be sortable. Just add a "sortable" key to the table output dictionary and set it's value to true
  • there are a couple of great Python libraries that produce JavaScript plots (Bokeh and Vincent). You can throw them into a getHTML method to add JavaScript plots to your spyre app (hoping to add a "bokeh" output type soon to make this integration a little easier).
  • you can link input values

Deploying

More Examples On GitHub

A couple of tricks

  • you can either name your output methods using the getType convention or you can have the name match the output id. This is useful if you've got multiple outputs of the same type.
  • if multiple outputs use the same data and it takes a long time to generate that data, there's a trick for caching data so you only have to load it once. See the stocks_example app in the examples directory of the git repo to see how (Warning: it's kind of hacky)