How to create dashboards using shaolin KungFu

1.1 Widgets defined as keyword arguments

It is possible to instantiate widgets by instantiating a KungFu object. The widgets will be created from ts keyword arguments as shown below.


In [1]:
from shaolin import KungFu
dashb = KungFu(int_slider=4, text="moco", dropdown=['Hello','World'],float_slider=(2.,10.,1.),box='2r')
dashb.widget


You can use the box parameter to set the layout distribution of the widgets. It is possible to set the desired number of rows or columns that our dashboard will have:


In [2]:
KungFu(int_slider=4, text="moco", dropdown=['Hello','World'],float_slider=(2.,10.,1.),box='1c')[0]


1.2 Instantiating from kwargs dict

It is possible to create a dashboard containing widgets that control the value of the desired keyword arguments as shown in the following cells. It is possible to use the same syntax that is used in the ipywidgets interact function, shaolin object notation syntax, or shaoscript syntax to define the widgets that will be displayed in the dashboard.


In [4]:
kwargs = {'num_xbins':(1,40,2,10),
         'num_ybins':(1,40,2,10),
         'grid': True,
         'link_bins':True,
         'plot_density':True,
         'reg_order':(1,10,1,1),
         'ci':"(1,100,1,95)$N=ci&d=Confidence intervals",
         'robust':[False],
         'scatter':True,
         'regression':True,
         'marginals':(['None','Histogram','KDE','Both'],'Both'),
         'bw':(0.001,10,0.05,0.5),
         'Title':'#Combining plots$N=title&D=',
         'dataset':['ALL','I','II','III','IV'],
         'save':False,
         #'@kernel':['gau','cos','biw','epa','tri','triw']
        }
kf = KungFu(**kwargs,box='4c|')# using '|' in the box description sorts the widtgets by its value type
kf[0]#indexing with any integer returns the widget



In [5]:
kf() #calling the dashboard returns a kwargs dict


Out[5]:
{'bw': 0.5,
 'ci': 95,
 'dataset': 'ALL',
 'grid': True,
 'link_bins': True,
 'marginals': 'Both',
 'num_xbins': 10,
 'num_ybins': 10,
 'plot_density': True,
 'reg_order': 1,
 'regression': True,
 'robust': False,
 'save': False,
 'scatter': True}

1.3 Instantiating from shaolin syntax

KungFu can be used the same way the Dashboard was used in former version of shaolin to manually define the layout.


In [6]:
from IPython.core.display import display
import seaborn as sns
%matplotlib inline
sns.set(style="ticks")
data = sns.load_dataset("anscombe")

In [7]:
#general
title = '#Exploratory plots$N=title&D='
marginals = "@['Both','None','Histogram','KDE']$d=Marginals"
dset = "@['ALL','I','II','III','IV']$D=Dataset"
x_cols = '@dd$D=X column&o='+str(data.columns.values.tolist())
y_cols = '@dd$D=Y column&o='+str(data.columns.values.tolist())+'&v='+data.columns[1]
save = "@[False]$D=Save plot&n=save"
data_layout = ['c$N=data_layout',[title,dset,['r$N=sub_row',[x_cols,y_cols]],marginals,save]]

#histograms
h_title = '#Historam options$N=h_title&D='
num_xbins = '@(1,40,2,10)$D=Num xbins&n=num_xbins'
num_ybins = '@(1,40,2,10)$D=Num ybins&n=num_ybins'
lbins = '@True$D=link_bins'
hist_layout = ['c$N=hist_layout',[h_title,num_xbins,num_ybins,lbins]]

#scatter
s_title = '#Scatterplot options$N=s_title&D='
scat = '@[True]$D=Plot Scatter&n=scatter' 
grid = '@True$D=Grid'
scatter_layout = ['c$N=scatter_layout',[s_title,scat,grid]]

#regression
r_title = '#Regression options$N=r_title&D='
reg = '@[True]$D=Plot Regression&n=regression'
robust = '@False$D=Robust'
reg_order = "@(1,10,1,1)$D=Reg order"
ci = "@(1,100,1,95)$N=ci&d=Confidence intervals"
reg_layout = ['c$N=reg_layout',[r_title,reg,reg_order,ci,robust]]

#kde
k_title = '#KDE plot options$N=k_title&D='
kde = '@[True]$D=Plot KDE&n=plot_density'
bw =  "@(0.001,10,0.05,0.5)$D=Bandwidth&n=bw"
kde_layout = ['@c$N=kde_layout',[k_title,kde,bw]]

#accordion as top level layout
dash = ['ac$N=dae_plot&t=General,Histogram,Scatter,KDE,Regression',[data_layout,
                                                                hist_layout,
                                                                scatter_layout,
                                                                kde_layout,
                                                                reg_layout,
                                                               ]
       ]

KungFu(dash=dash)[0]


Mimicking the interact function from ipywidgets

Lets see how we can copy the functionality of the interact function from ipywidgets. It is possible to use KungFu on a single function the same way interact would work. We can also create a Dashboard from the default arguments of a function.

Here we compare how we would use the ipywidgets interact function to build an interactive dashboard, and how we can achieve the same result with shaolin.


In [8]:
import numpy as np
from ipywidgets import interact, interactive, fixed
from IPython.core.display import clear_output,display
import matplotlib.pyplot as plt
import ipywidgets as widgets
import seaborn as sns
def draw_example_dist(n=10000):
    a = np.random.normal(loc=-15.,scale=3,size=n)
    b = np.random.normal(loc=20,scale=3,size=n)
    c = np.random.exponential(size=n,scale=3)
    return np.hstack([a,b,c])


def draw_regions(distribution='normal',
                 bins=7,
                 bw=0.18,
                 normed=False,
                 mean=False,
                 std=False,
                 percents=False,
                 hist=True):
   
    x = draw_example_dist() if distribution=='example_2' else np.random.standard_normal(10000)
    #x = draw_regions_data if distribution=='custom' else x
    fig = plt.figure(figsize=(14,8))
    ax = sns.kdeplot(x, cut=0,color='b',shade=True,alpha=0.3,bw=bw)
    
    d_bins = np.linspace(x.min(),x.max(),num=bins)
    if hist:
        n, bins, patches = ax.hist(x,bins=d_bins,normed=normed,rwidth=0.8,alpha=0.7)
    else:
        n=[1]
    maxx = 1 if normed else max(n)
    if mean:
        
        ax.axvline(x=x.mean(), ymin=0, ymax = maxx,
                    linewidth=6, color='r',label='Mean: {:.3f}'.format(x.mean()),alpha=1.)
    if std:
        m = x.mean()
        
        ax.axvline(x=m+x.std(), ymin=0, ymax = maxx, linewidth=5, color='g',label='Std: {:.3f}'.format(x.std()),alpha=0.8)
        ax.axvline(x=m-x.std(), ymin=0, ymax = maxx, linewidth=5, color='g',alpha=0.8)
    if percents:
        d = pd.Series(x).describe()
        ax.axvline(x=d.ix['min'],
                    ymin=0, ymax = maxx,
                    linewidth=5, 
                    color=(0.19130826141258903, 0.13147472185630074, 0.09409307479747722),
                    label='min: {:.2f}'.format(d.ix['min']),alpha=0.8)
        ax.axvline(x=d.ix['25%'],
                    ymin=0, ymax = maxx,
                    linewidth=5, 
                    color=(0.38717148143023966, 0.26607979423298955, 0.19042646089965626),
                    label='25%: {:.2f}'.format(d.ix['25%']),alpha=0.8)
        ax.axvline(x=d.ix['50%'],
                    ymin=0, ymax = maxx,
                    linewidth=5, 
                    color=(0.5830347014478903, 0.4006848666096784, 0.2867598470018353),
                    label='50%: {:.2f}'.format(d.ix['50%']),alpha=0.8)
        ax.axvline(x=d.ix['75%'],
                    ymin=0, ymax = maxx,
                    linewidth=5, 
                    color=(0.7743429628604792, 0.5321595884659791, 0.3808529217993126),
                    label='75%: {:.2f}'.format(d.ix['75%']),alpha=0.8)
        ax.axvline(x=d.ix['max'],
                    ymin=0, ymax = maxx,
                    linewidth=5, 
                    color=(0.9415558823529412, 0.663581294117647, 0.47400294117647046),
                    label='max%: {:.2f}'.format(d.ix['max']),alpha=0.8)
        
       # ax.plot((m-0.1, m+0.1), (0,max(n)), 'k-')
    plt.grid(linewidth=2)
    plt.title("Basic statistics",fontsize=20)
    plt.legend(loc='upper left',fontsize=18)
    plt.show()
    
    clear_output(True)

Using interact to create widgets


In [9]:
inter = interact(draw_regions,
             normed=False,distribution=['Normal','example_2','custom'])
display(inter)


<function __main__.draw_regions>

Same thing using KungFu


In [11]:
KungFu(func=draw_regions, interact=True, box='3r')[0]


It is also possible to overwrite the default values of the function parameters.


In [12]:
KungFu(func=draw_regions,
       bins=(1,100,2,50),
       bw=(0.01,10.,0.025,0.18),
       normed=False,
       distribution=['Normal','example_2'],
       interact=True,box="2c")[0]


Shaolin KungFu for automatic dashboard creation

Here it is an example on how to build a complex dashboard:


In [13]:
import numpy as np
import pandas as pd
import matplotlib.pylab as plt
import seaborn as sns
from IPython.core.display import clear_output
from shaolin import KungFu,Dashboard
class DensityPlot(KungFu):
    def __init__(self,data):
        #Kwargs expanding standard interact syntax with object notation and shaoscript
        d_kwargs= {'num_xbins':(1,40,2,10),
                 'num_ybins':(1,40,2,10),
                 'grid': True,
                 'link_bins':True,
                 'plot_density':True,
                 'reg_order':(1,10,1,1),
                 'ci':"(1,100,1,95)$N=ci&d=Confidence intervals",
                 'robust':[False],
                 'scatter':True,
                 'regression':True,
                 'marginals':(['None','Histogram','KDE','Both'],'Both'),
                 'bw':(0.001,10,0.05,0.5),
                 'Title':'#Combining plots$N=title&D=',
                 'dataset':['ALL','I','II','III','IV'],
                 'save':False,
                 #'@kernel':['gau','cos','biw','epa','tri','triw']
                }
        
        #general
        title = '#Exploratory plots$N=title&D='
        marginals = "@['Both','None','Histogram','KDE']$d=Marginals"
        
        dset = "@['ALL','I','II','III','IV']$D=Dataset"
        x_cols = '@dd$D=X column&o='+str(data.columns.values.tolist())
        y_cols = '@dd$D=Y column&o='+str(data.columns.values.tolist())+'&v='+data.columns[1]
        save = "@[False]$D=Save plot&n=save"
        data_layout = ['c$N=data_layout',[title,dset,['r$N=sub_row',[x_cols,y_cols]],marginals,save]]
        #histograms
        h_title = '#Historam options$N=h_title&D='
        num_xbins = '@(1,40,2,10)$D=Num xbins&n=num_xbins'
        num_ybins = '@(1,40,2,10)$D=Num ybins&n=num_ybins'
        lbins = '@True$D=link_bins'
        hist_layout = ['c$N=hist_layout',[h_title,num_xbins,num_ybins,lbins]]
        
        #scatter
        s_title = '#Scatterplot options$N=s_title&D='
        scat = '@[True]$D=Plot Scatter&n=scatter' 
        grid = '@True$D=Grid'
        scatter_layout = ['c$N=scatter_layout',[s_title,scat,grid]]
        #regression
        r_title = '#Regression options$N=r_title&D='
        reg = '@[True]$D=Plot Regression&n=regression'
        robust = '@False$D=Robust'
        reg_order = "@(1,10,1,1)$D=Reg order"
        ci = "@(1,100,1,95)$N=ci&d=Confidence intervals"
        reg_layout = ['c$N=reg_layout',[r_title,reg,reg_order,ci,robust]]
        #kde
        k_title = '#KDE plot options$N=k_title&D='
        kde = '@[True]$D=Plot KDE&n=plot_density'
        bw =  "@(0.001,10,0.05,0.5)$D=Bandwidth&n=bw"
        kde_layout = ['@c$N=kde_layout',[k_title,kde,bw]]
        dash = ['ac$N=dae_plot&t=General,Histogram,Scatter,KDE,Regression',[data_layout,
                                                                        hist_layout,
                                                                        scatter_layout,
                                                                        kde_layout,
                                                                        reg_layout,
                                                                       ]
               ]
        self.data = data
        KungFu.__init__(self,dash=dash,mode='interactive')
        self.num_xbins.widget.continuous_update=False
        self.num_ybins.widget.continuous_update=False
        self.density_plot()
        self.link_bins.observe(self.link_sliders)
        self.observe(self.density_plot)
        self.link_sliders()
        self._i_plot = 0
        
        
        
    
    def link_sliders(self,_=None):
        if self.link_bins():
            self.num_ybins.visible = False
            self.link('num_xbins','num_ybins')
        else:
            self.unlink('num_xbins','num_ybins')
            self.num_ybins.visible = True
            
    def density_plot(self,_=None):
        clear_output()
        if self.kwargs['dataset']=='ALL':
            self.x_column.value = 'x'
            self.y_column.value = 'y'
            subdf = self.data.copy()
        else:
            subdf = self.data[self.data['dataset']==self.dataset()].copy()
        x,y = subdf[self.x_column()],subdf[self.y_column()]
        x_regions = 10
        y_regions = 10
        x_bins = np.linspace(x.min(),x.max(),num=self.kwargs['num_xbins'])
        y_bins = np.linspace(y.min(),y.max(),num=self.kwargs['num_ybins'])
        
        g = sns.JointGrid(x=self.x_column(), y=self.y_column(), data=subdf)
        
        g.fig.set_figwidth(14)
        g.fig.set_figheight(9)
        if self.kwargs['plot_density']:
            g = g.plot_joint(sns.kdeplot, shade=True,alpha=0.5,legend=True,bw=self.kwargs['bw'],
                             gridsize=int((len(x_bins)+len(y_bins))/2),
                             clip=((x.min()*0.95,x.max()*1.05),(y.min()*0.95,y.max()*1.05)))
        
        if self.kwargs['scatter']:
            g = g.plot_joint(plt.scatter,s=80,alpha=0.8)
            
        
        if self.kwargs['marginals'] in ['Histogram','Both']:
            _ = g.ax_marg_x.hist(x, alpha=.6,
                                  bins=x_bins,normed=True)
            _ = g.ax_marg_y.hist(y, alpha=.6,
                                   orientation="horizontal",
                                   bins=y_bins,normed=True)
        if self.kwargs['marginals'] in ['KDE','Both']:
            clip = ((x.values.min()*0.95,x.values.max()*1.05),(y.values.min()*0.95,y.values.max()*1.05))
            g = g.plot_marginals(sns.kdeplot, shade=True,alpha=0.5,
                             gridsize=int((len(x_bins)+len(y_bins))/2))
                             
        if self.kwargs['regression']:
            g = g.plot_joint(sns.regplot,
                             robust=self.kwargs['robust'],
                             ci=self.kwargs['ci'],
                             order=self.kwargs['reg_order'],
                             truncate=True,
                             scatter_kws={"s": 0})
        if self.kwargs['grid']:
            plt.grid(linewidth=2) 
            plt.xticks(x_bins)
            plt.yticks(y_bins)
            
        plt.xlim(x.values.min()*0.95,x.values.max()*1.05)
        plt.ylim(y.values.min()*0.95,y.values.max()*1.05)
         
        
      
        if self.kwargs['save']:
            _ =plt.savefig("density_plot_{}.png".format(self._i_plot), dpi=100)
            self._i_plot += 1

In [14]:
DensityPlot(data)[0]



In [ ]: