In [1]:

    

#SKIP_COMPARE_OUTPUT
import pixiedust


    

    




Pixiedust database opened successfully
Table VERSION_TRACKER created successfully
Table METRICS_TRACKER created successfully

Share anonymous install statistics? (opt-out instructions)

PixieDust will record metadata on its environment the next time the package is installed or updated. The data is anonymized and aggregated to help plan for future releases, and records only the following values:

{
   "data_sent": currentDate,
   "runtime": "python",
   "application_version": currentPixiedustVersion,
   "space_id": nonIdentifyingUniqueId,
   "config": {
       "repository_id": "https://github.com/ibm-watson-data-lab/pixiedust",
       "target_runtimes": ["Data Science Experience"],
       "event_id": "web",
       "event_organizer": "dev-journeys"
   }
}
You can opt out by calling pixiedust.optOut() in a new cell.






    






        

            
                
            
            Pixiedust version 1.1.18
        
        






    




Pixiedust runtime updated. Please restart kernel
Table SPARK_PACKAGES created successfully
Table USER_PREFERENCES created successfully
Table service_connections created successfully






    





Warning: You are not running the latest version of PixieDust. Current is 1.1.18, Latest is 1.1.17






    






                
Please copy and run the following command in a new cell to upgrade: !pip install --user --upgrade pixiedust
            






    





Please restart kernel after upgrading.

In [2]:

    

data = [(2010, 'Camping Equipment', 3),
    (2010, 'Golf Equipment', 1),
    (2010, 'Mountaineering Equipment', 1),
    (2010, 'Outdoor Protection', 2),
    (2010, 'Personal Accessories', 2),
    (2011, 'Camping Equipment', 4),
    (2011, 'Golf Equipment', 5),
    (2011, 'Mountaineering Equipment',2),
    (2011, 'Outdoor Protection', 4),
    (2011, 'Personal Accessories', 2),
    (2012, 'Camping Equipment', 5),
    (2012, 'Golf Equipment', 5),
    (2012, 'Mountaineering Equipment', 3),
    (2012, 'Outdoor Protection', 5),
    (2012, 'Personal Accessories', 3),
    (2013, 'Camping Equipment', 8),
    (2013, 'Golf Equipment', 5),
    (2013, 'Mountaineering Equipment', 3),
    (2013, 'Outdoor Protection', 8),
    (2013, 'Personal Accessories', 4)]

columns = ["year","zone","unique_customers"]


    

In [3]:

    

#TARGET=SPARK
sqlContext = SQLContext(sc)
dd = sqlContext.createDataFrame( data, columns)
dd.count()


    

    
Out[3]:





20

In [4]:

    

#TARGET=PLAIN
import pandas
dd = pandas.DataFrame( data, columns=columns )


    

In [ ]:

    

#SKIP_COMPARE_OUTPUT
display(dd, no_gen_tests='true')


    

    





Hey, there's something awesome here! To see it, open this notebook outside GitHub, in a viewer like Jupyter
        

            
        
    

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
year:  int64
            
              
zone:  object
            
              
unique_customers:  int64
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 20 of 20 rows
            
          
          
          

            

              

                
                
year
                
                
zone
                
                
unique_customers
                
              
            
          
          

            

              

                

                  
                  
year
                  
                  
zone
                  
                  
unique_customers
                  
                
              
              

                
                

                  
                  
2010
                  
                  
Camping Equipment
                  
                  
3
                  
                
                
                

                  
                  
2010
                  
                  
Golf Equipment
                  
                  
1
                  
                
                
                

                  
                  
2010
                  
                  
Mountaineering Equipment
                  
                  
1
                  
                
                
                

                  
                  
2010
                  
                  
Outdoor Protection
                  
                  
2
                  
                
                
                

                  
                  
2010
                  
                  
Personal Accessories
                  
                  
2
                  
                
                
                

                  
                  
2011
                  
                  
Camping Equipment
                  
                  
4
                  
                
                
                

                  
                  
2011
                  
                  
Golf Equipment
                  
                  
5
                  
                
                
                

                  
                  
2011
                  
                  
Mountaineering Equipment
                  
                  
2
                  
                
                
                

                  
                  
2011
                  
                  
Outdoor Protection
                  
                  
4
                  
                
                
                

                  
                  
2011
                  
                  
Personal Accessories
                  
                  
2
                  
                
                
                

                  
                  
2012
                  
                  
Camping Equipment
                  
                  
5
                  
                
                
                

                  
                  
2012
                  
                  
Golf Equipment
                  
                  
5
                  
                
                
                

                  
                  
2012
                  
                  
Mountaineering Equipment
                  
                  
3
                  
                
                
                

                  
                  
2012
                  
                  
Outdoor Protection
                  
                  
5
                  
                
                
                

                  
                  
2012
                  
                  
Personal Accessories
                  
                  
3
                  
                
                
                

                  
                  
2013
                  
                  
Camping Equipment
                  
                  
8
                  
                
                
                

                  
                  
2013
                  
                  
Golf Equipment
                  
                  
5
                  
                
                
                

                  
                  
2013
                  
                  
Mountaineering Equipment
                  
                  
3
                  
                
                
                

                  
                  
2013
                  
                  
Outdoor Protection
                  
                  
8
                  
                
                
                

                  
                  
2013
                  
                  
Personal Accessories
                  
                  
4
                  
                
                
              
            
          
        
      
    
  
  
  
  
        

In [6]:

    

display(dd,cell_id='2850507A33444756801B43A74E3E6A56',no_gen_tests='true',showLegend='true',staticFigure='false',aggregation='SUM',rowCount='100',handlerId='pieChart',valueFields='unique_customers',rendererId='matplotlib',stacked='true',keyFields='zone',nostore_cw='1098',nostore_pixiedust='true',nostore_bokeh='false',prefix='ae087e77')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
                    
                            
                        
                    
                
        
    
    

        
        
Pie Chart Options
        
            
            
                

                    Chart Size: 100
                    
                    

                
            
            
            
            
        
            
            
            
            
                

                    
                    Show y label
                    
                
            
            
        
            
            
            
            
                

                    
                    D3 Rendering (mpld3)
                    
                
            
            
        
            
            
            
            
                

                    
                    Show legend
                    
                
            
            
        
        
    

In [7]:

    

display(dd,cell_id='2850507A33444756801B43A74E3E6A56',no_gen_tests='true',showLegend='true',staticFigure='false',aggregation='SUM',rowCount='100',handlerId='barChart',valueFields='unique_customers',rendererId='matplotlib',stacked='true',keyFields='zone',nostore_cw='1098',nostore_pixiedust='true',nostore_bokeh='false',prefix='2e7ed979')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
        Renderer:
        
            
                matplotlib
            
                bokeh
            
        
        
    
    





    

        

            
        
    


    

        

            
                    
                            
                        
                    
                
        
    
    

        
        
Bar Chart Options
        
            
            
                

                    Chart Size: 100
                    
                    

                
            
            
            
            
        
            
            
            
                

                    Cluster By:
                    
                    
                        
                            None
                        
                            year
                        
                    
                
            
            
            
        
            
            
            
            
                

                    
                    Time Series
                    
                
            
            
        
            
            
            
                

                    Orientation:
                    
                    
                        
                            vertical
                        
                            horizontal
                        
                    
                
            
            
            
        
            
            
            
            
                

                    
                    Show legend
                    
                
            
            
        
            
            
            
                

                    Sort By:
                    
                    
                        
                            Keys ASC
                        
                            Keys DESC
                        
                            Values ASC
                        
                            Values DESC
                        
                    
                
            
            
            
        
            
            
            
            
                

                    
                    D3 Rendering (mpld3)
                    
                
            
            
        
        
    

In [8]:

    

import pandas
d = {'col1': [None, 2, 1, 7, 1], 'col2': [6, None, 5, 9, 3], 'col3': ['ice cream', 'cookies', None, 'cake', 'pie']}
pdf2 = pandas.DataFrame(data=d)
pdf2.shape[0]


    

    
Out[8]:





5

In [9]:

    

display(pdf2,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',table_showrows='All',org_params='nostore_pixiedust',prefix='4046ac43',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 5 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
nan
                  
                  
6.0
                  
                  
ice cream
                  
                
                
                

                  
                  
2.0
                  
                  
nan
                  
                  
cookies
                  
                
                
                

                  
                  
1.0
                  
                  
5.0
                  
                  
None
                  
                
                
                

                  
                  
7.0
                  
                  
9.0
                  
                  
cake
                  
                
                
                

                  
                  
1.0
                  
                  
3.0
                  
                  
pie
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values
                        
                    
                
            
            
            
        
        
    

In [10]:

    

display(pdf2,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',table_showrows='Missing values',org_params='nostore_pixiedust',prefix='446b0b5f',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 3 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
nan
                  
                  
6.0
                  
                  
ice cream
                  
                
                
                

                  
                  
2.0
                  
                  
nan
                  
                  
cookies
                  
                
                
                

                  
                  
1.0
                  
                  
5.0
                  
                  
None
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values
                        
                    
                
            
            
            
        
        
    

In [11]:

    

display(pdf2,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',table_showrows='Not missing values',org_params='nostore_pixiedust',prefix='0923d868',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 2 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
7.0
                  
                  
9.0
                  
                  
cake
                  
                
                
                

                  
                  
1.0
                  
                  
3.0
                  
                  
pie
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values
                        
                    
                
            
            
            
        
        
    

In [12]:

    

#TARGET=SPARK
d = [(1, 3, 'pie'), (None, 6, 'ice cream'), (2, None, 'cookies'), (1, 5, None), (7, 9, 'cake')]
pdf = sqlContext.createDataFrame(d, ['col1', 'col2', 'col3'])
pdf.count()


    

    
Out[12]:





5

In [13]:

    

#TARGET=SPARK
display(pdf,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',table_showrows='Not missing values',org_params='nostore_pixiedust',prefix='a148c8f6',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 2 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
7.0
                  
                  
9.0
                  
                  
cake
                  
                
                
                

                  
                  
1.0
                  
                  
3.0
                  
                  
pie
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values
                        
                    
                
            
            
            
        
        
    

In [14]:

    

#TARGET=SPARK
display(pdf,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',table_showrows='Missing values',org_params='nostore_pixiedust',prefix='a4c4be27',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 3 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
nan
                  
                  
6.0
                  
                  
ice cream
                  
                
                
                

                  
                  
2.0
                  
                  
nan
                  
                  
cookies
                  
                
                
                

                  
                  
1.0
                  
                  
5.0
                  
                  
None
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values
                        
                    
                
            
            
            
        
        
    

In [15]:

    

#TARGET=SPARK
display(pdf,nostore_bokeh='false',cell_id='20499543AE4446FAA9707357577DE951',org_params='nostore_pixiedust',prefix='49fe4045',nostore_cw='1098',handlerId='tableView',nostore_vh='1050',nostore_pixiedust='true')


    

    









    

        
        
            
            Options
        
        
        
        
        
            Share
        
    
    
    

        
    
    





    

        

            
        
    


    

        

            
  
  
  
  

    
    
    

      

        

          Schema
        
      
      

        

          

            
Field types:
            
              
col1:  float64
            
              
col2:  float64
            
              
col3:  object
            
          
        
      
    
    
    
    

      
      

        

           Table
        
      
      
      

        

          
          
          
          

            
            Showing 5 of 5 rows
            
          
          
          

            

              

                
                
col1
                
                
col2
                
                
col3
                
              
            
          
          

            

              

                

                  
                  
col1
                  
                  
col2
                  
                  
col3
                  
                
              
              

                
                

                  
                  
nan
                  
                  
6.0
                  
                  
ice cream
                  
                
                
                

                  
                  
2.0
                  
                  
nan
                  
                  
cookies
                  
                
                
                

                  
                  
1.0
                  
                  
5.0
                  
                  
None
                  
                
                
                

                  
                  
7.0
                  
                  
9.0
                  
                  
cake
                  
                
                
                

                  
                  
1.0
                  
                  
3.0
                  
                  
pie
                  
                
                
              
            
          
        
      
    
  
  
  
  
        
    
    

        
        
Table View Options
        
            
            
            
            
                

                    
                    Hide Schema
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Search
                    
                
            
            
        
            
            
            
            
                

                    
                    Hide Row Count
                    
                
            
            
        
            
            
            
                

                    Show Rows:
                    
                    
                        
                            All
                        
                            Missing values
                        
                            Not missing values