In [2]:
import sys
print(sys.version)
import numpy as np
print(np.__version__)
import pandas as pd
print(pd.__version__)
Now we’ve covered numpy the basis for pandas. We’ve covered some of the more advanced python concepts like list comprehensions and lambda functions. Let’s jump back to our roadmap.
We’ve covered the general ecosystem. We’ve covered a lot of numpy, now let’s get our hands dirty with some real data and actually using pandas. I hope you’ve watched the numpy videos that we covered earlier, they may seem academic but they’re really going to provide a fantastic foundation for what we’re going to learn now.
Now I'm going to breeze through a couple of subjects right now. Don’t feel the need to take notes or even try this code yourself. You can if you like, but it’s mainly to introduce you to the power of pandas, not for you to copy.
Pandas is made up of a couple of core types.
We’ve got an index. The index is a way of querying the data in an array or Series or querying the data in a Series or DataFrame.
In [3]:
pd.Index
Out[3]:
We’ve got the Series. The Series is like a 1 dimensional array in numpy. It has some helper functions and an index that allows for querying of the data in simple ways.
We can make a simple Series from a numpy array.
In [4]:
pd.Series
Out[4]:
In [5]:
series_ex = pd.Series(np.arange(26))
series_ex
Out[5]:
Now that we’ve created it. We can see it has an index, that we just talked about, as well as values. When we print these out, they should look similar - just like numpy arrays. Now here is where the series gets powerful.
In [6]:
series_ex.index
Out[6]:
we can replace the index with our own index. In this example I’ll use the lower case values of ascii characters.
In [9]:
import string
lcase = string.ascii_lowercase
ucase = string.ascii_uppercase
print(lcase, ucase)
In [10]:
lcase = list(lcase)
ucase = list(ucase)
print(lcase)
print(ucase)
In [11]:
series_ex.index = lcase
In [12]:
series_ex.index
Out[12]:
In [13]:
series_ex
Out[13]:
Now we can query just like we would if an array. You can think of the Series like an extremely powerful array.
We can query either sections or specific values.
In [14]:
series_ex.ix['d':'k']
Out[14]:
In [15]:
series_ex.ix['f']
Out[15]:
Now don’t worry about the functions that I’m using. We’re going to go over those in detail - I just wanted to introduce the concept.
We’ve got the DataFrame which is like a matrix or series of series’. It also has an index (or multiple indexes).
In [16]:
pd.DataFrame
Out[16]:
Let’s go ahead and create one. We’ve make it from the lowercase, uppercase, and a number range.
In [19]:
letters = pd.DataFrame([lcase, ucase, list(range(26))])
letters
Out[19]:
Just like a numpy array we can transpose it.
In [20]:
letters = letters.transpose()
letters.head()
Out[20]:
In [21]:
letters.columns
Out[21]:
In [22]:
letters.index
Out[22]:
But now that we have columns as well as an index, we can rename the columns to better describe and query the data.
In [23]:
letters.columns = ['lowercase','uppercase','number']
In [24]:
letters.lowercase
Out[24]:
In [25]:
letters['lowercase']
Out[25]:
We can even set up a date range to associate each letter with a date. Now obviously this isn’t too helpful for the alphabet, but this allows you to do some amazing things once you are analyzing real data.
In [26]:
letters.index = pd.date_range('9/1/2012',periods=26)
In [27]:
letters
Out[27]:
In [28]:
letters['9-10-2012':'9-15-2012']
Out[28]:
Now if you don’t have any experience with pandas this is going to seem like a lot! Don’t worry we’re going to cover everything in the coming videos, I just wanted to give you an introduction to the amazingly expressive power of pandas and python. We’ve seen the building blocks with the Index, the Series, and the DataFrame.
Now let’s dive deeper into each one.