pandasPandas! They are adorable animals. You might think they are the worst animal ever but that is not true. You might sometimes think pandas is the worst library every, and that is only kind of true.
The important thing is use the right tool for the job. pandas is good for some stuff, SQL is good for some stuff, writing raw Python is good for some stuff. You'll figure it out as you go along.
Now let's start coding. Hopefully you did pip install pandas before you started up this notebook.
In [1]:
# import pandas, but call it pd. Why? Because that's What People Do.
In [2]:
import pandas as pd
When you import pandas, you use import pandas as pd. That means instead of typing pandas in your code you'll type pd.
You don't have to, but every other person on the planet will be doing it, so you might as well.
Now we're going to read in a file. Our file is called NBA-Census-10.14.2013.csv because we're sports moguls. pandas can read_ different types of files, so try to figure it out by typing pd.read_ and hitting tab for autocomplete.
In [3]:
# We're going to call this df, which means "data frame"
# It isn't in UTF-8 (I saved it from my mac!) so we need to set the encoding
df = pd.read_csv("NBA-Census-10.14.2013.csv", encoding ="mac_roman")
#this is a data frame (df)
A dataframe is basically a spreadsheet, except it lives in the world of Python or the statistical programming language R. They can't call it a spreadsheet because then people would think those programmers used Excel, which would make them boring and normal and they'd have to wear a tie every day.
Now let's look at our data, since that's what data is for
In [4]:
# Let's look at all of it
df
Out[4]:
If we scroll we can see all of it. But maybe we don't want to see all of it. Maybe we hate scrolling?
In [5]:
# Look at the first few rows
df.head() #shows first 5 rows
Out[5]:
...but maybe we want to see more than a measly five results?
In [6]:
# Let's look at MORE of the first few rows
df.head(10)
Out[6]:
But maybe we want to make a basketball joke and see the final four?
In [7]:
# Let's look at the final few rows
df.tail(4)
Out[7]:
So yes, head and tail work kind of like the terminal commands. That's nice, I guess.
But maybe we're incredibly demanding (which we are) and we want, say, the 6th through the 8th row (which we do). Don't worry (which I know you were), we can do that, too.
In [8]:
# Show the 6th through the 8th rows
df[5:8]
Out[8]:
In [9]:
# Get the names of the columns, just because
#columns_we_want = ['Name', 'Age']
#df[columns_we_want]
In [10]:
# If we want to be "correct" we add .values on the end of it
df.columns
Out[10]:
In [11]:
# Select only name and age
In [12]:
# Combing that with .head() to see not-so-many rows
columns_we_want = ['Name', 'Age']
df[columns_we_want].head()
Out[12]:
In [13]:
# We can also do this all in one line, even though it starts looking ugly
# (unlike the cute bears pandas looks ugly pretty often)
df[['Name', 'Age',]].head()
Out[13]:
NOTE: That was not df['Name', 'Age'], it was df[['Name', 'Age']]. You'll definitely type it wrong all of the time. When things break with pandas it's probably because you forgot to put in a million brackets.
In [14]:
df.head()
Out[14]:
I want to know how many people are in each position. Luckily, pandas can tell me!
In [15]:
# Grab the POS column, and count the different values in it.
df['POS'].value_counts()
Out[15]:
Now that was a little weird, yes - we used df['POS'] instead of df[['POS']] when viewing the data's details.
But now I'm curious about numbers: how old is everyone? Maybe we could, I don't know, get some statistics about age? Some statistics to describe age?
In [16]:
#race
race_counts = df['Race'].value_counts()
race_counts
Out[16]:
In [17]:
# Summary statistics for Age
df['Age'].describe()
Out[17]:
In [18]:
df.describe()
Out[18]:
In [19]:
# That's pretty good. Does it work for everything? How about the money?
df['2013 $'].describe()
#The result is the result, because the Money is a string.
Out[19]:
Unfortunately because that has dollar signs and commas it's thought of as a string. We'll fix it in a second, but let's try describing one more thing.
In [20]:
# Doing more describing
df['Ht (In.)'].describe()
Out[20]:
In [21]:
# Take another look at our inches, but only the first few
df['Ht (In.)'].head()
Out[21]:
In [22]:
# Divide those inches by 12
#number_of_inches = 300
#number_of_inches / 12
df['Ht (In.)'].head() / 12
Out[22]:
In [23]:
# Let's divide ALL of them by 12
df['Ht (In.)'] / 12
Out[23]:
In [24]:
# Can we get statistics on those?
height_in_feet = df['Ht (In.)'] / 12
height_in_feet.describe()
Out[24]:
In [25]:
# Let's look at our original data again
df.head(3)
Out[25]:
Okay that was nice but unfortunately we can't do anything with it. It's just sitting there, separate from our data. If this were normal code we could do blahblah['feet'] = blahblah['Ht (In.)'] / 12, but since this is pandas, we can't. Right? Right?
In [26]:
# Store a new column
df['feet'] = df['Ht (In.)'] / 12
df.head()
Out[26]:
That's cool, maybe we could do the same thing with their salary? Take out the $ and the , and convert it to an integer?
In [27]:
# Can't just use .replace
In [28]:
# Need to use this weird .str thing
In [29]:
# Can't just immediately replace the , either
In [30]:
# Need to use the .str thing before EVERY string method
In [31]:
# Describe still doesn't work.
In [32]:
# Let's convert it to an integer using .astype(int) before we describe it
In [ ]:
In [33]:
# Maybe we can just make them millions?
In [34]:
# Unfortunately one is "n/a" which is going to break our code, so we can make n/a be 0
In [35]:
# Remove the .head() piece and save it back into the dataframe
In [ ]:
In [36]:
# This is just the first few guys in the dataset. Can we order it?
In [37]:
# Let's try to sort them, ascending value
df.sort_values('feet')
Out[37]:
Those guys are making nothing! If only there were a way to sort from high to low, a.k.a. descending instead of ascending.
In [38]:
# It isn't descending = True, unfortunately
df.sort_values('feet', ascending=False).head()
Out[38]:
In [39]:
# We can use this to find the oldest guys in the league
df.sort_values('Age', ascending=False).head()
Out[39]:
In [40]:
# Or the youngest, by taking out 'ascending=False'
df.sort_values('feet').head()
Out[40]:
But sometimes instead of just looking at them, I want to do stuff with them. Play some games with them! Dunk on them~ describe them! And we don't want to dunk on everyone, only the players above 7 feet tall.
First, we need to check out boolean things.
In [41]:
# Get a big long list of True and False for every single row.
df['feet'] > 6.5
Out[41]:
In [42]:
# We could use value counts if we wanted
above_or_below_six_five = df['feet'] > 6.5
above_or_below_six_five.value_counts()
Out[42]:
In [43]:
# But we can also apply this to every single row to say whether YES we want it or NO we don't
In [44]:
# Instead of putting column names inside of the brackets, we instead
# put the True/False statements. It will only return the players above
# seven feet tall
df[df['feet'] > 6.5]
Out[44]:
In [45]:
df['Race'] == 'Asian'
Out[45]:
In [46]:
df[]
In [47]:
# Or only the guards
df['POS'] == 'G'.head()
#People below 6 feet
df['feet'] < 6.5
In [48]:
#Every column you ant to query needs parenthesis aroung it
#Guards that are higher than 6.5
#this is combination of both
df[(df['POS'] == 'G') & (df['feet'] < 6.5)].head()
Out[48]:
In [49]:
#We can save stuff
centers = df[df['POS'] == 'C']
guards = df[df['POS'] == 'G']
In [50]:
centers['feet'].describe()
Out[50]:
In [51]:
guards['feet'].describe()
Out[51]:
In [52]:
# It might be easier to break down the booleans into separate variables
In [53]:
# We can save this stuff
In [ ]:
In [54]:
# Maybe we can compare them to taller players?
In [55]:
!pip install matplotlib
In [ ]:
# This will scream we don't have matplotlib.
df['feet'].hist()
matplotlib is a graphing library. It's the Python way to make graphs!
In [151]:
%matplotlib inline
In [ ]:
df['feet'].hist()
In [ ]:
# this will open up a weird window that won't do anything
import matplot.
In [ ]:
# So instead you run this code
In [ ]:
plt.style.use('fivethirtyeight')
df['feet'].hist()
But that's ugly. There's a thing called ggplot for R that looks nice. We want to look nice. We want to look like ggplot.
In [ ]:
# Import matplotlib
# What's available?
In [ ]:
# Use ggplot
In [ ]:
# Make a histogram
In [ ]:
# Try some other styles
In [ ]:
That might look better with a little more customization. So let's customize it.
In [ ]:
# Pass in all sorts of stuff!
# Most from http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.hist.html
# .range() is a matplotlib thing
I want more graphics! Do tall people make more money?!?!
In [ ]:
In [ ]:
In [ ]:
# How does experience relate with the amount of money they're making?
In [ ]:
# At least we can assume height and weight are related
In [ ]:
# At least we can assume height and weight are related
# http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.plot.html
In [ ]:
In [ ]:
In [ ]:
# We can also use plt separately
# It's SIMILAR but TOTALLY DIFFERENT
In [ ]: