apply on a pandas Series and DataFramelambda functions work
In [1]:
# numpy and pandas related imports
import numpy as np
from pandas import Series, DataFrame
import pandas as pd
Let's make two Series and a DataFrame to use for our example
In [2]:
# for example, using lower and uppercase English letters
import string
string.lowercase, string.uppercase
Out[2]:
In [3]:
# we can make a list composed of the individual lowercase letters
list(string.lowercase)
Out[3]:
In [4]:
# create a pandas Series out of the list of lowercase letters
lower = Series(list(string.lowercase), name='lower')
print type(lower)
lower.head()
Out[4]:
In [5]:
# create a pandas Series out of the list of lowercase letters
upper = Series(list(string.uppercase), name='upper')
In [6]:
# concatenate the two Series as columns, using axis=1
# axis = 0 would result in two rows in the DataFrame
df = pd.concat((lower, upper), axis=1)
df.head()
Out[6]:
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.apply.html:
Series.apply(func, convert_dtype=True, args=(), **kwds)
Invoke function on values of Series.
In [7]:
# Let's start by using Series.apply
# http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.apply.html
# first of all, it's useful to find a way to use apply to return the exact same Series
def identity(s):
return s
lower.apply(identity)
Out[7]:
In [8]:
# show that identity yields the same Series -- first on element by element basis
lower.apply(identity) == lower
Out[8]:
In [9]:
# Check that match happens for every element in the Series using numpy.all
# http://docs.scipy.org/doc/numpy/reference/generated/numpy.all.html
np.all(lower.apply(identity) == lower)
Out[9]:
Sometimes it's convenient to write functions using lambda, especially short functions for doing a simple transformation of the parameters. Only some functions can be rewritten with lambda.
In [10]:
def add_preface(s):
return 'letter ' + s
lower.apply(add_preface)
Out[10]:
In [11]:
# rewrite with lambda
lower.apply(lambda s: 'letter ' + s)
Out[11]:
Another illustration of using apply -- using ord and chr
In [12]:
# ord: Given a string of length one, return an integer representing the Unicode code
# point of the character when the argument is a unicode object, or the value of the
# byte when the argument is an 8-bit string.
# http://docs.python.org/2.7/library/functions.html#ord
ord('a')
Out[12]:
In [13]:
# chr: Return a string of one character whose ASCII code is the integer i.
# http://docs.python.org/2.7/library/functions.html#chr
chr(97)
Out[13]:
In [14]:
# show that for the case of 'a', chr(ord()) returns what we start with:'a'
chr(ord('a')) == 'a'
Out[14]:
In [15]:
# we can test whether chr reverses ord for all the lower case letters
# note how we chain two apply together
np.all(lower.apply(ord).apply(chr) == lower)
Out[15]:
Note that we read off a specific series from the DataFrame
In [16]:
type(df.upper)
Out[16]:
In [17]:
# transform
df.upper.apply(lambda s: s.lower())
Out[17]:
apply can also be applied to a DataFrame
http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.apply.html
DataFrame.apply(func, axis=0, broadcast=False, raw=False, reduce=None, args=(), **kwds)
Applies function along input axis of DataFrame.
Objects passed to functions are Series objects having index either the DataFrame’s index (axis=0) or the columns (axis=1). Return type depends on whether passed function aggregates, or the reduce argument if the DataFrame is empty.
In [18]:
# let's show that whether we use apply on columns (axis=0) or rows (axis=1), we get the same
# result
def identity(s):
return s
np.all(df.apply(identity, axis=0) == df.apply(identity, axis=1))
Out[18]:
In [19]:
# for each column, first lower and then upper, return the index
def index(s):
return s.index
df.apply(index, axis=0)
Out[19]:
In [20]:
# for each row (axis=1), first lower and then upper, return the index
# (which are the column names)
def index(s):
return s.index
df.apply(index, axis=1)
Out[20]:
In [21]:
# it might be easier to see the difference between axis=0 vs axis=1
# by using join
# Consider what you get with
"".join(df.lower)
Out[21]:
In [22]:
# Now compare (axis=0)
df.apply(lambda s: "".join(s), axis=0)
Out[22]:
In [23]:
# join with axis=1
df.apply(lambda s: "".join(s), axis=1)
Out[23]:
In [24]:
# note that you can access use the index in your function passed to apply
df.apply(lambda s: s['upper'] + s['lower'], axis=1)
Out[24]:
In [24]:
In [ ]: