In [1]:
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
Before going into the perception experiment, let's first talk about some handy datasets that you can play with.
It's nice to have clean datasets handy to practice data visualization. There is a nice small package called vega-datasets, from the altair project.
You can install the package by running
$ pip install vega-datasets
or
$ pip3 install vega-datasets
Once you install the package, you can import and see the list of datasets:
In [2]:
from vega_datasets import data
data.list_datasets()
Out[2]:
or you can work with only smaller, local datasets.
In [3]:
from vega_datasets import local_data
local_data.list_datasets()
Out[3]:
Ah, we have the anscombe data here! Let's see the description of the dataset.
In [4]:
local_data.anscombe.description
Out[4]:
In [5]:
df = local_data.anscombe()
df.head()
Out[5]:
Q1: can you draw a scatterplot of the dataset "I"? You can filter the dataframe based on the Series column and use plot function that you used for the Snow's map.
In [6]:
# TODO: put your code here
Out[6]:
Let's look at a slightly more complicated dataset.
In [7]:
car_df = local_data.cars()
car_df.head()
Out[7]:
Pandas provides useful summary functions. It identifies numerical data columns and provides you with a table of summary statistics.
In [9]:
car_df.describe()
Out[9]:
If you ask to draw a histogram, you get all of them. :)
In [10]:
car_df.hist()
Out[10]:
Well this is too small. You can check out the documentation and change the size of the figure.
Q2: by consulting the documentation, can you make the figure larger so that we can see all the labels clearly? And then make the layout 2 x 3 not 3 x 2, then change the number of bins to 20?
In [11]:
# TODO: put your code here
Let's do an experiment! The procedure is as follows:
First, let's define the length of a short and a long bar. We also create two empty lists to store perceived and actual length.
In [12]:
import random
import time
import numpy as np
l_short_bar = 1
l_long_bar = 10
perceived_length_list = []
actual_length_list = []
Let's run the experiment.
The random module in Python provides various random number generators, and the random.uniform(a,b) function returns a floating point number in [a,b].
We can plot horizontal bars using the pyplot.barh() function. Using this function, we can produce a bar graph that looks like this:
In [13]:
mystery_length = random.uniform(1, 10) # generate a number between 1 and 10. this is the *actual* length.
plt.barh(np.arange(3), [l_short_bar, mystery_length, l_long_bar], align='center')
plt.yticks(np.arange(3), ('1', '?', '10'))
plt.xticks([]) # no hint!
Out[13]:
Btw, np.arange is used to create a simple integer list [0, 1, 2].
In [150]:
np.arange(3)
Out[150]:
Now let's define a function to perform the experiment once. When you run this function, it picks a random number between 1.0 and 10.0 and show the bar chart. Then it asks you to input your estimate of the length of the middle bar. It then saves that number to the perceived_length_list and the actual answer to the actual_length_list.
In [14]:
def run_exp_once():
mystery_length = random.uniform(1, 10) # generate a number between 1 and 10.
plt.barh(np.arange(3), [l_short_bar, mystery_length, l_long_bar], height=0.5, align='center')
plt.yticks(np.arange(3), ('1', '?', '10'))
plt.xticks([]) # no hint!
plt.show()
perceived_length_list.append( float(input()) )
actual_length_list.append(mystery_length)
In [15]:
run_exp_once()
Now, run the experiment many times to gather your data. Check the two lists to make sure that you have the proper dataset. The length of the two lists should be the same.
In [ ]:
# TODO: Run your experiment many times here
Now we can draw the scatter plot of perceived and actual length. The matplotlib's scatter() function will do this. This is the backend of the pandas' scatterplot. Here is an example of how to use scatter:
In [19]:
plt.scatter(x=[1,5,10], y=[1,10, 5])
Out[19]:
Q3: Now plot your result using the scatter() function. You should also use plt.title(), plt.xlabel(), and plt.ylabel() to label your axes and the plot itself.
In [21]:
# TODO: put your code here
Out[21]:
After plotting, let's fit the relation between actual and perceived lengths using a polynomial function. We can easily do it using curve_fit(f, x, y) in Scipy, which is to fit $x$ and $y$ using the function f. In our case, $f = a*x^b +c$. For instance, we can check whether this works by creating a fake dataset that follows the exact form:
In [22]:
from scipy.optimize import curve_fit
def func(x, a, b, c):
return a * np.power(x, b) + c
x = np.arange(20) # [0,1,2,3, ..., 19]
y = np.power(x, 2) # [0,1,4,9, ... ]
popt, pcov = curve_fit(func, x, y)
print('{:.2f} x^{:.2f} + {:.2f}'.format(*popt))
Q4: Now fit your data! Do you see roughly linear relationship between the actual and the perceived lengths? It's ok if you don't!
In [ ]:
# TODO: your code here
Similar to the above experiment, we now represent a random number as a circle, and the area of the circle is equal to the number.
First, calculate the radius of a circle from its area and then plot using the Circle() function. plt.Circle((0,0), r) will plot a circle centered at (0,0) with radius r.
In [24]:
n1 = 0.005
n2 = 0.05
radius1 = np.sqrt(n1/np.pi) # area = pi * r * r
radius2 = np.sqrt(n2/np.pi)
random_radius = np.sqrt(n1*random.uniform(1,10)/np.pi)
plt.axis('equal')
plt.axis('off')
circ1 = plt.Circle( (0,0), radius1, clip_on=False )
circ2 = plt.Circle( (4*radius2,0), radius2, clip_on=False )
rand_circ = plt.Circle((2*radius2,0), random_radius, clip_on=False )
plt.gca().add_artist(circ1)
plt.gca().add_artist(circ2)
plt.gca().add_artist(rand_circ)
Out[24]:
Let's have two lists for this experiment.
In [159]:
perceived_area_list = []
actual_area_list = []
And define a function for the experiment.
In [161]:
def run_area_exp_once(n1=0.005, n2=0.05):
radius1 = np.sqrt(n1/np.pi) # area = pi * r * r
radius2 = np.sqrt(n2/np.pi)
mystery_number = random.uniform(1,10)
random_radius = np.sqrt(n1*mystery_number/math.pi)
plt.axis('equal')
plt.axis('off')
circ1 = plt.Circle( (0,0), radius1, clip_on=False )
circ2 = plt.Circle( (4*radius2,0), radius2, clip_on=False )
rand_circ = plt.Circle((2*radius2,0), random_radius, clip_on=False )
plt.gca().add_artist(circ1)
plt.gca().add_artist(circ2)
plt.gca().add_artist(rand_circ)
plt.show()
perceived_area_list.append( float(input()) )
actual_area_list.append(mystery_number)
Q5: Now you can run the experiment many times, plot the result, and fit a power-law curve!
In [25]:
# TODO: put your code here. You can use multiple cells.
In [ ]:
In [ ]:
In [ ]:
What is your result? How are the exponents different from each other?
In [ ]: