In this exercise, we will perform some rudimentary practices similar to those of an actual data scientist.
Part of a data scientist's job is to use her or his intuition and insight to
write algorithms and heuristics. A data scientist also creates mathematical models
to make predictions based on some attributes from the data that they are examining.
We would like for you to take your knowledge and intuition about the Titanic and its passengers' attributes to predict whether or not the passengers survived or perished. You can read more about the Titanic and specifics about this dataset at: http://en.wikipedia.org/wiki/RMS_Titanic http://www.kaggle.com/c/titanic-gettingStarted
In this exercise and the following ones, you are given a list of Titantic passengers
and their associated information. More information about the data can be seen at the
link below:
http://www.kaggle.com/c/titanic-gettingStarted/data.
For this exercise, you need to write a simple heuristic that will use
the passengers' gender to predict if that person survived the Titanic disaster.
You prediction should be 78% accurate or higher.
Here's a simple heuristic to start off:
You can access the gender of a passenger via passenger['Sex'].
passenger['Sex'] will return a string "male".passenger['Sex'] will return a string "female".Write your prediction back into the "predictions" dictionary.
The key of the dictionary should be the passenger's id (which can be accessed via passenger["PassengerId"]) and the associated value should be "1" if the passenger survied or "0" otherwise.
For example,
You can also look at the Titantic data that you will be working with at the link below: https://s3.amazonaws.com/content.udacity-data.com/courses/ud359/titanic_data.csv
In [3]:
import numpy
import pandas
from pandas.core import datetools
#import statsmodels.api as sm
def simple_heuristic(file_path):
'''Read the assignment description above.
'''
# Create a dictionary to store all the value
predictions = {}
# Read from csv file
df = pandas.read_csv(file_path)
# Using for loop to read through the list:
# read the passenger's id using pandas's function interrows() - Iterate over DataFrame rows as (index, Series) pairs.
for passenger_index, passenger in df.iterrows():
passenger_id = passenger['PassengerId']
# Your code here:
# For example, let's assume that if the passenger
# is a male, then the passenger survived.
# if passenger['Sex'] == 'male':
# predictions[passenger_id] = 1
if passenger['Sex'] == 'male':
predictions[passenger_id] = 0
elif passenger['Sex'] == 'female':
predictions[passenger_id] = 1
return predictions
In [6]:
simple_heuristic('l2-ps1-data-titanic.csv')
Out[6]:
You are given a list of Titantic passengers and their associated information. More information about the data can be seen at the link below: http://www.kaggle.com/c/titanic-gettingStarted/data
For this exercise, you need to write a more sophisticated algorithm that will use the passengers' gender and their socioeconomical class and age to predict if they survived the Titanic diaster.
You prediction should be 79% accurate or higher.
Here's the algorithm:
predict the passenger survived if:
- 1) If the passenger is
femaleor- 2) if his/her socioeconomic status is
highAND if the passenger isunder 18Otherwise, your algorithm should predict that the passenger perished in the disaster.
Or more specifically in terms of coding:
female or (high status and under 18)
You can access the gender of a passenger via passenger['Sex'].
You can access the socioeconomic status of a passenger via passenger['Pclass']:
123You can access the age of a passenger via passenger['Age'].
Write your prediction back into the "predictions" dictionary. The key of the dictionary should be the Passenger's id (which can be accessed via passenger["PassengerId"]) and the associated value should be 1 if the passenger survived or 0 otherwise.
passenger_id = passenger['PassengerId']predictions[passenger_id] = 1passenger_id = passenger['PassengerId']predictions[passenger_id] = 0You can also look at the Titantic data that you will be working with at the link below: https://s3.amazonaws.com/content.udacity-data.com/courses/ud359/titanic_data.csv
In [7]:
import numpy
import pandas
import statsmodels.api as sm
def complex_heuristic(file_path):
'''
Read the quiz 2 description above
'''
predictions = {}
df = pandas.read_csv(file_path)
for passenger_index, passenger in df.iterrows():
passenger_id = passenger['PassengerId']
#
# your code here
# for example, assuming that passengers who are male
# and older than 18 surived:
# if passenger['Sex'] == 'male' or passenger['Age'] < 18:
# predictions[passenger_id] = 1
#
predictions[passenger_id] = 0
if passenger['Sex'] == 'female' or passenger['Age'] < 18 and passenger['Pclass'] == 1:
predictions[passenger_id] = 1
return predictions
You are given a list of Titantic passengers and their associated information. More information about the data can be seen at the link below: http://www.kaggle.com/c/titanic-gettingStarted/data
For this exercise, you need to write a custom heuristic that will take in some combination of the passenger's attributes and predict if the passenger survived the Titanic diaster.
Can your custom heuristic beat 80% accuracy?
The available attributes are:
Pclass Passenger ClassName NameSex SexAge AgeSibSp Number of Siblings/Spouses AboardParch Number of Parents/Children AboardTicket Ticket NumberFare Passenger FareCabin CabinEmbarked Port of EmbarkationC = Cherbourg; Q = Queenstown; S = Southampton)SPECIAL NOTES:
Pclass is a proxy for socioeconomic status (SES)
Age is in years; fractional if age less than one
With respect to the family relation variables (i.e. SibSp and Parch) some relations were ignored. The following are the definitions used for SibSp and Parch.
Write your prediction back into the "predictions" dictionary.
the associating value should be 1 if the passenger survvied or 0 otherwise.
For example,
You can also look at the Titantic data that you will be working with at the link below: https://s3.amazonaws.com/content.udacity-data.com/courses/ud359/titanic_data.csv
In [ ]:
import numpy
import pandas
import statsmodels.api as sm
def custom_heuristic(file_path):
'''
Read the quiz 3 description above.
'''
predictions = {}
df = pandas.read_csv(file_path)
for passenger_index, passenger in df.iterrows():
#
# your code here
#
passenger_id = passenger['passengerId']
predictions[passenger_id] = 0
# assume that all women and children not in 3rd class survived
if (passenger['Sex']=='female' or passenger['Age'] < 15) and passenger['Pclass'] != 3:
predictions[passenger_id] = 1
return predictions