Exercise 06

Data preparation and model evaluation exercise with Titanic data

We'll be working with a dataset from Kaggle's Titanic competition: data, data dictionary

Goal: Predict survival based on passenger characteristics

The sinking of the RMS Titanic is one of the most infamous shipwrecks in history. On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This sensational tragedy shocked the international community and led to better safety regulations for ships.

One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.

In this challenge, we ask you to complete the analysis of what sorts of people were likely to survive. In particular, we ask you to apply the tools of machine learning to predict which passengers survived the tragedy.

Read the data into Pandas



In [1]:

    
import pandas as pd
url = 'https://raw.githubusercontent.com/justmarkham/DAT8/master/data/titanic.csv'
titanic = pd.read_csv(url, index_col='PassengerId')
titanic.head()









    Out[1]:






  
    
      
      Survived
      Pclass
      Name
      Sex
      Age
      SibSp
      Parch
      Ticket
      Fare
      Cabin
      Embarked
    
    
      PassengerId
      
      
      
      
      
      
      
      
      
      
      
    
  
  
    
      1
      0
      3
      Braund, Mr. Owen Harris
      male
      22
      1
      0
      A/5 21171
      7.2500
      NaN
      S
    
    
      2
      1
      1
      Cumings, Mrs. John Bradley (Florence Briggs Th...
      female
      38
      1
      0
      PC 17599
      71.2833
      C85
      C
    
    
      3
      1
      3
      Heikkinen, Miss. Laina
      female
      26
      0
      0
      STON/O2. 3101282
      7.9250
      NaN
      S
    
    
      4
      1
      1
      Futrelle, Mrs. Jacques Heath (Lily May Peel)
      female
      35
      1
      0
      113803
      53.1000
      C123
      S
    
    
      5
      0
      3
      Allen, Mr. William Henry
      male
      35
      0
      0
      373450
      8.0500
      NaN
      S

Exercise 6.1

Impute the missing values of the age and Embarked



In [2]:

    
titanic.Age.fillna(titanic.Age.median(), inplace=True)
titanic.isnull().sum()









    Out[2]:





Survived      0
Pclass        0
Name          0
Sex           0
Age           0
SibSp         0
Parch         0
Ticket        0
Fare          0
Cabin       687
Embarked      2
dtype: int64



In [3]:

    
titanic.Embarked.mode()









    Out[3]:





0    S
dtype: object



In [4]:

    
titanic.Embarked.fillna('S', inplace=True)
titanic.isnull().sum()









    Out[4]:





Survived      0
Pclass        0
Name          0
Sex           0
Age           0
SibSp         0
Parch         0
Ticket        0
Fare          0
Cabin       687
Embarked      0
dtype: int64

Exercise 6.3

Convert the Sex and Embarked to categorical features



In [5]:

    
titanic['Sex_Female'] = titanic.Sex.map({'male':0, 'female':1})
titanic.head()









    Out[5]:






  
    
      
      Survived
      Pclass
      Name
      Sex
      Age
      SibSp
      Parch
      Ticket
      Fare
      Cabin
      Embarked
      Sex_Female
    
    
      PassengerId
      
      
      
      
      
      
      
      
      
      
      
      
    
  
  
    
      1
      0
      3
      Braund, Mr. Owen Harris
      male
      22
      1
      0
      A/5 21171
      7.2500
      NaN
      S
      0
    
    
      2
      1
      1
      Cumings, Mrs. John Bradley (Florence Briggs Th...
      female
      38
      1
      0
      PC 17599
      71.2833
      C85
      C
      1
    
    
      3
      1
      3
      Heikkinen, Miss. Laina
      female
      26
      0
      0
      STON/O2. 3101282
      7.9250
      NaN
      S
      1
    
    
      4
      1
      1
      Futrelle, Mrs. Jacques Heath (Lily May Peel)
      female
      35
      1
      0
      113803
      53.1000
      C123
      S
      1
    
    
      5
      0
      3
      Allen, Mr. William Henry
      male
      35
      0
      0
      373450
      8.0500
      NaN
      S
      0



In [34]:

    
embarkedummy = pd.get_dummies(titanic.Embarked, prefix='Embarked')
embarkedummy.drop(embarkedummy.columns[0], axis=1, inplace=True)
titanic = pd.concat([titanic, embarkedummy], axis=1)
titanic.head()









    Out[34]:






  
    
      
      Survived
      Pclass
      Name
      Sex
      Age
      SibSp
      Parch
      Ticket
      Fare
      Cabin
      Embarked
      Sex_Female
      Embarked_Q
      Embarked_S
      Embarked_Q
      Embarked_S
      Embarked_Q
      Embarked_S
    
    
      PassengerId
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
  
  
    
      1
      0
      3
      Braund, Mr. Owen Harris
      male
      22
      1
      0
      A/5 21171
      7.2500
      NaN
      S
      0
      0
      1
      0
      1
      0
      1
    
    
      2
      1
      1
      Cumings, Mrs. John Bradley (Florence Briggs Th...
      female
      38
      1
      0
      PC 17599
      71.2833
      C85
      C
      1
      0
      0
      0
      0
      0
      0
    
    
      3
      1
      3
      Heikkinen, Miss. Laina
      female
      26
      0
      0
      STON/O2. 3101282
      7.9250
      NaN
      S
      1
      0
      1
      0
      1
      0
      1
    
    
      4
      1
      1
      Futrelle, Mrs. Jacques Heath (Lily May Peel)
      female
      35
      1
      0
      113803
      53.1000
      C123
      S
      1
      0
      1
      0
      1
      0
      1
    
    
      5
      0
      3
      Allen, Mr. William Henry
      male
      35
      0
      0
      373450
      8.0500
      NaN
      S
      0
      0
      1
      0
      1
      0
      1

Exercise 6.3 (2 points)

From the set of features ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked']

*Note, use the created categorical features for Sex and Embarked

Select the features that maximize the accuracy the model using K-Fold cross-validation



In [22]:

    
y = titanic['Survived']



In [23]:

    
features = ['Pclass', 'Age', 'SibSp', 'Parch', 'Fare','Sex_Female', 'Embarked_Q', 'Embarked_S']



In [24]:

    
import numpy as np
def comb(n,k) :
    return np.math.factorial(n) / (np.math.factorial(n-k) * np.math.factorial(k))



In [25]:

    
np.sum([comb(8,i) for i in range(0,8)])









    Out[25]:





255.0



In [26]:

    
import itertools

possible_models = []
for i in range(1,len(features)+1):
    possible_models.extend(list(itertools.combinations(features,i)))

possible_models









    Out[26]:





[('Pclass',),
 ('Age',),
 ('SibSp',),
 ('Parch',),
 ('Fare',),
 ('Sex_Female',),
 ('Embarked_Q',),
 ('Embarked_S',),
 ('Pclass', 'Age'),
 ('Pclass', 'SibSp'),
 ('Pclass', 'Parch'),
 ('Pclass', 'Fare'),
 ('Pclass', 'Sex_Female'),
 ('Pclass', 'Embarked_Q'),
 ('Pclass', 'Embarked_S'),
 ('Age', 'SibSp'),
 ('Age', 'Parch'),
 ('Age', 'Fare'),
 ('Age', 'Sex_Female'),
 ('Age', 'Embarked_Q'),
 ('Age', 'Embarked_S'),
 ('SibSp', 'Parch'),
 ('SibSp', 'Fare'),
 ('SibSp', 'Sex_Female'),
 ('SibSp', 'Embarked_Q'),
 ('SibSp', 'Embarked_S'),
 ('Parch', 'Fare'),
 ('Parch', 'Sex_Female'),
 ('Parch', 'Embarked_Q'),
 ('Parch', 'Embarked_S'),
 ('Fare', 'Sex_Female'),
 ('Fare', 'Embarked_Q'),
 ('Fare', 'Embarked_S'),
 ('Sex_Female', 'Embarked_Q'),
 ('Sex_Female', 'Embarked_S'),
 ('Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp'),
 ('Pclass', 'Age', 'Parch'),
 ('Pclass', 'Age', 'Fare'),
 ('Pclass', 'Age', 'Sex_Female'),
 ('Pclass', 'Age', 'Embarked_Q'),
 ('Pclass', 'Age', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch'),
 ('Pclass', 'SibSp', 'Fare'),
 ('Pclass', 'SibSp', 'Sex_Female'),
 ('Pclass', 'SibSp', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Embarked_S'),
 ('Pclass', 'Parch', 'Fare'),
 ('Pclass', 'Parch', 'Sex_Female'),
 ('Pclass', 'Parch', 'Embarked_Q'),
 ('Pclass', 'Parch', 'Embarked_S'),
 ('Pclass', 'Fare', 'Sex_Female'),
 ('Pclass', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Fare', 'Embarked_S'),
 ('Pclass', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch'),
 ('Age', 'SibSp', 'Fare'),
 ('Age', 'SibSp', 'Sex_Female'),
 ('Age', 'SibSp', 'Embarked_Q'),
 ('Age', 'SibSp', 'Embarked_S'),
 ('Age', 'Parch', 'Fare'),
 ('Age', 'Parch', 'Sex_Female'),
 ('Age', 'Parch', 'Embarked_Q'),
 ('Age', 'Parch', 'Embarked_S'),
 ('Age', 'Fare', 'Sex_Female'),
 ('Age', 'Fare', 'Embarked_Q'),
 ('Age', 'Fare', 'Embarked_S'),
 ('Age', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'Sex_Female', 'Embarked_S'),
 ('Age', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Parch', 'Fare'),
 ('SibSp', 'Parch', 'Sex_Female'),
 ('SibSp', 'Parch', 'Embarked_Q'),
 ('SibSp', 'Parch', 'Embarked_S'),
 ('SibSp', 'Fare', 'Sex_Female'),
 ('SibSp', 'Fare', 'Embarked_Q'),
 ('SibSp', 'Fare', 'Embarked_S'),
 ('SibSp', 'Sex_Female', 'Embarked_Q'),
 ('SibSp', 'Sex_Female', 'Embarked_S'),
 ('SibSp', 'Embarked_Q', 'Embarked_S'),
 ('Parch', 'Fare', 'Sex_Female'),
 ('Parch', 'Fare', 'Embarked_Q'),
 ('Parch', 'Fare', 'Embarked_S'),
 ('Parch', 'Sex_Female', 'Embarked_Q'),
 ('Parch', 'Sex_Female', 'Embarked_S'),
 ('Parch', 'Embarked_Q', 'Embarked_S'),
 ('Fare', 'Sex_Female', 'Embarked_Q'),
 ('Fare', 'Sex_Female', 'Embarked_S'),
 ('Fare', 'Embarked_Q', 'Embarked_S'),
 ('Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch'),
 ('Pclass', 'Age', 'SibSp', 'Fare'),
 ('Pclass', 'Age', 'SibSp', 'Sex_Female'),
 ('Pclass', 'Age', 'SibSp', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Fare'),
 ('Pclass', 'Age', 'Parch', 'Sex_Female'),
 ('Pclass', 'Age', 'Parch', 'Embarked_Q'),
 ('Pclass', 'Age', 'Parch', 'Embarked_S'),
 ('Pclass', 'Age', 'Fare', 'Sex_Female'),
 ('Pclass', 'Age', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Age', 'Fare', 'Embarked_S'),
 ('Pclass', 'Age', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Fare'),
 ('Pclass', 'SibSp', 'Parch', 'Sex_Female'),
 ('Pclass', 'SibSp', 'Parch', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Parch', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Fare', 'Sex_Female'),
 ('Pclass', 'SibSp', 'Fare', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Fare', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Parch', 'Fare', 'Sex_Female'),
 ('Pclass', 'Parch', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Parch', 'Fare', 'Embarked_S'),
 ('Pclass', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Fare'),
 ('Age', 'SibSp', 'Parch', 'Sex_Female'),
 ('Age', 'SibSp', 'Parch', 'Embarked_Q'),
 ('Age', 'SibSp', 'Parch', 'Embarked_S'),
 ('Age', 'SibSp', 'Fare', 'Sex_Female'),
 ('Age', 'SibSp', 'Fare', 'Embarked_Q'),
 ('Age', 'SibSp', 'Fare', 'Embarked_S'),
 ('Age', 'SibSp', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'SibSp', 'Sex_Female', 'Embarked_S'),
 ('Age', 'SibSp', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Parch', 'Fare', 'Sex_Female'),
 ('Age', 'Parch', 'Fare', 'Embarked_Q'),
 ('Age', 'Parch', 'Fare', 'Embarked_S'),
 ('Age', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Age', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Age', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Parch', 'Fare', 'Sex_Female'),
 ('SibSp', 'Parch', 'Fare', 'Embarked_Q'),
 ('SibSp', 'Parch', 'Fare', 'Embarked_S'),
 ('SibSp', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('SibSp', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('SibSp', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('SibSp', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('SibSp', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Sex_Female'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Sex_Female'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Sex_Female'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Sex_Female'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Embarked_Q'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Age', 'SibSp', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Age', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('SibSp', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Parch', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'SibSp', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass', 'Age', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass',
  'SibSp',
  'Parch',
  'Fare',
  'Sex_Female',
  'Embarked_Q',
  'Embarked_S'),
 ('Age', 'SibSp', 'Parch', 'Fare', 'Sex_Female', 'Embarked_Q', 'Embarked_S'),
 ('Pclass',
  'Age',
  'SibSp',
  'Parch',
  'Fare',
  'Sex_Female',
  'Embarked_Q',
  'Embarked_S')]



In [27]:

    
import itertools

possible_models = [] 
for i in range(1,len(features)+1):
    possible_models.extend(list(itertools.combinations(features,i)))



In [29]:

    
from sklearn.linear_model import LogisticRegression
from sklearn.cross_validation import cross_val_score

Y = titanic.Survived

resultado = pd.DataFrame(index=possible_models,columns=['presicion'])
for i in range(len(possible_models)):
    X = titanic[list(possible_models[i])]
    reglogistica = LogisticRegression(C=1e9)
    resultado.iloc[i] = cross_val_score(reglogistica, X, Y, cv=10, scoring='accuracy').mean()



In [30]:

    
resultado.head()









    Out[30]:






  
    
      
      presicion
    
  
  
    
      (Pclass,)
      0.67927
    
    
      (Age,)
      0.61617
    
    
      (SibSp,)
      0.61617
    
    
      (Parch,)
      0.60833
    
    
      (Fare,)
      0.663487



In [31]:

    
resultado.sort_values('presicion',ascending=False).head(1)









    Out[31]:






  
    
      
      presicion
    
  
  
    
      (Pclass, Age, SibSp, Sex_Female, Embarked_S)
      0.801369

Bonus Exercise 6.4 (3 points)

Now which are the best set of features selected by AUC



In [ ]:

	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
PassengerId
1	0	3	Braund, Mr. Owen Harris	male	22	1	0	A/5 21171	7.2500	NaN	S
2	1	1	Cumings, Mrs. John Bradley (Florence Briggs Th...	female	38	1	0	PC 17599	71.2833	C85	C
3	1	3	Heikkinen, Miss. Laina	female	26	0	0	STON/O2. 3101282	7.9250	NaN	S
4	1	1	Futrelle, Mrs. Jacques Heath (Lily May Peel)	female	35	1	0	113803	53.1000	C123	S
5	0	3	Allen, Mr. William Henry	male	35	0	0	373450	8.0500	NaN	S

	presicion
(Pclass,)	0.67927
(Age,)	0.61617
(SibSp,)	0.61617
(Parch,)	0.60833
(Fare,)	0.663487