This session will cover the basics of Scikit-Learn, a popular package containing a collection of tools for machine learning written in Python. See more at http://scikit-learn.org.
Let's start by loading some pre-existing datasets in the scikit-learn, which comes with a few standard datasets. For example, the iris and digits datasets for classification and the boston house prices dataset for regression. Using these existing datasets, we can easily test the algorithms that we are interested in.
A dataset is a dictionary-like object that holds all the data and some metadata about the data. This data is stored in the .data member, which is a n_samples, n_features array. In the case of supervised problem, one or more response variables are stored in the .target member. More details on the different datasets can be found in the dedicated section.
In [1]:
import warnings
warnings.filterwarnings("ignore")
from sklearn import datasets
import matplotlib.pyplot as plt
import numpy as np
plt.style.use('seaborn-poster')
%matplotlib inline
In [2]:
iris = datasets.load_iris()
In [3]:
print(iris.feature_names)
# only print the first 10 samples
print(iris.data[:10])
print('We have %d data samples with %d features'%(iris.data.shape[0], iris.data.shape[1]))
The data is always a 2D array, shape (n_samples, n_features), although the original data may have had a different shape. The following prints out the target names and the representatoin of the target using 0, 1, 2. Each of them represent a class.
In [4]:
print(iris.target_names)
print(set(iris.target))
In [5]:
digits = datasets.load_digits()
print('We have %d samples'%len(digits.target))
In [6]:
print(digits.data)
print('The targets are:')
print(digits.target_names)
In the digits, each original sample is an image of shape (8, 8) and it is flattened into a 64 dimension vector here.
In [7]:
print(digits.data.shape)
In [8]:
## plot the first 64 samples, and get a sense of the data
fig = plt.figure(figsize = (8,8))
fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)
for i in range(64):
ax = fig.add_subplot(8, 8, i+1, xticks=[], yticks=[])
ax.imshow(digits.images[i],cmap=plt.cm.binary,interpolation='nearest')
ax.text(0, 7, str(digits.target[i]))
The Boston housing dataset reports the median value of owner-occupied homes in various places in the Boston area, together with several variables which might help to explain the variation in median value, such as Crime (CRIM), areas of non-retail business in the town (INDUS), the age of people who own the house (AGE), and there are many other attributes that you can find the details here.
In [9]:
boston = datasets.load_boston()
print(boston.DESCR)
In [10]:
boston.feature_names
Out[10]:
In [11]:
# let's just plot the average number of rooms per dwelling with the price
plt.figure(figsize = (10,8))
plt.plot(boston.data[:,5], boston.target, 'o')
plt.xlabel('Number of rooms')
plt.ylabel('Price (thousands)')
Out[11]:
In [12]:
from sklearn.linear_model import LinearRegression
Estimator parameters: All the parameters of an estimator can be set when it is instantiated, and have suitable default values:
In [13]:
# you can check the parameters as
LinearRegression?
In [14]:
# let's change one parameter
model = LinearRegression(normalize=True)
print(model.normalize)
In [15]:
print(model)
Estimated Model parameters: When data is fit with an estimator, parameters are estimated from the data at hand. All the estimated parameters are attributes of the estimator object ending by an underscore:
In [16]:
x = np.arange(10)
y = 2 * x + 1
In [17]:
plt.figure(figsize = (10,8))
plt.plot(x,y,'o')
Out[17]:
In [18]:
# generate noise between -1 to 1
# this seed is just to make sure your results are the same as mine
np.random.seed(42)
noise = 2 * np.random.rand(10) - 1
# add noise to the data
y_noise = y + noise
In [19]:
plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')
Out[19]:
In [20]:
# The input data for sklearn is 2D: (samples == 10 x features == 1)
X = x[:, np.newaxis]
print(X)
print(y_noise)
In [21]:
# model fitting is via the fit function
model.fit(X, y_noise)
Out[21]:
In [22]:
# underscore at the end indicates a fit parameter
print(model.coef_)
print(model.intercept_)
In [23]:
# then we can use the fitted model to predict new data
predicted = model.predict(X)
In [24]:
plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')
plt.plot(x,predicted, label = 'Prediction')
plt.legend()
Out[24]:
In the next section, we will use support vector machine (SVM) to do a classification problem, have a look of the sklearn API, and find out which class we will use for a classification problem using SVM. (hint: we will use the one with C support).
In [ ]:
%load ../solutions/solution_01.py