

In [1]:

    
%matplotlib inline

Selecting dimensionality reduction with Pipeline and GridSearchCV

This example constructs a pipeline that does dimensionality reduction followed by prediction with a support vector classifier. It demonstrates the use of GridSearchCV and Pipeline to optimize over different classes of estimators in a single CV run -- unsupervised PCA and NMF dimensionality reductions are compared to univariate feature selection during the grid search.

Additionally, Pipeline can be instantiated with the memory argument to memoize the transformers within the pipeline, avoiding to fit again the same transformers over and over.

Note that the use of memory to enable caching becomes interesting when the fitting of a transformer is costly.

Illustration of Pipeline and GridSearchCV

#

This section illustrates the use of a Pipeline with GridSearchCV



In [2]:

    
# Authors: Robert McGibbon, Joel Nothman, Guillaume Lemaitre

from __future__ import print_function, division

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2

print(__doc__)

pipe = Pipeline([
    ('reduce_dim', PCA()),
    ('classify', LinearSVC())
])

N_FEATURES_OPTIONS = [2, 4, 8]
C_OPTIONS = [1, 10, 100, 1000]
param_grid = [
    {
        'reduce_dim': [PCA(iterated_power=7), NMF()],
        'reduce_dim__n_components': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
    {
        'reduce_dim': [SelectKBest(chi2)],
        'reduce_dim__k': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
]
reducer_labels = ['PCA', 'NMF', 'KBest(chi2)']

grid = GridSearchCV(pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

mean_scores = np.array(grid.cv_results_['mean_test_score'])
# scores are in the order of param_grid iteration, which is alphabetical
mean_scores = mean_scores.reshape(len(C_OPTIONS), -1, len(N_FEATURES_OPTIONS))
# select score for best C
mean_scores = mean_scores.max(axis=0)
bar_offsets = (np.arange(len(N_FEATURES_OPTIONS)) *
               (len(reducer_labels) + 1) + .5)

plt.figure()
COLORS = 'bgrcmyk'
for i, (label, reducer_scores) in enumerate(zip(reducer_labels, mean_scores)):
    plt.bar(bar_offsets + i, reducer_scores, label=label, color=COLORS[i])

plt.title("Comparing feature reduction techniques")
plt.xlabel('Reduced number of features')
plt.xticks(bar_offsets + len(reducer_labels) / 2, N_FEATURES_OPTIONS)
plt.ylabel('Digit classification accuracy')
plt.ylim((0, 1))
plt.legend(loc='upper left')









    



Automatically created module for IPython interactive environment






    Out[2]:





<matplotlib.legend.Legend at 0x7f5e1ce52978>

Caching transformers within a Pipeline

#

It is sometimes worthwhile storing the state of a specific transformer since it could be used again. Using a pipeline in GridSearchCV triggers such situations. Therefore, we use the argument memory to enable caching.

.. warning:: Note that this example is, however, only an illustration since for this specific case fitting PCA is not necessarily slower than loading the cache. Hence, use the memory constructor parameter when the fitting of a transformer is costly.



In [3]:

    
from tempfile import mkdtemp
from shutil import rmtree
from sklearn.externals.joblib import Memory

# Create a temporary folder to store the transformers of the pipeline
cachedir = mkdtemp()
memory = Memory(cachedir=cachedir, verbose=10)
cached_pipe = Pipeline([('reduce_dim', PCA()),
                        ('classify', LinearSVC())],
                       memory=memory)

# This time, a cached pipeline will be used within the grid search
grid = GridSearchCV(cached_pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

# Delete the temporary cache before exiting
rmtree(cachedir)









    



________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/7bb38927c34d6eccd13f99f27face686
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cedb15364a394e2e0c34edc661947a78
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cf9470dca2983ecf6f6c572172ab886b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5728c52f62e3500d3ce506bae65f2268
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b1644cbf62bb2549d7863bc578e6f701
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/f9a99e861765cce383dad32b4f9e7fdb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/87df8cc8e2dcfdf4f54f8baa28bef2bf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6424129fdae9750488104413a0cfc32d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/524a6ba951f04a6c7bcfd21e59a5262b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/24771e8acf46f54b97a56055a38f81c1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/1d1c509547dc7976dec45942b2968050
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d2abaecc3d56cf4a9432ca80728fcdeb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/aba1b54ff4b741f54d476a8d17209cc4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6101f78765f4eebaa1a2535b030c4521
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/18741a335473e49d64449563e22d4b51
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b2eed2e6cbd1a3e6d7e5dfbbdf86856d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3bc30505ba640b8b5296caf98b90c3eb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5c417a2929a2410b8420c81c40657bf2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/7bb38927c34d6eccd13f99f27face686
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cedb15364a394e2e0c34edc661947a78
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cf9470dca2983ecf6f6c572172ab886b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5728c52f62e3500d3ce506bae65f2268
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b1644cbf62bb2549d7863bc578e6f701
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/f9a99e861765cce383dad32b4f9e7fdb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/87df8cc8e2dcfdf4f54f8baa28bef2bf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6424129fdae9750488104413a0cfc32d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/524a6ba951f04a6c7bcfd21e59a5262b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/24771e8acf46f54b97a56055a38f81c1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/1d1c509547dc7976dec45942b2968050
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d2abaecc3d56cf4a9432ca80728fcdeb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/aba1b54ff4b741f54d476a8d17209cc4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6101f78765f4eebaa1a2535b030c4521
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/18741a335473e49d64449563e22d4b51
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b2eed2e6cbd1a3e6d7e5dfbbdf86856d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3bc30505ba640b8b5296caf98b90c3eb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5c417a2929a2410b8420c81c40657bf2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/7bb38927c34d6eccd13f99f27face686
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cedb15364a394e2e0c34edc661947a78
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/cf9470dca2983ecf6f6c572172ab886b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5728c52f62e3500d3ce506bae65f2268
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b1644cbf62bb2549d7863bc578e6f701
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/f9a99e861765cce383dad32b4f9e7fdb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/87df8cc8e2dcfdf4f54f8baa28bef2bf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6424129fdae9750488104413a0cfc32d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/524a6ba951f04a6c7bcfd21e59a5262b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/24771e8acf46f54b97a56055a38f81c1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/1d1c509547dc7976dec45942b2968050
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d2abaecc3d56cf4a9432ca80728fcdeb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/aba1b54ff4b741f54d476a8d17209cc4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/6101f78765f4eebaa1a2535b030c4521
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/18741a335473e49d64449563e22d4b51
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b2eed2e6cbd1a3e6d7e5dfbbdf86856d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3bc30505ba640b8b5296caf98b90c3eb
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/5c417a2929a2410b8420c81c40657bf2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x7f5e1da0b268>), None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/e590f9356da7c4ea0aa8a11bc86e4c10
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/a479aac6d57ef99a2f270b903ae36893
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/bce696de6e7f3805a58c19eec937f3af
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b3e00ebf99161d1df6c0e3b463b4e9e6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/46bd674671c11ad204c9d4d10c519318
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/0d0635eac80f3d5be46e8b71056ccff9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3c0527ceb916b768d686ffc66d414262
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d9a70decb8e7f0cb34cb52a7a8f1f83a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/c77a89643d692d20b9e4e9bc54023f58
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/e590f9356da7c4ea0aa8a11bc86e4c10
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/a479aac6d57ef99a2f270b903ae36893
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/bce696de6e7f3805a58c19eec937f3af
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b3e00ebf99161d1df6c0e3b463b4e9e6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/46bd674671c11ad204c9d4d10c519318
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/0d0635eac80f3d5be46e8b71056ccff9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3c0527ceb916b768d686ffc66d414262
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d9a70decb8e7f0cb34cb52a7a8f1f83a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/c77a89643d692d20b9e4e9bc54023f58
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/e590f9356da7c4ea0aa8a11bc86e4c10
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/a479aac6d57ef99a2f270b903ae36893
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/bce696de6e7f3805a58c19eec937f3af
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/b3e00ebf99161d1df6c0e3b463b4e9e6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/46bd674671c11ad204c9d4d10c519318
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/0d0635eac80f3d5be46e8b71056ccff9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/3c0527ceb916b768d686ffc66d414262
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/d9a70decb8e7f0cb34cb52a7a8f1f83a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpr04cztt1/joblib/sklearn/pipeline/_fit_transform_one/c77a89643d692d20b9e4e9bc54023f58
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False), 
None, array([[ 0., ...,  0.],
       ..., 
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min

The PCA fitting is only computed at the evaluation of the first configuration of the C parameter of the LinearSVC classifier. The other configurations of C will trigger the loading of the cached PCA estimator data, leading to save processing time. Therefore, the use of caching the pipeline using memory is highly beneficial when fitting a transformer is costly.



In [5]:

    
plt.show()



In [6]:

    
import numpy as np
from sklearn.decomposition import PCA
X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
pca = PCA(n_components=2)
pca.fit(X)


print(pca.explained_variance_ratio_)  

print(pca.singular_values_)









    



[ 0.99244289  0.00755711]
[ 6.30061232  0.54980396]



In [ ]: