In [61]:
# This tells matplotlib not to try opening a new window for each plot.
%matplotlib inline
# General libraries.
import re
import time as time
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
import matplotlib.cm as cm
from matplotlib.ticker import FormatStrFormatter
from itertools import product
import pandas as pd
from IPython.display import display, HTML
# feature analysis and selection
from sklearn.decomposition import PCA, KernelPCA
from sklearn.feature_selection import SelectKBest
from sklearn.feature_extraction import DictVectorizer
# Preprocessing
from sklearn.preprocessing import FunctionTransformer, LabelEncoder, OneHotEncoder, Imputer
from sklearn.model_selection import train_test_split, cross_val_score
# Processing
from sklearn.pipeline import Pipeline, make_pipeline, FeatureUnion
from sklearn.metrics import explained_variance_score, mean_absolute_error, mean_squared_error, r2_score
from sklearn import metrics
# SKLearn
from statsmodels.regression.linear_model import OLS
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.svm import SVR
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import BernoulliNB, MultinomialNB
#from sklearn.grid_search import GridSearchCV
from sklearn.model_selection import GridSearchCV
from sklearn.cluster import KMeans
from sklearn.mixture import GaussianMixture
Data Fields
SOC, pH, Ca, P, Sand are the five target variables for predictions. The data have been monotonously transformed from the original measurements and thus include negative values.
PIDN: unique soil sample identifier
SOC: Soil organic carbon
P: Mehlich-3 extractable Phosphorus
Sand: Sand content
m7497.96 - m599.76: There are 3,578 mid-infrared absorbance measurements. For example, the "m7497.96" column is the absorbance at wavenumber 7497.96 cm-1. We suggest you to remove spectra CO2 bands which are in the region m2379.76 to m2352.76, but you do not have to.
Depth: Depth of the soil sample (2 categories: "Topsoil", "Subsoil")
Some potential spatial predictors from remote sensing data sources are included. Short variable descriptions are provided below and additional descriptions can be found at AfSIS data. The data have been mean centered and scaled.
In [62]:
# Load training data
X = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=range(1, 3594)) # Load columns 1 to 3594 inclusive
n = np.genfromtxt('training.csv',
delimiter=',',
max_rows = 1,
names = True,
usecols=range(1, 3594)) # Load columns 1 to 3594 inclusive
feature_names = np.asarray(n.dtype.names)
Depth = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3594) # Load Depth values
PIDN = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=0) # Load the PIDN for reference
Ca = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3595) # Load Mehlich-3 extractable Calcium data
P = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3596) # Load Mehlich-3 extractable Phosphorus data
pH = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3597) # Load pH data
SOC = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3598) # Load Soil Organic Carbon data
Sand = np.genfromtxt('training.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3599) # Load Sand Content data
# Outcome (or response) variable list
y_var_labels = ['Ca', 'P', 'pH', 'SOC', 'Sand']
y_vars = [Ca, P, pH, SOC, Sand]
# Color map for outcome variables
colors = ['orange', 'yellowgreen', 'powderblue', 'sienna', 'tan']
In [63]:
# Load test data
test_x = np.genfromtxt('sorted_test.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=range(1, 3594)) # Load columns 0 to 3594 inclusive
test_depth = np.genfromtxt('sorted_test.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=3594) # Load Depth values
test_ids = np.genfromtxt('sorted_test.csv',
delimiter=',',
dtype=None,
skip_header = 1,
usecols=0) # Load columns 0 to 3594 inclusive
In [71]:
# Transform depth and concatenate to X and test_x for use
le = LabelEncoder()
depth_enc = le.fit(Depth).transform(Depth).astype(np.float64)
test_depth_enc = le.fit(test_depth).transform(test_depth).astype(np.float64)
X_wDepth = np.concatenate((X, depth_enc.reshape(1,-1).T), axis=1)
test_x_wdepth = np.concatenate((test_x, test_depth_enc.reshape(1,-1).T), axis=1)
In [64]:
# Inspect the data shapes
print "Training data shape: ", X.shape
print "Feature name shape: ", feature_names.shape
print "PIDN data shape: ", PIDN.shape
print "Depth data shape: ", Depth.shape
print "Ca data shape: ", Ca.shape
print "P data shape: ", P.shape
print "pH data shape: ", pH.shape
print "SOC data shape: ", SOC.shape
print "Sand data shape: ", Sand.shape
print "Test data shape: ", test_x.shape
print "Test_ids shape: ", test_ids.shape
Training data shape: (1157, 3593)
Feature name shape: (3593,)
PIDN data shape: (1157,)
Depth data shape: (1157,)
Ca data shape: (1157,)
P data shape: (1157,)
pH data shape: (1157,)
SOC data shape: (1157,)
Sand data shape: (1157,)
Test data shape: (727, 3593)
Test_ids shape: (727,)
In [65]:
# Inspect the data in the five response variables
print "Ca: total = %d, max = %0.2f, mean = %0.2f, min = %0.2f" % (Ca.shape[0], np.max(Ca), np.mean(Ca), np.min(Ca))
print "P: total = %d, max = %0.2f, mean = %0.2f, min = %0.2f" % (P.shape[0], np.max(P), np.mean(P), np.min(P))
print "pH: total = %d, max = %0.2f, mean = %0.2f, min = %0.2f" % (pH.shape[0], np.max(pH), np.mean(pH), np.min(pH))
print "SOC: total = %d, max = %0.2f, mean = %0.2f, min = %0.2f" % (SOC.shape[0], np.max(SOC), np.mean(SOC), np.min(SOC))
print "Sand: total = %d, max = %0.2f, mean = %0.2f, min = %0.2f" % (Sand.shape[0], np.max(Sand),
np.mean(Sand), np.min(Sand))
def plot_hist(ind, data, max_y, title, color):
counts, bins, patches = ax[ind].hist(data, facecolor=color, edgecolor='gray')
# set the ticks to be at the edges of the bins.
ax[ind].set_xticks(bins)
# set the limits for x and y
ax[ind].set_xlim([np.min(data),np.max(data)])
ax[ind].set_ylim([0,max_y])
# set the xaxis's tick labels to be formatted with 1 decimal place
ax[ind].xaxis.set_major_formatter(FormatStrFormatter('%0.1f'))
ax[ind].set_title(title, fontsize=18)
# Label the raw counts and the percentages below the x-axis
bin_centers = 0.5 * np.diff(bins) + bins[:-1]
for count, x in zip(counts, bin_centers):
# Label the raw counts
ax[ind].annotate(str(count), xy=(x, 0), xycoords=('data', 'axes fraction'),
xytext=(0, -18), textcoords='offset points', va='top', ha='center')
# Label the percentages
percent = '%0.1f%%' % (100 * float(count) / counts.sum())
ax[ind].annotate(percent, xy=(x, 0), xycoords=('data', 'axes fraction'),
xytext=(0, -32), textcoords='offset points', va='top', ha='center')
fig, ax = plt.subplots(3, 2, figsize=(15, 20))
fig.subplots_adjust(hspace = 0.5, wspace=.2)
ax = ax.ravel()
# Ca
plot_hist(0, Ca, Ca.shape[0], 'Ca Value Histogram', colors[0])
# P
plot_hist(1, P, P.shape[0], 'P Value Histogram', colors[1])
#pH
plot_hist(2, pH, pH.shape[0], 'pH Value Histogram', colors[2])
#SOC
plot_hist(3, SOC, SOC.shape[0], 'SOC Value Histogram', colors[3])
#Sand
plot_hist(4, Sand, Sand.shape[0], 'Sand Value Histogram', colors[4])
# delete the last subplot
fig.delaxes(ax[5])
Ca: total = 1157, max = 9.65, mean = 0.01, min = -0.54
P: total = 1157, max = 13.27, mean = -0.01, min = -0.42
pH: total = 1157, max = 3.42, mean = -0.03, min = -1.89
SOC: total = 1157, max = 7.62, mean = 0.08, min = -0.86
Sand: total = 1157, max = 2.25, mean = -0.01, min = -1.49
In [66]:
# Inspect the data in the predictor variables
def plot_data(ind, data_x, data_y, aspect, title, color):
ax[ind].set_title(title, fontsize=18)
ax[ind].set_xlabel('Predictor Values', fontsize=14)
ax[ind].set_ylabel('Ca Values', fontsize=12)
ax[ind].set_aspect(aspect = aspect, adjustable='box')
ax[ind].grid(True)
ax[ind].scatter(data_x, data_y, color = color, alpha = 0.2, marker = 'o', edgecolors = 'black')
# set up the grid plot
fig, ax = plt.subplots(2, 3, figsize=(15, 20))
#fig.subplots_adjust(hspace = 0.5, wspace=.2)
ax = ax.ravel()
# select the predictor range (note this is influenced by the PCA below)
my_col = 20
X_sub = np.ravel(X[:,:my_col].reshape(-1,1))
# Ca
plot_data(0, X_sub, np.repeat(Ca, my_col), 0.1, 'Ca vs. %d Predictors' % my_col, colors[0])
# P
plot_data(1, X_sub, np.repeat(P, my_col), 0.1, 'P vs. %d Predictors' % my_col, colors[1])
#pH
plot_data(2, X_sub, np.repeat(pH, my_col), 0.2, 'pH vs. %d Predictors' % my_col, colors[2])
#SOC
plot_data(3, X_sub, np.repeat(SOC, my_col), 0.1, 'SOC vs. %d Predictors' % my_col, colors[3])
#Sand
plot_data(4, X_sub, np.repeat(Sand, my_col), 0.35, 'Sand vs. %d Predictors' % my_col, colors[4])
# delete the last subplot
fig.delaxes(ax[5])
Which features have more impact?
There are over three thousand features in this data, with few rows. Thus, we have a large k but small n data set to work with. Perhaps there is a subset of features to focus on.
Below, we investigate two variations of PCA to explain variances over the features. We observe that the first 20 components explain increasing portions of the variance, however after 20 components, the subsequent ones don't really help. The first 70-80 features will explain ~100% of the variance.
In [68]:
# Linear PCA using all of the features
n_comp = feature_names.shape[0]
pca_lin = PCA(n_components = n_comp)
pca_lin.fit(X)
pca_lin_cumsum = np.cumsum(pca_lin.explained_variance_ratio_)
# Non-linear kernel RBF PCA using all of the features
pca_kern = KernelPCA(n_components = n_comp, kernel = 'rbf')
pca_kern.fit(X)
# build the explained variance ratio list for pca_kern
explained_var_ratio_kern = []
for i in range(0, pca_kern.lambdas_.shape[0]):
explained_var_ratio_kern.append(pca_kern.lambdas_[i]/sum(pca_kern.lambdas_))
pca_kern_cumsum = np.cumsum(np.asarray(explained_var_ratio_kern))
# Plot the Information Gain graph
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(pca_lin_cumsum, color = 'purple', marker = 'o', ms = 5, mfc = 'red', label = 'pca_lin')
ax.plot(pca_kern_cumsum, color = 'purple', marker = 'o', ms = 5, mfc = 'yellow', label = 'pca_kern')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.91), shadow=False, scatterpoints=1)
fig.suptitle('Cummulative Information Gain', fontsize=18)
plt.xlabel('Number of Components', fontsize=14)
plt.ylabel('Cummulative Variance Ratio', fontsize=12)
plt.grid(True)
ax.set_xlim([0,30])
ax.set_ylim([0.5,1.0])
# Output variance fractions
print '\n-------------------------------------------'
print 'Fraction of the total variance in the training explained by first k components: \n'
for k in range(1,76):
print("%d \t %s \t %s \t %s" % (k, '{0:.2f}%'.format(pca_lin_cumsum[k-1] * 100),
'{0:.2f}%'.format(pca_kern_cumsum[k-1] * 100), feature_names[k-1]))
-------------------------------------------
Fraction of the total variance in the training explained by first k components:
1 70.54% 66.91% m749796
2 79.38% 75.72% m749604
3 85.54% 82.10% m749411
4 89.31% 85.88% m749218
5 91.69% 88.83% m749025
6 93.63% 90.85% m748832
7 95.14% 92.57% m748639
8 96.16% 94.10% m748446
9 96.75% 95.11% m748254
10 97.27% 95.74% m748061
11 97.74% 96.29% m747868
12 98.10% 96.76% m747675
13 98.42% 97.14% m747482
14 98.69% 97.49% m747289
15 98.90% 97.77% m747097
16 99.06% 98.00% m746904
17 99.20% 98.21% m746711
18 99.34% 98.40% m746518
19 99.44% 98.55% m746325
20 99.50% 98.69% m746132
21 99.56% 98.80% m745939
22 99.62% 98.90% m745747
23 99.67% 98.99% m745554
24 99.72% 99.07% m745361
25 99.75% 99.14% m745168
26 99.78% 99.20% m744975
27 99.81% 99.26% m744782
28 99.83% 99.31% m744589
29 99.85% 99.36% m744397
30 99.86% 99.40% m744204
31 99.88% 99.43% m744011
32 99.89% 99.46% m743818
33 99.90% 99.49% m743625
34 99.91% 99.52% m743432
35 99.92% 99.54% m74324
36 99.93% 99.57% m743047
37 99.93% 99.59% m742854
38 99.94% 99.61% m742661
39 99.94% 99.63% m742468
40 99.95% 99.65% m742275
41 99.95% 99.67% m742082
42 99.96% 99.69% m74189
43 99.96% 99.70% m741697
44 99.96% 99.71% m741504
45 99.96% 99.73% m741311
46 99.97% 99.74% m741118
47 99.97% 99.75% m740925
48 99.97% 99.76% m740733
49 99.97% 99.77% m74054
50 99.98% 99.78% m740347
51 99.98% 99.79% m740154
52 99.98% 99.80% m739961
53 99.98% 99.80% m739768
54 99.98% 99.81% m739575
55 99.98% 99.82% m739383
56 99.98% 99.83% m73919
57 99.99% 99.83% m738997
58 99.99% 99.84% m738804
59 99.99% 99.84% m738611
60 99.99% 99.85% m738418
61 99.99% 99.85% m738225
62 99.99% 99.86% m738033
63 99.99% 99.86% m73784
64 99.99% 99.87% m737647
65 99.99% 99.87% m737454
66 99.99% 99.88% m737261
67 99.99% 99.88% m737068
68 99.99% 99.88% m736876
69 99.99% 99.89% m736683
70 99.99% 99.89% m73649
71 99.99% 99.89% m736297
72 99.99% 99.90% m736104
73 99.99% 99.90% m735911
74 99.99% 99.90% m735718
75 100.00% 99.90% m735526
MODELING WORK ZONE BELOW
In [84]:
# Linear Regression with PCA combinations
y_pipelines_lin = []
y_scores_lin = []
start = time.time()
for ind, y in enumerate(y_vars):
X_train, X_test, y_train, y_test = train_test_split(X_wDepth, y, test_size=0.33, random_state=42)
# set up the train and test data
print '\n----------', y_var_labels[ind]
pca = PCA()
linear = LinearRegression()
steps = [('pca', pca), ('linear', linear)]
pipeline = Pipeline(steps)
parameters = dict(pca__n_components=list(range(20, 90, 10)),
linear__normalize=[True, False])
cv = GridSearchCV(pipeline, param_grid=parameters, verbose=0)
cv.fit(X_train, y_train)
print 'Cross_val_score: ', cross_val_score(cv, X_test, y_test)
y_predictions = cv.predict(X_test)
mse = mean_squared_error(y_test, y_predictions)
print 'Explained variance score: ', explained_variance_score(y_test, y_predictions)
print 'Mean absolute error: ', mean_absolute_error(y_test, y_predictions)
print 'Mean squared error: ', mse
print 'R2 score: ', r2_score(y_test, y_predictions)
display(pd.DataFrame.from_dict(cv.cv_results_))
# capture the best pipeline estimator and mse value
y_pipelines_lin.append(cv.best_estimator_)
y_scores_lin.append(mse)
print 'Completed in %0.2f sec' % (start-time.time())
---------- Ca
Cross_val_score: [ 0.85570742 0.63792012 0.80330096]
Explained variance score: 0.901076165334
Mean absolute error: 0.196188379361
Mean squared error: 0.168774592688
R2 score: 0.90099366767
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_linear__normalize
param_pca__n_components
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.401595
0.016937
0.751867
0.825906
True
20
{u'pca__n_components': 20, u'linear__normalize...
13
0.785879
0.823166
0.727543
0.836301
0.742047
0.818251
0.080252
0.003579
0.024812
0.007619
1
0.292449
0.016768
0.809155
0.875621
True
30
{u'pca__n_components': 30, u'linear__normalize...
12
0.845905
0.865919
0.781084
0.889915
0.800332
0.871029
0.008462
0.000870
0.027196
0.010321
2
0.413034
0.026947
0.838997
0.913629
True
40
{u'pca__n_components': 40, u'linear__normalize...
10
0.893391
0.900660
0.776750
0.934486
0.846640
0.905741
0.032198
0.012370
0.047939
0.014894
3
0.392399
0.022573
0.857942
0.923122
True
50
{u'pca__n_components': 50, u'linear__normalize...
1
0.903522
0.916935
0.803975
0.939682
0.866154
0.912749
0.017331
0.006179
0.041065
0.011834
4
0.368343
0.022410
0.853537
0.927500
True
60
{u'pca__n_components': 60, u'linear__normalize...
3
0.899093
0.922126
0.810122
0.942066
0.851220
0.918309
0.034430
0.004476
0.036371
0.010417
5
0.347605
0.020352
0.849985
0.935038
True
70
{u'pca__n_components': 70, u'linear__normalize...
6
0.900830
0.930248
0.800939
0.944889
0.847989
0.929978
0.016688
0.000166
0.040818
0.006966
6
0.421688
0.023069
0.849196
0.941703
True
80
{u'pca__n_components': 80, u'linear__normalize...
7
0.906221
0.937466
0.789157
0.949431
0.851987
0.938212
0.017380
0.001076
0.047847
0.005473
7
0.301222
0.015382
0.751867
0.825906
False
20
{u'pca__n_components': 20, u'linear__normalize...
14
0.785879
0.823166
0.727543
0.836301
0.742047
0.818251
0.036415
0.000671
0.024812
0.007619
8
0.325893
0.016419
0.809155
0.875621
False
30
{u'pca__n_components': 30, u'linear__normalize...
11
0.845906
0.865919
0.781084
0.889915
0.800332
0.871029
0.012175
0.000476
0.027196
0.010321
9
0.451195
0.025117
0.839001
0.913628
False
40
{u'pca__n_components': 40, u'linear__normalize...
9
0.893389
0.900659
0.776744
0.934486
0.846659
0.905738
0.087783
0.010919
0.047942
0.014894
10
0.482947
0.020592
0.857938
0.923122
False
50
{u'pca__n_components': 50, u'linear__normalize...
2
0.903511
0.916937
0.803973
0.939682
0.866154
0.912749
0.065446
0.000751
0.041062
0.011834
11
0.437326
0.026386
0.853423
0.927471
False
60
{u'pca__n_components': 60, u'linear__normalize...
4
0.899137
0.922128
0.809907
0.942034
0.851048
0.918251
0.063638
0.006914
0.036478
0.010419
12
0.454451
0.023779
0.850031
0.935036
False
70
{u'pca__n_components': 70, u'linear__normalize...
5
0.900850
0.930238
0.801001
0.944883
0.848046
0.929988
0.026515
0.002409
0.040800
0.006963
13
0.445644
0.029473
0.848784
0.941826
False
80
{u'pca__n_components': 80, u'linear__normalize...
8
0.906874
0.937749
0.787414
0.949531
0.851838
0.938196
0.044929
0.009692
0.048833
0.005452
---------- P
Cross_val_score: [ 0.16601529 -0.07910374 0.02768116]
Explained variance score: 0.0895600523212
Mean absolute error: 0.5352618264
Mean squared error: 1.10647335821
R2 score: 0.0891821398915
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_linear__normalize
param_pca__n_components
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.398007
0.019905
-0.051894
0.114146
True
20
{u'pca__n_components': 20, u'linear__normalize...
4
0.018079
0.133428
0.033473
0.088777
-0.207504
0.120232
0.082744
0.005006
0.110107
0.018730
1
0.293720
0.015602
-0.091704
0.143315
True
30
{u'pca__n_components': 30, u'linear__normalize...
5
0.020377
0.169673
0.045032
0.101407
-0.340954
0.158865
0.007554
0.000078
0.176364
0.029960
2
0.389901
0.018203
-0.110959
0.182868
True
40
{u'pca__n_components': 40, u'linear__normalize...
10
0.023842
0.206224
0.043404
0.143164
-0.400644
0.199216
0.033227
0.000919
0.204796
0.028220
3
0.457877
0.023460
-0.113389
0.255722
True
50
{u'pca__n_components': 50, u'linear__normalize...
13
-0.000942
0.301696
0.052708
0.202320
-0.392371
0.263151
0.090917
0.006886
0.198293
0.040909
4
0.455800
0.020258
-0.096064
0.297641
True
60
{u'pca__n_components': 60, u'linear__normalize...
7
-0.009222
0.346567
0.062305
0.247829
-0.341613
0.298526
0.039363
0.000397
0.175903
0.040314
5
0.395460
0.024339
-0.033347
0.355513
True
70
{u'pca__n_components': 70, u'linear__normalize...
1
0.050257
0.408086
0.159736
0.298802
-0.310358
0.359652
0.024229
0.003467
0.200729
0.044711
6
0.428809
0.024844
-0.110948
0.412695
True
80
{u'pca__n_components': 80, u'linear__normalize...
9
0.027658
0.460679
0.067044
0.371651
-0.428082
0.405754
0.008546
0.003195
0.224608
0.036675
7
0.320130
0.015826
-0.051894
0.114146
False
20
{u'pca__n_components': 20, u'linear__normalize...
3
0.018079
0.133428
0.033473
0.088777
-0.207504
0.120232
0.059182
0.001832
0.110107
0.018730
8
0.290308
0.015603
-0.091704
0.143315
False
30
{u'pca__n_components': 30, u'linear__normalize...
6
0.020377
0.169673
0.045031
0.101407
-0.340954
0.158865
0.004506
0.000116
0.176364
0.029960
9
0.489253
0.018448
-0.111010
0.182870
False
40
{u'pca__n_components': 40, u'linear__normalize...
11
0.023842
0.206226
0.043393
0.143165
-0.400789
0.199218
0.077873
0.001934
0.204862
0.028221
10
0.402547
0.021939
-0.113359
0.255721
False
50
{u'pca__n_components': 50, u'linear__normalize...
12
-0.000933
0.301693
0.052713
0.202320
-0.392293
0.263151
0.024878
0.004634
0.198260
0.040908
11
0.416457
0.021030
-0.096146
0.297048
False
60
{u'pca__n_components': 60, u'linear__normalize...
8
-0.009566
0.345376
0.059921
0.247490
-0.339129
0.298280
0.069836
0.000872
0.173978
0.039971
12
0.347005
0.020706
-0.033658
0.355659
False
70
{u'pca__n_components': 70, u'linear__normalize...
2
0.049498
0.408939
0.159181
0.298690
-0.309977
0.359348
0.008959
0.000295
0.200271
0.045085
13
0.533211
0.022799
-0.121338
0.414614
False
80
{u'pca__n_components': 80, u'linear__normalize...
14
0.024289
0.462132
0.064487
0.372969
-0.453355
0.408741
0.103923
0.001862
0.235118
0.036637
---------- pH
Cross_val_score: [ 0.68826016 0.71926307 0.76570321]
Explained variance score: 0.799937498717
Mean absolute error: 0.30094291544
Mean squared error: 0.176772099465
R2 score: 0.799273781985
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_linear__normalize
param_pca__n_components
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.701983
0.026909
0.684171
0.720647
True
20
{u'pca__n_components': 20, u'linear__normalize...
14
0.646453
0.733411
0.712437
0.710280
0.693770
0.718250
0.171247
0.000568
0.027787
0.009594
1
0.459710
0.020405
0.733384
0.782674
True
30
{u'pca__n_components': 30, u'linear__normalize...
11
0.714661
0.788172
0.742067
0.786209
0.743498
0.773640
0.106891
0.003574
0.013278
0.006438
2
0.610304
0.019331
0.759927
0.808876
True
40
{u'pca__n_components': 40, u'linear__normalize...
8
0.747043
0.815274
0.760863
0.806183
0.771923
0.805171
0.140424
0.001415
0.010182
0.004543
3
0.457173
0.020900
0.757657
0.821223
True
50
{u'pca__n_components': 50, u'linear__normalize...
10
0.751396
0.823192
0.762175
0.814440
0.759423
0.826036
0.031502
0.001770
0.004576
0.004935
4
0.474770
0.029363
0.762740
0.840075
True
60
{u'pca__n_components': 60, u'linear__normalize...
5
0.748152
0.848737
0.786107
0.834763
0.754018
0.836724
0.072326
0.006740
0.016680
0.006177
5
0.564476
0.022850
0.767861
0.846745
True
70
{u'pca__n_components': 70, u'linear__normalize...
4
0.749647
0.859065
0.791295
0.837879
0.762711
0.843291
0.140697
0.000234
0.017393
0.008987
6
0.685873
0.032198
0.785268
0.865361
True
80
{u'pca__n_components': 80, u'linear__normalize...
1
0.760928
0.868712
0.817554
0.855661
0.777418
0.871709
0.127607
0.003770
0.023780
0.006967
7
0.563744
0.024616
0.684171
0.720647
False
20
{u'pca__n_components': 20, u'linear__normalize...
13
0.646453
0.733411
0.712437
0.710280
0.693770
0.718250
0.071561
0.010831
0.027787
0.009594
8
0.493350
0.018993
0.733384
0.782674
False
30
{u'pca__n_components': 30, u'linear__normalize...
12
0.714661
0.788172
0.742066
0.786209
0.743498
0.773640
0.039291
0.003280
0.013278
0.006438
9
0.669867
0.021422
0.759928
0.808878
False
40
{u'pca__n_components': 40, u'linear__normalize...
7
0.747048
0.815277
0.760854
0.806180
0.771933
0.805177
0.094120
0.004917
0.010184
0.004543
10
0.607884
0.021201
0.757657
0.821219
False
50
{u'pca__n_components': 50, u'linear__normalize...
9
0.751394
0.823193
0.762177
0.814440
0.759425
0.826025
0.089288
0.001730
0.004577
0.004931
11
0.447556
0.041380
0.762721
0.840135
False
60
{u'pca__n_components': 60, u'linear__normalize...
6
0.748366
0.848732
0.786207
0.834818
0.753647
0.836854
0.044462
0.016182
0.016730
0.006136
12
0.546687
0.031930
0.768089
0.846758
False
70
{u'pca__n_components': 70, u'linear__normalize...
3
0.749771
0.859039
0.791506
0.837917
0.763060
0.843317
0.056947
0.013150
0.017410
0.008960
13
0.591968
0.029392
0.785012
0.865390
False
80
{u'pca__n_components': 80, u'linear__normalize...
2
0.760769
0.868679
0.816720
0.855711
0.777642
0.871781
0.037755
0.006266
0.023435
0.006961
---------- SOC
Cross_val_score: [ 0.87037411 0.89622083 0.85961808]
Explained variance score: 0.898629932712
Mean absolute error: 0.257032142328
Mean squared error: 0.154347468616
R2 score: 0.897618782823
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_linear__normalize
param_pca__n_components
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.399074
0.016594
0.799626
0.812817
True
20
{u'pca__n_components': 20, u'linear__normalize...
14
0.832424
0.798065
0.786214
0.820028
0.780114
0.820358
0.063428
0.003096
0.023369
0.010432
1
0.371319
0.017284
0.823940
0.846879
True
30
{u'pca__n_components': 30, u'linear__normalize...
11
0.835253
0.843558
0.823200
0.845433
0.813325
0.851647
0.019578
0.001353
0.008970
0.003457
2
0.456614
0.018377
0.832803
0.862331
True
40
{u'pca__n_components': 40, u'linear__normalize...
9
0.837140
0.857774
0.838184
0.862447
0.823068
0.866771
0.042116
0.001555
0.006890
0.003674
3
0.490796
0.022081
0.843454
0.886960
True
50
{u'pca__n_components': 50, u'linear__normalize...
8
0.849107
0.884587
0.854185
0.885183
0.827048
0.891110
0.088483
0.003310
0.011773
0.002944
4
0.562996
0.041530
0.866698
0.905370
True
60
{u'pca__n_components': 60, u'linear__normalize...
5
0.882216
0.900789
0.870770
0.906006
0.847047
0.909315
0.036247
0.011029
0.014647
0.003510
5
0.534162
0.040645
0.870927
0.919710
True
70
{u'pca__n_components': 70, u'linear__normalize...
4
0.889836
0.914927
0.869019
0.925610
0.853852
0.918592
0.099857
0.013325
0.014757
0.004432
6
0.708832
0.028850
0.881796
0.930048
True
80
{u'pca__n_components': 80, u'linear__normalize...
2
0.898598
0.923672
0.876698
0.938034
0.870026
0.928437
0.047880
0.007608
0.012211
0.005973
7
0.328411
0.017010
0.799626
0.812817
False
20
{u'pca__n_components': 20, u'linear__normalize...
13
0.832424
0.798065
0.786214
0.820028
0.780114
0.820358
0.011248
0.002965
0.023369
0.010432
8
0.324564
0.016928
0.823940
0.846879
False
30
{u'pca__n_components': 30, u'linear__normalize...
12
0.835253
0.843558
0.823200
0.845433
0.813325
0.851647
0.015036
0.001612
0.008970
0.003457
9
0.437030
0.018673
0.832803
0.862331
False
40
{u'pca__n_components': 40, u'linear__normalize...
10
0.837137
0.857774
0.838185
0.862447
0.823070
0.866772
0.051659
0.001621
0.006889
0.003675
10
0.467132
0.029230
0.843455
0.886960
False
50
{u'pca__n_components': 50, u'linear__normalize...
7
0.849106
0.884589
0.854183
0.885183
0.827054
0.891107
0.019858
0.009494
0.011770
0.002943
11
0.420701
0.021187
0.866605
0.905351
False
60
{u'pca__n_components': 60, u'linear__normalize...
6
0.882007
0.900656
0.870642
0.906101
0.847106
0.909295
0.019005
0.001170
0.014536
0.003566
12
0.460716
0.024483
0.871034
0.919729
False
70
{u'pca__n_components': 70, u'linear__normalize...
3
0.889924
0.914948
0.869230
0.925625
0.853876
0.918613
0.082493
0.004462
0.014777
0.004430
13
0.530572
0.030255
0.881927
0.930071
False
80
{u'pca__n_components': 80, u'linear__normalize...
1
0.898539
0.923689
0.876747
0.938036
0.870430
0.928487
0.064933
0.008048
0.012048
0.005963
---------- Sand
Cross_val_score: [ 0.87066726 0.82473215 0.86466119]
Explained variance score: 0.883016553299
Mean absolute error: 0.23952072496
Mean squared error: 0.11549187919
R2 score: 0.882836536731
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_linear__normalize
param_pca__n_components
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.426531
0.023525
0.717685
0.751043
True
20
{u'pca__n_components': 20, u'linear__normalize...
14
0.705694
0.763371
0.724223
0.749680
0.723185
0.740078
0.071271
0.010680
0.008506
0.009558
1
0.333824
0.016410
0.765231
0.807077
True
30
{u'pca__n_components': 30, u'linear__normalize...
11
0.768001
0.806540
0.738767
0.818617
0.788916
0.796075
0.015012
0.000762
0.020554
0.009211
2
0.439170
0.029930
0.805838
0.852251
True
40
{u'pca__n_components': 40, u'linear__normalize...
10
0.794451
0.860239
0.785730
0.858806
0.837376
0.837707
0.026046
0.012110
0.022562
0.010300
3
0.443421
0.053730
0.818983
0.871655
True
50
{u'pca__n_components': 50, u'linear__normalize...
8
0.810215
0.874381
0.802672
0.877151
0.844096
0.863431
0.011374
0.048742
0.018006
0.005924
4
0.389781
0.020891
0.825129
0.881398
True
60
{u'pca__n_components': 60, u'linear__normalize...
6
0.822379
0.884167
0.803426
0.886319
0.849592
0.873707
0.009026
0.002292
0.018935
0.005509
5
0.391120
0.022294
0.834415
0.889789
True
70
{u'pca__n_components': 70, u'linear__normalize...
4
0.832762
0.890024
0.804232
0.899813
0.866257
0.879530
0.027148
0.000451
0.025332
0.008282
6
0.472420
0.023430
0.844644
0.896479
True
80
{u'pca__n_components': 80, u'linear__normalize...
1
0.849223
0.896167
0.817790
0.905128
0.866902
0.888143
0.026895
0.000793
0.020298
0.006938
7
0.303684
0.016479
0.717685
0.751043
False
20
{u'pca__n_components': 20, u'linear__normalize...
13
0.705694
0.763371
0.724223
0.749680
0.723185
0.740078
0.007885
0.001943
0.008506
0.009558
8
0.358066
0.015952
0.765231
0.807077
False
30
{u'pca__n_components': 30, u'linear__normalize...
12
0.768001
0.806540
0.738767
0.818617
0.788915
0.796075
0.038798
0.000164
0.020554
0.009211
9
0.396145
0.021046
0.805840
0.852251
False
40
{u'pca__n_components': 40, u'linear__normalize...
9
0.794465
0.860240
0.785731
0.858805
0.837369
0.837708
0.002398
0.004244
0.022556
0.010300
10
0.508582
0.018919
0.818983
0.871656
False
50
{u'pca__n_components': 50, u'linear__normalize...
7
0.810215
0.874382
0.802675
0.877154
0.844093
0.863433
0.060894
0.001111
0.018004
0.005924
11
0.359351
0.020233
0.825443
0.881443
False
60
{u'pca__n_components': 60, u'linear__normalize...
5
0.823527
0.884384
0.803236
0.886280
0.849573
0.873666
0.004271
0.000648
0.018953
0.005554
12
0.401505
0.021691
0.834488
0.889710
False
70
{u'pca__n_components': 70, u'linear__normalize...
3
0.832921
0.889896
0.803970
0.899803
0.866578
0.879431
0.024306
0.000868
0.025567
0.008318
13
0.461868
0.022603
0.844411
0.896472
False
80
{u'pca__n_components': 80, u'linear__normalize...
2
0.849006
0.896232
0.817251
0.905023
0.866958
0.888159
0.023773
0.001191
0.020539
0.006887
Completed in -242.73 sec
In [85]:
print len(y_pipelines_lin)
print y_scores_lin
5
[0.16877459268842748, 1.1064733582082102, 0.17677209946462416, 0.1543474686162602, 0.11549187918978619]
In [86]:
# Linear Regression with PCA and SelectKBest
y_pipelines_linsel = []
y_scores_linsel = []
start = time.time()
for ind, y in enumerate(y_vars):
X_train, X_test, y_train, y_test = train_test_split(X_wDepth, y, test_size=0.33, random_state=42)
# set up the train and test data
print '\n----------', y_var_labels[ind]
pca = PCA(n_components=2)
selection = SelectKBest(k=1)
combined_features = FeatureUnion([('pca', pca), ('univ_select', selection)])
linear = LinearRegression()
steps = [('features', combined_features), ('linear', linear)]
pipeline = Pipeline(steps)
parameters = dict(features__pca__n_components=list(range(20, 90, 10)),
features__univ_select__k=[1, 2, 3],
linear__normalize=[True, False])
cv = GridSearchCV(pipeline, param_grid=parameters, verbose=0)
cv.fit(X_train, y_train)
print 'Cross_val_score: ', cross_val_score(cv, X_test, y_test)
y_predictions = cv.predict(X_test)
mse = mean_squared_error(y_test, y_predictions)
print 'Explained variance score: ', explained_variance_score(y_test, y_predictions)
print 'Mean absolute error: ', mean_absolute_error(y_test, y_predictions)
print 'Mean squared error: ', mse
print 'R2 score: ', r2_score(y_test, y_predictions)
display(pd.DataFrame.from_dict(cv.cv_results_))
# capture the best pipeline estimator and mse value
y_pipelines_linsel.append(cv.best_estimator_)
y_scores_linsel.append(mse)
print 'Completed in %0.2f sec' % (start-time.time())
---------- Ca
Cross_val_score: [ 0.85955475 0.60303506 0.81811134]
Explained variance score: 0.901317065989
Mean absolute error: 0.19647374878
Mean squared error: 0.168426718485
R2 score: 0.901197737183
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_features__pca__n_components
param_features__univ_select__k
param_linear__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.539103
0.026988
0.752042
0.829107
20
1
True
{u'features__pca__n_components': 20, u'linear_...
41
0.788843
0.832023
0.726700
0.836373
0.740440
0.818926
0.108656
0.008613
0.026669
0.007415
1
0.604635
0.027206
0.752042
0.829107
20
1
False
{u'features__pca__n_components': 20, u'linear_...
42
0.788843
0.832023
0.726700
0.836373
0.740440
0.818926
0.046825
0.005809
0.026669
0.007415
2
0.592394
0.021880
0.756330
0.841899
20
2
True
{u'features__pca__n_components': 20, u'linear_...
39
0.788653
0.832152
0.740763
0.853116
0.739450
0.840429
0.045160
0.004012
0.022906
0.008621
3
0.579496
0.022891
0.756330
0.841899
20
2
False
{u'features__pca__n_components': 20, u'linear_...
40
0.788653
0.832152
0.740763
0.853116
0.739450
0.840429
0.115812
0.000615
0.022906
0.008621
4
0.528474
0.021775
0.759498
0.846282
20
3
True
{u'features__pca__n_components': 20, u'linear_...
38
0.807484
0.840276
0.733925
0.854120
0.736901
0.844450
0.099467
0.002653
0.034018
0.005798
5
0.452069
0.021932
0.759498
0.846282
20
3
False
{u'features__pca__n_components': 20, u'linear_...
37
0.807484
0.840277
0.733925
0.854120
0.736901
0.844450
0.035102
0.002728
0.034018
0.005798
6
0.503984
0.020971
0.822455
0.883166
30
1
True
{u'features__pca__n_components': 30, u'linear_...
33
0.872791
0.873698
0.781452
0.897798
0.812926
0.878002
0.019615
0.000646
0.037904
0.010495
7
0.525240
0.024944
0.822455
0.883166
30
1
False
{u'features__pca__n_components': 30, u'linear_...
34
0.872791
0.873698
0.781452
0.897798
0.812926
0.878002
0.075887
0.002825
0.037904
0.010495
8
0.689936
0.039335
0.823143
0.891287
30
2
True
{u'features__pca__n_components': 30, u'linear_...
32
0.869171
0.875419
0.780098
0.917019
0.819981
0.881421
0.178154
0.017534
0.036444
0.018360
9
0.614220
0.021543
0.823143
0.891287
30
2
False
{u'features__pca__n_components': 30, u'linear_...
31
0.869171
0.875419
0.780098
0.917019
0.819981
0.881421
0.020998
0.001582
0.036444
0.018360
10
0.632997
0.040475
0.819006
0.894127
30
3
True
{u'features__pca__n_components': 30, u'linear_...
35
0.871962
0.876598
0.764386
0.923123
0.820466
0.882660
0.098059
0.013386
0.043944
0.020652
11
0.585624
0.029730
0.819006
0.894127
30
3
False
{u'features__pca__n_components': 30, u'linear_...
36
0.871962
0.876598
0.764384
0.923123
0.820466
0.882660
0.114507
0.007714
0.043945
0.020652
12
0.720528
0.023914
0.840391
0.914103
40
1
True
{u'features__pca__n_components': 40, u'linear_...
30
0.897578
0.902073
0.776721
0.934485
0.846652
0.905752
0.178621
0.001352
0.049554
0.014490
13
0.593216
0.022713
0.840393
0.914103
40
1
False
{u'features__pca__n_components': 40, u'linear_...
29
0.897579
0.902073
0.776731
0.934486
0.846647
0.905751
0.076715
0.001324
0.049549
0.014491
14
0.603689
0.024769
0.841395
0.914362
40
2
True
{u'features__pca__n_components': 40, u'linear_...
26
0.895196
0.902306
0.777508
0.934580
0.851274
0.906200
0.041078
0.003477
0.048565
0.014385
15
0.667583
0.026451
0.841396
0.914363
40
2
False
{u'features__pca__n_components': 40, u'linear_...
25
0.895196
0.902306
0.777508
0.934581
0.851277
0.906203
0.055578
0.001631
0.048566
0.014384
16
0.727962
0.026950
0.840735
0.914650
40
3
True
{u'features__pca__n_components': 40, u'linear_...
27
0.896954
0.902639
0.778296
0.934827
0.846738
0.906484
0.110818
0.004085
0.048643
0.014354
17
0.712662
0.029420
0.840735
0.914650
40
3
False
{u'features__pca__n_components': 40, u'linear_...
28
0.896954
0.902639
0.778303
0.934827
0.846731
0.906483
0.100488
0.000828
0.048639
0.014354
18
0.755150
0.027293
0.857615
0.923408
50
1
True
{u'features__pca__n_components': 50, u'linear_...
1
0.902955
0.917546
0.803933
0.939915
0.865783
0.912763
0.116676
0.004190
0.040848
0.011834
19
0.620018
0.027487
0.857613
0.923408
50
1
False
{u'features__pca__n_components': 50, u'linear_...
2
0.902956
0.917546
0.803935
0.939915
0.865772
0.912762
0.055128
0.004705
0.040847
0.011835
20
0.596315
0.026770
0.855246
0.924307
50
2
True
{u'features__pca__n_components': 50, u'linear_...
3
0.898507
0.918196
0.803933
0.939915
0.863129
0.914811
0.035642
0.003127
0.039021
0.011123
21
0.615047
0.026151
0.855245
0.924308
50
2
False
{u'features__pca__n_components': 50, u'linear_...
4
0.898507
0.918196
0.803932
0.939915
0.863127
0.914813
0.049764
0.004060
0.039022
0.011122
22
0.802325
0.032247
0.854666
0.924771
50
3
True
{u'features__pca__n_components': 50, u'linear_...
5
0.898369
0.918504
0.803945
0.939915
0.861514
0.915893
0.177967
0.006298
0.038863
0.010762
23
0.762000
0.025096
0.854659
0.924769
50
3
False
{u'features__pca__n_components': 50, u'linear_...
6
0.898370
0.918505
0.803940
0.939915
0.861498
0.915886
0.081498
0.001835
0.038865
0.010763
24
0.517741
0.024029
0.852391
0.927792
60
1
True
{u'features__pca__n_components': 60, u'linear_...
9
0.898319
0.922786
0.807300
0.942327
0.851374
0.918262
0.022135
0.000940
0.037177
0.010443
25
0.545551
0.024418
0.852549
0.927808
60
1
False
{u'features__pca__n_components': 60, u'linear_...
7
0.898389
0.922801
0.807266
0.942320
0.851812
0.918302
0.039676
0.000083
0.037216
0.010425
26
0.643192
0.033595
0.852315
0.928000
60
2
True
{u'features__pca__n_components': 60, u'linear_...
10
0.900963
0.923203
0.805780
0.942325
0.850015
0.918473
0.104336
0.002541
0.038904
0.010311
27
0.596503
0.039021
0.852439
0.927994
60
2
False
{u'features__pca__n_components': 60, u'linear_...
8
0.900974
0.923202
0.805955
0.942334
0.850201
0.918447
0.082178
0.010859
0.038836
0.010324
28
0.610975
0.037933
0.848354
0.928860
60
3
True
{u'features__pca__n_components': 60, u'linear_...
12
0.900954
0.923366
0.809849
0.942410
0.834054
0.920804
0.048030
0.011078
0.038553
0.009638
29
0.594453
0.026549
0.848561
0.928826
60
3
False
{u'features__pca__n_components': 60, u'linear_...
11
0.900868
0.923292
0.809795
0.942423
0.834818
0.920762
0.080564
0.001838
0.038439
0.009670
30
0.595763
0.031590
0.847591
0.935700
70
1
True
{u'features__pca__n_components': 70, u'linear_...
15
0.902088
0.930583
0.800391
0.944881
0.840082
0.931635
0.053271
0.002515
0.041868
0.006506
31
0.538564
0.025456
0.847196
0.935740
70
1
False
{u'features__pca__n_components': 70, u'linear_...
16
0.902066
0.930589
0.799344
0.944949
0.839965
0.931682
0.057103
0.000877
0.042259
0.006527
32
0.503076
0.033897
0.843672
0.936248
70
2
True
{u'features__pca__n_components': 70, u'linear_...
17
0.902182
0.931289
0.791164
0.945232
0.837444
0.932222
0.015748
0.005204
0.045550
0.006364
33
0.525525
0.028135
0.843447
0.936234
70
2
False
{u'features__pca__n_components': 70, u'linear_...
19
0.902191
0.931290
0.790455
0.945165
0.837469
0.932245
0.021072
0.002296
0.045826
0.006328
34
0.519668
0.030298
0.842425
0.936410
70
3
True
{u'features__pca__n_components': 70, u'linear_...
23
0.902298
0.931374
0.790882
0.945254
0.833863
0.932603
0.008360
0.004603
0.045900
0.006273
35
0.568633
0.032991
0.842254
0.936408
70
3
False
{u'features__pca__n_components': 70, u'linear_...
24
0.902278
0.931265
0.790605
0.945305
0.833646
0.932655
0.035905
0.009279
0.046009
0.006316
36
0.751007
0.029718
0.847955
0.942405
80
1
True
{u'features__pca__n_components': 80, u'linear_...
13
0.908429
0.938013
0.789211
0.949734
0.845989
0.939469
0.032160
0.004364
0.048706
0.005216
37
0.809686
0.032912
0.847792
0.942389
80
1
False
{u'features__pca__n_components': 80, u'linear_...
14
0.908252
0.938088
0.789128
0.949631
0.845763
0.939449
0.120121
0.005055
0.048669
0.005151
38
0.668799
0.027097
0.843628
0.942660
80
2
True
{u'features__pca__n_components': 80, u'linear_...
18
0.908377
0.938062
0.777541
0.950057
0.844713
0.939860
0.064891
0.000548
0.053436
0.005282
39
0.629298
0.033938
0.843016
0.942668
80
2
False
{u'features__pca__n_components': 80, u'linear_...
21
0.908295
0.938076
0.776339
0.950133
0.844161
0.939795
0.018010
0.008392
0.053894
0.005325
40
0.892139
0.037772
0.842734
0.942825
80
3
True
{u'features__pca__n_components': 80, u'linear_...
22
0.907911
0.938059
0.773845
0.950303
0.846194
0.940112
0.128831
0.012391
0.054805
0.005354
41
1.227470
0.057133
0.843377
0.942851
80
3
False
{u'features__pca__n_components': 80, u'linear_...
20
0.907878
0.938047
0.775279
0.950361
0.846725
0.940144
0.174541
0.030172
0.054202
0.005379
---------- P
Cross_val_score: [ 0.16251155 -0.03036356 0.02650919]
Explained variance score: 0.0944525895806
Mean absolute error: 0.529338262902
Mean squared error: 1.10076421175
R2 score: 0.0938817492596
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_features__pca__n_components
param_features__univ_select__k
param_linear__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.455604
0.022077
-0.055244
0.123904
20
1
True
{u'features__pca__n_components': 20, u'linear_...
7
0.016400
0.161322
0.038001
0.089139
-0.220411
0.121250
0.023503
0.002379
0.117011
0.029528
1
0.431819
0.022018
-0.055244
0.123904
20
1
False
{u'features__pca__n_components': 20, u'linear_...
8
0.016400
0.161322
0.038001
0.089139
-0.220411
0.121250
0.027850
0.001538
0.117011
0.029528
2
0.409718
0.023444
-0.101066
0.136045
20
2
True
{u'features__pca__n_components': 20, u'linear_...
18
0.015288
0.161533
0.047993
0.090671
-0.366929
0.155931
0.006436
0.001358
0.188286
0.032166
3
0.434585
0.023096
-0.101066
0.136045
20
2
False
{u'features__pca__n_components': 20, u'linear_...
17
0.015288
0.161533
0.047993
0.090671
-0.366929
0.155931
0.017884
0.001982
0.188286
0.032166
4
0.428819
0.022489
-0.104515
0.137724
20
3
True
{u'features__pca__n_components': 20, u'linear_...
24
0.015676
0.162036
0.050256
0.095059
-0.379943
0.156077
0.004762
0.001125
0.195080
0.030267
5
0.470618
0.021617
-0.104515
0.137724
20
3
False
{u'features__pca__n_components': 20, u'linear_...
23
0.015676
0.162036
0.050256
0.095059
-0.379943
0.156077
0.051558
0.001277
0.195080
0.030267
6
0.446046
0.025402
-0.094627
0.151383
30
1
True
{u'features__pca__n_components': 30, u'linear_...
13
0.019094
0.188323
0.029969
0.104768
-0.333386
0.161057
0.015575
0.003476
0.168723
0.034790
7
0.538472
0.028034
-0.094627
0.151383
30
1
False
{u'features__pca__n_components': 30, u'linear_...
14
0.019094
0.188323
0.029969
0.104768
-0.333386
0.161057
0.066348
0.007351
0.168723
0.034790
8
0.454721
0.030642
-0.103313
0.161089
30
2
True
{u'features__pca__n_components': 30, u'linear_...
22
0.012466
0.194450
0.056026
0.108772
-0.378879
0.180045
0.028394
0.012229
0.195477
0.037458
9
0.575994
0.025420
-0.103313
0.161089
30
2
False
{u'features__pca__n_components': 30, u'linear_...
21
0.012466
0.194449
0.056026
0.108772
-0.378879
0.180045
0.070110
0.004568
0.195477
0.037458
10
0.496535
0.023393
-0.119457
0.165427
30
3
True
{u'features__pca__n_components': 30, u'linear_...
27
0.021777
0.200712
0.053387
0.113771
-0.434082
0.181797
0.071138
0.001690
0.222633
0.037334
11
0.409199
0.021490
-0.119457
0.165427
30
3
False
{u'features__pca__n_components': 30, u'linear_...
28
0.021777
0.200712
0.053387
0.113771
-0.434082
0.181797
0.010212
0.000284
0.222633
0.037334
12
0.483537
0.022777
-0.103206
0.185210
40
1
True
{u'features__pca__n_components': 40, u'linear_...
19
0.024184
0.206230
0.046647
0.146026
-0.380943
0.203374
0.008139
0.000324
0.196414
0.027732
13
0.500499
0.022700
-0.103207
0.185208
40
1
False
{u'features__pca__n_components': 40, u'linear_...
20
0.024181
0.206230
0.046647
0.146021
-0.380943
0.203373
0.016988
0.000126
0.196413
0.027734
14
0.499758
0.026124
-0.138904
0.200041
40
2
True
{u'features__pca__n_components': 40, u'linear_...
30
0.021176
0.219848
0.011761
0.155941
-0.450269
0.224333
0.016772
0.001525
0.219989
0.031237
15
0.548157
0.023411
-0.138939
0.200043
40
2
False
{u'features__pca__n_components': 40, u'linear_...
31
0.021178
0.219848
0.011754
0.155939
-0.450370
0.224341
0.077087
0.000306
0.220035
0.031240
16
0.587155
0.025776
-0.150940
0.204666
40
3
True
{u'features__pca__n_components': 40, u'linear_...
38
0.020684
0.224117
-0.002015
0.165424
-0.472153
0.224456
0.055801
0.002799
0.227102
0.027749
17
0.509973
0.023603
-0.150927
0.204659
40
3
False
{u'features__pca__n_components': 40, u'linear_...
37
0.020691
0.224095
-0.001990
0.165427
-0.472147
0.224456
0.025480
0.000875
0.227106
0.027742
18
0.519296
0.026681
-0.114216
0.255872
50
1
True
{u'features__pca__n_components': 50, u'linear_...
25
-0.001931
0.301734
0.053438
0.202325
-0.394591
0.263558
0.015180
0.001361
0.199350
0.040946
19
0.581832
0.026332
-0.114236
0.255879
50
1
False
{u'features__pca__n_components': 50, u'linear_...
26
-0.001964
0.301748
0.053405
0.202329
-0.394585
0.263559
0.080148
0.001510
0.199331
0.040949
20
0.522491
0.027206
-0.142347
0.259874
50
2
True
{u'features__pca__n_components': 50, u'linear_...
35
-0.000108
0.301971
0.036789
0.204701
-0.464273
0.272950
0.028234
0.004139
0.227914
0.040773
21
0.508995
0.024721
-0.142352
0.259872
50
2
False
{u'features__pca__n_components': 50, u'linear_...
36
-0.000099
0.301962
0.036758
0.204706
-0.464267
0.272948
0.015752
0.000466
0.227906
0.040767
22
0.507940
0.025197
-0.141815
0.263464
50
3
True
{u'features__pca__n_components': 50, u'linear_...
32
-0.012600
0.307997
0.029104
0.209031
-0.442451
0.273364
0.023790
0.000982
0.213057
0.041005
23
0.502831
0.024325
-0.141831
0.263467
50
3
False
{u'features__pca__n_components': 50, u'linear_...
33
-0.012597
0.307995
0.029048
0.209043
-0.442443
0.273364
0.006700
0.000503
0.213040
0.040999
24
0.500483
0.025116
-0.097808
0.308089
60
1
True
{u'features__pca__n_components': 60, u'linear_...
16
-0.025428
0.376003
0.061710
0.248719
-0.329987
0.299545
0.072069
0.000359
0.167832
0.052313
25
0.522354
0.028964
-0.097461
0.308443
60
1
False
{u'features__pca__n_components': 60, u'linear_...
15
-0.025999
0.377080
0.063012
0.248584
-0.329673
0.299666
0.058284
0.003584
0.168019
0.052824
26
0.560629
0.028651
-0.079406
0.315215
60
2
True
{u'features__pca__n_components': 60, u'linear_...
10
-0.016677
0.380495
0.066754
0.250370
-0.288539
0.314779
0.067468
0.004244
0.151614
0.053124
27
0.471231
0.028366
-0.078707
0.315680
60
2
False
{u'features__pca__n_components': 60, u'linear_...
9
-0.013260
0.380746
0.066552
0.250947
-0.289668
0.315347
0.013181
0.003345
0.152550
0.052991
28
0.461863
0.026099
-0.083381
0.316342
60
3
True
{u'features__pca__n_components': 60, u'linear_...
12
-0.023034
0.383191
0.064202
0.250843
-0.291544
0.314990
0.006391
0.000198
0.151305
0.054039
29
0.478799
0.026645
-0.083030
0.316519
60
3
False
{u'features__pca__n_components': 60, u'linear_...
11
-0.022826
0.382873
0.064127
0.250950
-0.290625
0.315733
0.025753
0.000968
0.150887
0.053860
30
0.487834
0.027268
-0.040256
0.369828
70
1
True
{u'features__pca__n_components': 70, u'linear_...
4
0.014722
0.444990
0.168962
0.304888
-0.304666
0.359606
0.017281
0.000450
0.197119
0.057651
31
0.528503
0.028850
-0.041355
0.369900
70
1
False
{u'features__pca__n_components': 70, u'linear_...
5
0.015248
0.445101
0.168856
0.305054
-0.308389
0.359546
0.070474
0.000831
0.198796
0.057641
32
0.731406
0.031201
-0.036778
0.371842
70
2
True
{u'features__pca__n_components': 70, u'linear_...
2
0.013016
0.444949
0.172128
0.310500
-0.295670
0.360075
0.121273
0.002563
0.194088
0.055516
33
0.615494
0.031165
-0.035555
0.371871
70
2
False
{u'features__pca__n_components': 70, u'linear_...
1
0.013609
0.444780
0.172705
0.310547
-0.293168
0.360285
0.038809
0.005538
0.193234
0.055410
34
0.617304
0.032364
-0.039605
0.372880
70
3
True
{u'features__pca__n_components': 70, u'linear_...
3
0.017417
0.447266
0.166148
0.311577
-0.302600
0.359798
0.134283
0.001784
0.195462
0.056162
35
0.585116
0.033060
-0.042037
0.372892
70
3
False
{u'features__pca__n_components': 70, u'linear_...
6
0.016960
0.446515
0.166655
0.311626
-0.309954
0.360534
0.059948
0.003866
0.198891
0.055757
36
0.594623
0.035681
-0.138568
0.418606
80
1
True
{u'features__pca__n_components': 80, u'linear_...
29
0.027985
0.472555
0.025271
0.377655
-0.469605
0.405606
0.049609
0.005157
0.233855
0.039818
37
0.736642
0.042960
-0.142233
0.418861
80
1
False
{u'features__pca__n_components': 80, u'linear_...
34
0.028364
0.472456
0.023994
0.377719
-0.479719
0.406408
0.058834
0.008016
0.238414
0.039666
38
0.784370
0.039634
-0.167753
0.420753
80
2
True
{u'features__pca__n_components': 80, u'linear_...
39
0.032776
0.475370
0.031741
0.378888
-0.568552
0.407999
0.132305
0.005985
0.283134
0.040408
39
0.644676
0.035321
-0.171190
0.421306
80
2
False
{u'features__pca__n_components': 80, u'linear_...
42
0.032741
0.475327
0.031153
0.379371
-0.578256
0.409220
0.131172
0.007964
0.287561
0.040095
40
0.679356
0.028644
-0.169459
0.423026
80
3
True
{u'features__pca__n_components': 80, u'linear_...
40
0.037307
0.475454
0.032448
0.380544
-0.578932
0.413080
0.016961
0.000906
0.289268
0.039380
41
0.550567
0.029679
-0.170369
0.423140
80
3
False
{u'features__pca__n_components': 80, u'linear_...
41
0.038140
0.475394
0.033708
0.380349
-0.583763
0.413677
0.012676
0.001139
0.292036
0.039375
---------- pH
Cross_val_score: [ 0.73217844 0.75485549 0.76651478]
Explained variance score: 0.797277374776
Mean absolute error: 0.303479467731
Mean squared error: 0.178771031117
R2 score: 0.797003978142
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_features__pca__n_components
param_features__univ_select__k
param_linear__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.526293
0.024599
0.684262
0.721143
20
1
True
{u'features__pca__n_components': 20, u'linear_...
37
0.646707
0.734460
0.712128
0.710383
0.694097
0.718587
0.057334
0.002543
0.027606
0.009994
1
0.539544
0.023279
0.684262
0.721143
20
1
False
{u'features__pca__n_components': 20, u'linear_...
38
0.646707
0.734460
0.712128
0.710383
0.694097
0.718587
0.094308
0.002616
0.027606
0.009994
2
0.460707
0.028236
0.683038
0.722121
20
2
True
{u'features__pca__n_components': 20, u'linear_...
41
0.644830
0.734736
0.710619
0.710800
0.693813
0.720827
0.010052
0.005969
0.027925
0.009815
3
0.480729
0.022917
0.683038
0.722121
20
2
False
{u'features__pca__n_components': 20, u'linear_...
42
0.644830
0.734736
0.710619
0.710800
0.693813
0.720827
0.043582
0.001709
0.027925
0.009815
4
0.431968
0.021229
0.683205
0.727116
20
3
True
{u'features__pca__n_components': 20, u'linear_...
39
0.626677
0.738249
0.712982
0.713573
0.710174
0.729525
0.011053
0.000861
0.040065
0.010217
5
0.507374
0.022569
0.683205
0.727116
20
3
False
{u'features__pca__n_components': 20, u'linear_...
40
0.626677
0.738249
0.712982
0.713573
0.710174
0.729525
0.104224
0.001374
0.040065
0.010217
6
0.472568
0.022878
0.734785
0.783162
30
1
True
{u'features__pca__n_components': 30, u'linear_...
31
0.714870
0.788666
0.745455
0.787138
0.744108
0.773681
0.033538
0.001100
0.014120
0.006733
7
0.469445
0.022558
0.734785
0.783161
30
1
False
{u'features__pca__n_components': 30, u'linear_...
32
0.714870
0.788665
0.745455
0.787138
0.744108
0.773681
0.020284
0.000640
0.014120
0.006732
8
0.468389
0.022907
0.732791
0.785814
30
2
True
{u'features__pca__n_components': 30, u'linear_...
36
0.709503
0.791493
0.747498
0.787346
0.741464
0.778603
0.026280
0.001372
0.016682
0.005373
9
0.457075
0.022274
0.732791
0.785814
30
2
False
{u'features__pca__n_components': 30, u'linear_...
35
0.709503
0.791494
0.747498
0.787346
0.741464
0.778603
0.018070
0.000878
0.016682
0.005373
10
0.469478
0.026212
0.733746
0.788373
30
3
True
{u'features__pca__n_components': 30, u'linear_...
33
0.701822
0.792944
0.747429
0.787365
0.752109
0.784810
0.023823
0.002117
0.022697
0.003396
11
0.480810
0.024434
0.733746
0.788373
30
3
False
{u'features__pca__n_components': 30, u'linear_...
34
0.701822
0.792944
0.747429
0.787365
0.752109
0.784810
0.030714
0.000672
0.022697
0.003396
12
0.552060
0.023433
0.757842
0.809165
40
1
True
{u'features__pca__n_components': 40, u'linear_...
22
0.744996
0.815522
0.757372
0.806712
0.771208
0.805260
0.017807
0.000347
0.010710
0.004534
13
0.531845
0.027364
0.757842
0.809167
40
1
False
{u'features__pca__n_components': 40, u'linear_...
21
0.745000
0.815522
0.757362
0.806709
0.771214
0.805270
0.025327
0.001415
0.010711
0.004532
14
0.561854
0.025228
0.757123
0.811713
40
2
True
{u'features__pca__n_components': 40, u'linear_...
24
0.741505
0.818124
0.759653
0.807928
0.770270
0.809086
0.029783
0.001486
0.011882
0.004558
15
0.760030
0.030268
0.757124
0.811720
40
2
False
{u'features__pca__n_components': 40, u'linear_...
23
0.741505
0.818125
0.759650
0.807925
0.770278
0.809110
0.154612
0.003096
0.011885
0.004555
16
0.734238
0.028781
0.758357
0.813087
40
3
True
{u'features__pca__n_components': 40, u'linear_...
20
0.737968
0.819561
0.759587
0.807944
0.777594
0.811756
0.137146
0.003383
0.016206
0.004835
17
0.589614
0.026544
0.758357
0.813086
40
3
False
{u'features__pca__n_components': 40, u'linear_...
19
0.737966
0.819561
0.759589
0.807944
0.777594
0.811754
0.036824
0.000862
0.016207
0.004835
18
0.584386
0.027880
0.752692
0.824531
50
1
True
{u'features__pca__n_components': 50, u'linear_...
30
0.751157
0.823246
0.749385
0.823805
0.757540
0.826541
0.006362
0.004994
0.003500
0.001440
19
0.555195
0.025975
0.752695
0.824530
50
1
False
{u'features__pca__n_components': 50, u'linear_...
29
0.751158
0.823244
0.749392
0.823805
0.757540
0.826541
0.013893
0.001989
0.003498
0.001440
20
0.550402
0.024755
0.753146
0.825391
50
2
True
{u'features__pca__n_components': 50, u'linear_...
27
0.750333
0.825188
0.752192
0.824422
0.756923
0.826564
0.004326
0.000569
0.002774
0.000886
21
0.555799
0.028959
0.753141
0.825390
50
2
False
{u'features__pca__n_components': 50, u'linear_...
28
0.750334
0.825189
0.752182
0.824424
0.756919
0.826557
0.029637
0.004286
0.002773
0.000882
22
0.558297
0.029979
0.756357
0.826397
50
3
True
{u'features__pca__n_components': 50, u'linear_...
25
0.747617
0.825828
0.759691
0.825387
0.761796
0.827975
0.039008
0.002506
0.006251
0.001131
23
0.571644
0.026993
0.756328
0.826395
50
3
False
{u'features__pca__n_components': 50, u'linear_...
26
0.747612
0.825829
0.759701
0.825391
0.761704
0.827966
0.017453
0.000726
0.006229
0.001125
24
0.500115
0.032632
0.764113
0.840495
60
1
True
{u'features__pca__n_components': 60, u'linear_...
16
0.748134
0.849445
0.786408
0.834737
0.757857
0.837302
0.027055
0.008757
0.016243
0.006415
25
0.469566
0.030240
0.764185
0.840466
60
1
False
{u'features__pca__n_components': 60, u'linear_...
15
0.748008
0.849312
0.785953
0.834811
0.758656
0.837274
0.008249
0.003530
0.015981
0.006336
26
0.478644
0.030291
0.762275
0.841284
60
2
True
{u'features__pca__n_components': 60, u'linear_...
18
0.743717
0.850517
0.785115
0.836113
0.758066
0.837223
0.008023
0.004121
0.017166
0.006544
27
0.491381
0.027537
0.762310
0.841385
60
2
False
{u'features__pca__n_components': 60, u'linear_...
17
0.743447
0.850698
0.785145
0.836126
0.758411
0.837330
0.032186
0.002842
0.017250
0.006604
28
0.499675
0.027345
0.767069
0.842933
60
3
True
{u'features__pca__n_components': 60, u'linear_...
14
0.747630
0.852072
0.787389
0.836425
0.766263
0.840302
0.038623
0.002254
0.016247
0.006653
29
0.472722
0.026165
0.767206
0.842873
60
3
False
{u'features__pca__n_components': 60, u'linear_...
13
0.748120
0.851901
0.787266
0.836389
0.766305
0.840330
0.010835
0.000900
0.015999
0.006584
30
0.498887
0.027674
0.767913
0.848580
70
1
True
{u'features__pca__n_components': 70, u'linear_...
12
0.746791
0.859974
0.793587
0.842296
0.763443
0.843470
0.015795
0.001070
0.019370
0.008071
31
0.497776
0.028587
0.768133
0.848725
70
1
False
{u'features__pca__n_components': 70, u'linear_...
11
0.746978
0.860422
0.794124
0.842249
0.763379
0.843505
0.018908
0.000663
0.019544
0.008287
32
0.517131
0.027752
0.768725
0.849192
70
2
True
{u'features__pca__n_components': 70, u'linear_...
9
0.747902
0.861467
0.795190
0.842706
0.763162
0.843402
0.028962
0.000752
0.019707
0.008685
33
0.500153
0.026732
0.768449
0.848988
70
2
False
{u'features__pca__n_components': 70, u'linear_...
10
0.747810
0.861296
0.793576
0.842277
0.764043
0.843391
0.022354
0.000203
0.018947
0.008715
34
0.497653
0.027407
0.772762
0.851663
70
3
True
{u'features__pca__n_components': 70, u'linear_...
8
0.753456
0.863179
0.793911
0.842341
0.770995
0.849469
0.024586
0.000413
0.016568
0.008647
35
0.499726
0.034937
0.773148
0.851731
70
3
False
{u'features__pca__n_components': 70, u'linear_...
7
0.753485
0.863238
0.793856
0.842223
0.772179
0.849732
0.018561
0.004668
0.016501
0.008695
36
0.576356
0.036384
0.782546
0.866248
80
1
True
{u'features__pca__n_components': 80, u'linear_...
3
0.760214
0.868707
0.810224
0.858451
0.777285
0.871586
0.008223
0.008972
0.020759
0.005637
37
0.579505
0.032532
0.782459
0.866298
80
1
False
{u'features__pca__n_components': 80, u'linear_...
4
0.760060
0.868649
0.810671
0.858520
0.776731
0.871725
0.011013
0.001873
0.021061
0.005641
38
0.583922
0.028821
0.777981
0.867709
80
2
True
{u'features__pca__n_components': 80, u'linear_...
6
0.760239
0.868729
0.798649
0.861811
0.775123
0.872588
0.004630
0.000205
0.015815
0.004458
39
0.626013
0.029186
0.778234
0.867617
80
2
False
{u'features__pca__n_components': 80, u'linear_...
5
0.760418
0.868843
0.799191
0.861417
0.775163
0.872590
0.052565
0.001325
0.015982
0.004643
40
0.581026
0.033601
0.784190
0.869783
80
3
True
{u'features__pca__n_components': 80, u'linear_...
1
0.770955
0.872084
0.799622
0.862490
0.782044
0.874774
0.005301
0.003040
0.011805
0.005272
41
0.595790
0.035433
0.784133
0.869792
80
3
False
{u'features__pca__n_components': 80, u'linear_...
2
0.771050
0.872389
0.798195
0.862457
0.783204
0.874529
0.014395
0.005520
0.011105
0.005259
---------- SOC
Cross_val_score: [ 0.7908808 0.89644486 0.85925249]
Explained variance score: 0.898572054172
Mean absolute error: 0.256826504096
Mean squared error: 0.154457127791
R2 score: 0.897546044087
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_features__pca__n_components
param_features__univ_select__k
param_linear__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.521211
0.022289
0.801045
0.813881
20
1
True
{u'features__pca__n_components': 20, u'linear_...
41
0.830676
0.798594
0.786840
0.820089
0.785505
0.822959
0.076854
0.001333
0.021000
0.010873
1
0.420244
0.021205
0.801045
0.813881
20
1
False
{u'features__pca__n_components': 20, u'linear_...
42
0.830676
0.798594
0.786840
0.820089
0.785505
0.822959
0.002498
0.001246
0.021000
0.010873
2
0.426807
0.022061
0.804188
0.815591
20
2
True
{u'features__pca__n_components': 20, u'linear_...
37
0.833388
0.799074
0.786874
0.820094
0.792189
0.827607
0.025540
0.001396
0.020800
0.012076
3
0.421932
0.020190
0.804188
0.815591
20
2
False
{u'features__pca__n_components': 20, u'linear_...
38
0.833388
0.799074
0.786874
0.820094
0.792189
0.827607
0.026729
0.000369
0.020800
0.012076
4
0.405382
0.020134
0.802430
0.821659
20
3
True
{u'features__pca__n_components': 20, u'linear_...
39
0.836657
0.803088
0.793067
0.825275
0.777434
0.836614
0.001331
0.000159
0.025073
0.013924
5
0.414270
0.020052
0.802430
0.821659
20
3
False
{u'features__pca__n_components': 20, u'linear_...
40
0.836657
0.803088
0.793067
0.825275
0.777434
0.836614
0.004303
0.000170
0.025073
0.013924
6
0.435031
0.021687
0.823852
0.848407
30
1
True
{u'features__pca__n_components': 30, u'linear_...
35
0.834534
0.843930
0.824421
0.848622
0.812559
0.852669
0.005691
0.000201
0.008983
0.003571
7
0.449671
0.022753
0.823852
0.848407
30
1
False
{u'features__pca__n_components': 30, u'linear_...
36
0.834534
0.843930
0.824421
0.848622
0.812559
0.852669
0.028329
0.000643
0.008983
0.003571
8
0.441006
0.021449
0.830513
0.853216
30
2
True
{u'features__pca__n_components': 30, u'linear_...
31
0.851727
0.854542
0.825961
0.850723
0.813769
0.854381
0.014210
0.000054
0.015831
0.001764
9
0.419900
0.021510
0.830513
0.853216
30
2
False
{u'features__pca__n_components': 30, u'linear_...
32
0.851727
0.854542
0.825961
0.850723
0.813769
0.854381
0.008712
0.000111
0.015831
0.001764
10
0.434198
0.022085
0.828020
0.858038
30
3
True
{u'features__pca__n_components': 30, u'linear_...
33
0.856062
0.855606
0.828349
0.851768
0.799540
0.866739
0.013815
0.000409
0.023083
0.006349
11
0.440835
0.022299
0.828020
0.858038
30
3
False
{u'features__pca__n_components': 30, u'linear_...
34
0.856062
0.855606
0.828349
0.851768
0.799540
0.866739
0.022987
0.000860
0.023083
0.006349
12
0.508459
0.022723
0.832485
0.862826
40
1
True
{u'features__pca__n_components': 40, u'linear_...
30
0.836628
0.858008
0.838908
0.863541
0.821903
0.866929
0.004217
0.000214
0.007533
0.003677
13
0.518365
0.027712
0.832485
0.862826
40
1
False
{u'features__pca__n_components': 40, u'linear_...
29
0.836629
0.858008
0.838911
0.863539
0.821900
0.866931
0.015242
0.004880
0.007536
0.003678
14
0.562056
0.023254
0.835322
0.865408
40
2
True
{u'features__pca__n_components': 40, u'linear_...
26
0.847746
0.863771
0.837340
0.864498
0.820832
0.867957
0.047969
0.000543
0.011083
0.001826
15
0.625781
0.027032
0.835322
0.865407
40
2
False
{u'features__pca__n_components': 40, u'linear_...
25
0.847748
0.863769
0.837339
0.864496
0.820832
0.867957
0.094348
0.005727
0.011084
0.001827
16
0.578219
0.026466
0.833659
0.870949
40
3
True
{u'features__pca__n_components': 40, u'linear_...
27
0.850240
0.863963
0.847252
0.869562
0.803420
0.879323
0.048182
0.004001
0.021396
0.006347
17
0.625638
0.025436
0.833659
0.870949
40
3
False
{u'features__pca__n_components': 40, u'linear_...
28
0.850241
0.863962
0.847251
0.869561
0.803420
0.879324
0.112700
0.002867
0.021396
0.006348
18
0.577053
0.025683
0.844963
0.888631
50
1
True
{u'features__pca__n_components': 50, u'linear_...
21
0.852872
0.887607
0.855094
0.886187
0.826892
0.892100
0.042139
0.001256
0.012798
0.002520
19
0.535779
0.023953
0.844960
0.888632
50
1
False
{u'features__pca__n_components': 50, u'linear_...
22
0.852869
0.887606
0.855089
0.886191
0.826893
0.892099
0.009551
0.000144
0.012796
0.002519
20
0.534975
0.025589
0.845754
0.890552
50
2
True
{u'features__pca__n_components': 50, u'linear_...
19
0.853523
0.891510
0.853378
0.886450
0.830330
0.893695
0.010421
0.002062
0.010896
0.003035
21
0.532923
0.024654
0.845751
0.890552
50
2
False
{u'features__pca__n_components': 50, u'linear_...
20
0.853520
0.891510
0.853377
0.886451
0.830324
0.893696
0.004818
0.000454
0.010898
0.003035
22
0.524229
0.026910
0.840514
0.890984
50
3
True
{u'features__pca__n_components': 50, u'linear_...
23
0.852801
0.891540
0.852526
0.886515
0.816168
0.894898
0.017232
0.002409
0.017199
0.003445
23
0.544089
0.024621
0.840513
0.890984
50
3
False
{u'features__pca__n_components': 50, u'linear_...
24
0.852799
0.891539
0.852530
0.886514
0.816163
0.894900
0.023225
0.000255
0.017202
0.003446
24
0.466422
0.025479
0.867095
0.906795
60
1
True
{u'features__pca__n_components': 60, u'linear_...
13
0.882302
0.900745
0.872251
0.909635
0.846673
0.910006
0.016307
0.000372
0.014999
0.004281
25
0.472042
0.025904
0.867081
0.906901
60
1
False
{u'features__pca__n_components': 60, u'linear_...
14
0.882335
0.900803
0.872579
0.909857
0.846269
0.910043
0.014807
0.000307
0.015232
0.004313
26
0.487591
0.026340
0.864236
0.908568
60
2
True
{u'features__pca__n_components': 60, u'linear_...
16
0.876137
0.903019
0.867313
0.911470
0.849213
0.911215
0.026644
0.001006
0.011208
0.003925
27
0.482391
0.026118
0.864328
0.908516
60
2
False
{u'features__pca__n_components': 60, u'linear_...
15
0.876418
0.903122
0.867223
0.911301
0.849298
0.911127
0.013891
0.000604
0.011262
0.003815
28
0.481183
0.025662
0.860012
0.908890
60
3
True
{u'features__pca__n_components': 60, u'linear_...
17
0.877592
0.903176
0.867114
0.911607
0.835261
0.911887
0.017101
0.000736
0.018000
0.004042
29
0.490647
0.027217
0.859949
0.908813
60
3
False
{u'features__pca__n_components': 60, u'linear_...
18
0.878110
0.903235
0.866875
0.911458
0.834793
0.911748
0.030095
0.001483
0.018354
0.003946
30
0.498701
0.033660
0.870870
0.920428
70
1
True
{u'features__pca__n_components': 70, u'linear_...
7
0.889973
0.914974
0.867632
0.927366
0.854931
0.918943
0.017480
0.005364
0.014492
0.005167
31
0.490923
0.027290
0.870854
0.920414
70
1
False
{u'features__pca__n_components': 70, u'linear_...
8
0.890075
0.914972
0.867599
0.927372
0.854814
0.918897
0.010878
0.000271
0.014582
0.005174
32
0.515409
0.030708
0.869550
0.921816
70
2
True
{u'features__pca__n_components': 70, u'linear_...
10
0.885970
0.918807
0.867564
0.927390
0.855052
0.919252
0.026539
0.004561
0.012704
0.003945
33
0.508293
0.027374
0.869700
0.921814
70
2
False
{u'features__pca__n_components': 70, u'linear_...
9
0.885992
0.918820
0.867624
0.927402
0.855422
0.919220
0.020530
0.000390
0.012570
0.003955
34
0.496810
0.032608
0.868919
0.921985
70
3
True
{u'features__pca__n_components': 70, u'linear_...
12
0.888458
0.919007
0.866971
0.927667
0.851251
0.919281
0.011955
0.005765
0.015257
0.004020
35
0.494839
0.027380
0.869001
0.921991
70
3
False
{u'features__pca__n_components': 70, u'linear_...
11
0.888343
0.918969
0.867094
0.927655
0.851491
0.919347
0.014148
0.000428
0.015110
0.004008
36
0.614999
0.030461
0.880851
0.930795
80
1
True
{u'features__pca__n_components': 80, u'linear_...
1
0.897448
0.924267
0.875162
0.939497
0.869878
0.928619
0.025939
0.002429
0.011955
0.006405
37
0.609707
0.033718
0.880782
0.930762
80
1
False
{u'features__pca__n_components': 80, u'linear_...
2
0.897211
0.924242
0.875074
0.939547
0.869998
0.928496
0.035606
0.005309
0.011822
0.006451
38
0.603181
0.032707
0.879803
0.931840
80
2
True
{u'features__pca__n_components': 80, u'linear_...
3
0.894539
0.927212
0.875478
0.939756
0.869336
0.928550
0.029106
0.002902
0.010736
0.005625
39
0.637093
0.031580
0.879680
0.931817
80
2
False
{u'features__pca__n_components': 80, u'linear_...
4
0.894429
0.927308
0.875131
0.939506
0.869422
0.928638
0.098898
0.003854
0.010706
0.005464
40
0.663934
0.030527
0.877328
0.932094
80
3
True
{u'features__pca__n_components': 80, u'linear_...
5
0.896032
0.927376
0.874816
0.940056
0.861063
0.928849
0.057695
0.001657
0.014391
0.005662
41
0.636079
0.029724
0.877072
0.931955
80
3
False
{u'features__pca__n_components': 80, u'linear_...
6
0.895516
0.927308
0.874663
0.939894
0.860967
0.928664
0.093412
0.000800
0.014211
0.005641
---------- Sand
Cross_val_score: [ 0.8660768 0.8284549 0.86634199]
Explained variance score: 0.888987592216
Mean absolute error: 0.236546806376
Mean squared error: 0.109696097705
R2 score: 0.888716204079
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_features__pca__n_components
param_features__univ_select__k
param_linear__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.557018
0.030715
0.723456
0.754630
20
1
True
{u'features__pca__n_components': 20, u'linear_...
41
0.708228
0.763865
0.734734
0.751713
0.727466
0.748313
0.050004
0.010695
0.011189
0.006676
1
0.461978
0.019401
0.723456
0.754630
20
1
False
{u'features__pca__n_components': 20, u'linear_...
42
0.708228
0.763865
0.734734
0.751713
0.727466
0.748313
0.021658
0.001033
0.011189
0.006676
2
0.464271
0.022543
0.730042
0.760747
20
2
True
{u'features__pca__n_components': 20, u'linear_...
40
0.705771
0.765478
0.736990
0.758313
0.747459
0.758450
0.023035
0.003743
0.017718
0.003346
3
0.458089
0.021778
0.730042
0.760747
20
2
False
{u'features__pca__n_components': 20, u'linear_...
39
0.705771
0.765478
0.736990
0.758313
0.747459
0.758450
0.013902
0.003098
0.017718
0.003346
4
0.454933
0.019413
0.755829
0.774223
20
3
True
{u'features__pca__n_components': 20, u'linear_...
37
0.708652
0.766030
0.757428
0.773334
0.801590
0.783306
0.004902
0.000571
0.037971
0.007081
5
0.440131
0.019050
0.755829
0.774223
20
3
False
{u'features__pca__n_components': 20, u'linear_...
38
0.708652
0.766030
0.757428
0.773334
0.801590
0.783306
0.011106
0.000357
0.037971
0.007081
6
0.460359
0.021714
0.772673
0.810179
30
1
True
{u'features__pca__n_components': 30, u'linear_...
35
0.768543
0.806694
0.747281
0.821013
0.802212
0.802830
0.040030
0.002761
0.022601
0.007821
7
0.456100
0.019853
0.772673
0.810179
30
1
False
{u'features__pca__n_components': 30, u'linear_...
36
0.768543
0.806694
0.747281
0.821013
0.802212
0.802831
0.019675
0.000164
0.022601
0.007821
8
0.487452
0.020546
0.777828
0.814035
30
2
True
{u'features__pca__n_components': 30, u'linear_...
34
0.767343
0.809825
0.750785
0.821864
0.815396
0.810415
0.044592
0.000725
0.027387
0.005542
9
0.501726
0.030382
0.777828
0.814035
30
2
False
{u'features__pca__n_components': 30, u'linear_...
33
0.767343
0.809825
0.750785
0.821864
0.815396
0.810415
0.015212
0.013528
0.027387
0.005542
10
0.495870
0.022594
0.790180
0.819740
30
3
True
{u'features__pca__n_components': 30, u'linear_...
31
0.772398
0.811724
0.762329
0.829222
0.835882
0.818275
0.069018
0.003258
0.032546
0.007218
11
0.448914
0.021454
0.790180
0.819740
30
3
False
{u'features__pca__n_components': 30, u'linear_...
32
0.772398
0.811724
0.762329
0.829222
0.835882
0.818275
0.013376
0.002067
0.032546
0.007218
12
0.515744
0.020720
0.805838
0.856219
40
1
True
{u'features__pca__n_components': 40, u'linear_...
30
0.796512
0.864833
0.784678
0.860406
0.836361
0.843419
0.003095
0.000124
0.022097
0.009230
13
0.521306
0.023493
0.805884
0.856217
40
1
False
{u'features__pca__n_components': 40, u'linear_...
29
0.796512
0.864833
0.784679
0.860406
0.836497
0.843413
0.010254
0.003358
0.022159
0.009232
14
0.607341
0.023096
0.808081
0.859794
40
2
True
{u'features__pca__n_components': 40, u'linear_...
27
0.797024
0.864900
0.784680
0.860402
0.842583
0.854079
0.121846
0.001462
0.024888
0.004438
15
0.555552
0.021134
0.808078
0.859794
40
2
False
{u'features__pca__n_components': 40, u'linear_...
28
0.797026
0.864899
0.784670
0.860405
0.842580
0.854079
0.038250
0.000075
0.024890
0.004439
16
0.601110
0.022913
0.812496
0.861729
40
3
True
{u'features__pca__n_components': 40, u'linear_...
26
0.797006
0.864914
0.788382
0.863948
0.852161
0.856326
0.120713
0.001647
0.028241
0.003841
17
0.601864
0.037135
0.812497
0.861730
40
3
False
{u'features__pca__n_components': 40, u'linear_...
25
0.797005
0.864914
0.788384
0.863951
0.852162
0.856325
0.091311
0.022829
0.028240
0.003842
18
0.572880
0.023073
0.820941
0.874399
50
1
True
{u'features__pca__n_components': 50, u'linear_...
23
0.816552
0.876597
0.798133
0.880123
0.848156
0.866477
0.028398
0.000533
0.020644
0.005784
19
0.594553
0.024487
0.820940
0.874398
50
1
False
{u'features__pca__n_components': 50, u'linear_...
24
0.816552
0.876596
0.798132
0.880120
0.848152
0.866478
0.048878
0.000322
0.020643
0.005782
20
0.556988
0.023851
0.822177
0.874960
50
2
True
{u'features__pca__n_components': 50, u'linear_...
21
0.816663
0.876934
0.798449
0.880184
0.851440
0.867761
0.032029
0.001164
0.021969
0.005260
21
0.571599
0.027943
0.822176
0.874959
50
2
False
{u'features__pca__n_components': 50, u'linear_...
22
0.816662
0.876934
0.798446
0.880182
0.851443
0.867761
0.023973
0.003345
0.021972
0.005260
22
0.572743
0.022977
0.825356
0.876068
50
3
True
{u'features__pca__n_components': 50, u'linear_...
20
0.816634
0.876938
0.802098
0.881109
0.857369
0.870159
0.048868
0.000837
0.023381
0.004512
23
0.557765
0.024122
0.825356
0.876069
50
3
False
{u'features__pca__n_components': 50, u'linear_...
19
0.816637
0.876939
0.802095
0.881110
0.857370
0.870159
0.027137
0.001066
0.023382
0.004513
24
0.504011
0.026222
0.828412
0.883298
60
1
True
{u'features__pca__n_components': 60, u'linear_...
17
0.826015
0.884853
0.805166
0.887547
0.854065
0.877493
0.019710
0.001540
0.020022
0.004249
25
0.510764
0.023744
0.828272
0.883313
60
1
False
{u'features__pca__n_components': 60, u'linear_...
18
0.825851
0.884881
0.805210
0.887639
0.853764
0.877419
0.011103
0.000219
0.019884
0.004317
26
0.510834
0.035741
0.829251
0.884060
60
2
True
{u'features__pca__n_components': 60, u'linear_...
16
0.826156
0.885247
0.803785
0.887867
0.857823
0.879064
0.018786
0.009763
0.022155
0.003691
27
0.496351
0.024017
0.829318
0.884085
60
2
False
{u'features__pca__n_components': 60, u'linear_...
15
0.825874
0.885344
0.804198
0.887782
0.857895
0.879130
0.004878
0.000068
0.022043
0.003643
28
0.501100
0.025562
0.831808
0.884907
60
3
True
{u'features__pca__n_components': 60, u'linear_...
13
0.827106
0.885760
0.804760
0.887864
0.863575
0.881097
0.015189
0.001332
0.024226
0.002828
29
0.515190
0.024539
0.831783
0.884905
60
3
False
{u'features__pca__n_components': 60, u'linear_...
14
0.827427
0.885748
0.804477
0.887866
0.863463
0.881099
0.029267
0.000515
0.024262
0.002826
30
0.529060
0.028845
0.835215
0.891653
70
1
True
{u'features__pca__n_components': 70, u'linear_...
12
0.833982
0.890133
0.804544
0.899811
0.867123
0.885013
0.021869
0.003253
0.025546
0.006136
31
0.560635
0.026237
0.835405
0.891646
70
1
False
{u'features__pca__n_components': 70, u'linear_...
11
0.833877
0.890118
0.804950
0.899814
0.867393
0.885007
0.038624
0.001099
0.025499
0.006141
32
0.519889
0.029008
0.836000
0.893314
70
2
True
{u'features__pca__n_components': 70, u'linear_...
9
0.834359
0.890495
0.797866
0.900686
0.875780
0.888761
0.012223
0.005080
0.031809
0.005261
33
0.525165
0.035795
0.835936
0.893322
70
2
False
{u'features__pca__n_components': 70, u'linear_...
10
0.834192
0.890494
0.797814
0.900662
0.875809
0.888810
0.022771
0.011796
0.031845
0.005235
34
0.517272
0.027509
0.836672
0.893948
70
3
True
{u'features__pca__n_components': 70, u'linear_...
8
0.835704
0.891134
0.796933
0.901465
0.877383
0.889245
0.016708
0.002982
0.032829
0.005371
35
0.521266
0.027062
0.836728
0.893935
70
3
False
{u'features__pca__n_components': 70, u'linear_...
7
0.835643
0.891143
0.796925
0.901459
0.877621
0.889202
0.030661
0.002282
0.032932
0.005379
36
0.584862
0.028047
0.845804
0.897511
80
1
True
{u'features__pca__n_components': 80, u'linear_...
5
0.850921
0.896441
0.821547
0.905277
0.864925
0.890817
0.012852
0.001695
0.018065
0.005952
37
0.585904
0.029049
0.845660
0.897559
80
1
False
{u'features__pca__n_components': 80, u'linear_...
6
0.850876
0.896452
0.821137
0.905379
0.864948
0.890847
0.020541
0.003151
0.018252
0.005984
38
0.577818
0.030924
0.847378
0.898759
80
2
True
{u'features__pca__n_components': 80, u'linear_...
4
0.850660
0.896704
0.819745
0.906002
0.871717
0.893571
0.043109
0.004548
0.021331
0.005279
39
0.564118
0.026724
0.847600
0.898831
80
2
False
{u'features__pca__n_components': 80, u'linear_...
3
0.851302
0.896884
0.819775
0.906080
0.871709
0.893528
0.008944
0.000467
0.021350
0.005306
40
0.582777
0.028402
0.847728
0.899165
80
3
True
{u'features__pca__n_components': 80, u'linear_...
2
0.851702
0.897333
0.818966
0.906538
0.872501
0.893625
0.039071
0.001119
0.022022
0.005428
41
0.585593
0.026860
0.847885
0.899070
80
3
False
{u'features__pca__n_components': 80, u'linear_...
1
0.851740
0.896946
0.819711
0.906681
0.872188
0.893583
0.028587
0.000699
0.021584
0.005554
Completed in -878.54 sec
In [87]:
print len(y_pipelines_linsel)
print y_scores_linsel
5
[0.16842671848506297, 1.1007642117503149, 0.17877103111731135, 0.15445712779083717, 0.10969609770534818]
In [88]:
# Ridge Regression with PCA combinations
y_pipelines_ridge = []
y_scores_ridge = []
start = time.time()
for ind, y in enumerate(y_vars):
X_train, X_test, y_train, y_test = train_test_split(X_wDepth, y, test_size=0.33, random_state=42)
# set up the train and test data
print '\n----------', y_var_labels[ind]
pca = PCA()
ridge = Ridge()
steps = [('pca', pca), ('ridge', ridge)]
pipeline = Pipeline(steps)
parameters = dict(pca__n_components=list(range(20, 90, 10)),
ridge__alpha=np.linspace(0.0, 0.5, 5),
ridge__normalize=[True, False])
cv = GridSearchCV(pipeline, param_grid=parameters, verbose=0)
cv.fit(X_train, y_train)
print 'Cross_val_score: ', cross_val_score(cv, X_test, y_test)
y_predictions = cv.predict(X_test)
mse = mean_squared_error(y_test, y_predictions)
print 'Explained variance score: ', explained_variance_score(y_test, y_predictions)
print 'Mean absolute error: ', mean_absolute_error(y_test, y_predictions)
print 'Mean squared error: ', mse
print 'R2 score: ', r2_score(y_test, y_predictions)
display(pd.DataFrame.from_dict(cv.cv_results_))
# capture the best pipeline estimator and mse value
y_pipelines_ridge.append(cv.best_estimator_)
y_scores_ridge.append(mse)
print 'Completed in %0.2f sec' % (start-time.time())
---------- Ca
Cross_val_score: [ 0.83355056 0.74513654 0.86852704]
Explained variance score: 0.901076393629
Mean absolute error: 0.196188430286
Mean squared error: 0.168774203968
R2 score: 0.900993895701
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_ridge__alpha
param_ridge__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.447142
0.017420
0.751867
0.825906
20
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
65
0.785879
0.823166
0.727543
0.836301
0.742048
0.818251
0.084956
0.002561
0.024812
0.007619
1
0.344565
0.014887
0.751867
0.825906
20
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
66
0.785879
0.823166
0.727543
0.836301
0.742047
0.818251
0.083496
0.000415
0.024812
0.007619
2
0.365075
0.015547
0.756259
0.815710
20
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
60
0.814786
0.813004
0.748384
0.825976
0.705381
0.808149
0.112105
0.001013
0.045024
0.007525
3
0.380999
0.023149
0.751946
0.825906
20
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
64
0.786149
0.823166
0.727671
0.836300
0.741885
0.818251
0.053809
0.009556
0.024916
0.007619
4
0.426613
0.019070
0.743885
0.792870
20
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
67
0.816852
0.790240
0.746732
0.802849
0.667789
0.785521
0.118118
0.002596
0.060907
0.007314
5
0.336156
0.021557
0.752023
0.825905
20
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
63
0.786417
0.823165
0.727797
0.836299
0.741722
0.818250
0.022640
0.007412
0.025021
0.007619
6
0.488623
0.022276
0.723257
0.764475
20
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
69
0.804618
0.761939
0.733264
0.774097
0.631572
0.757390
0.108155
0.008844
0.071021
0.007052
7
0.321908
0.017872
0.752100
0.825903
20
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
62
0.786683
0.823163
0.727922
0.836298
0.741560
0.818249
0.019730
0.004694
0.025125
0.007619
8
0.293679
0.016120
0.698844
0.734139
20
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
70
0.784850
0.731704
0.713712
0.743378
0.597637
0.727334
0.013567
0.001398
0.077172
0.006773
9
0.315615
0.015625
0.752176
0.825901
20
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
61
0.786947
0.823161
0.728047
0.836296
0.741398
0.818247
0.022597
0.000817
0.025230
0.007619
10
0.316556
0.018938
0.809155
0.875621
30
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
48
0.845905
0.865919
0.781084
0.889915
0.800332
0.871029
0.015845
0.001910
0.027195
0.010321
11
0.361826
0.022145
0.809155
0.875621
30
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
47
0.845906
0.865919
0.781084
0.889915
0.800332
0.871029
0.044176
0.007991
0.027196
0.010321
12
0.433557
0.028742
0.810163
0.864811
30
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
41
0.862095
0.855228
0.804990
0.878929
0.763202
0.860275
0.053069
0.014926
0.040551
0.010193
13
0.363919
0.036307
0.809236
0.875618
30
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
46
0.846159
0.865916
0.781413
0.889912
0.799992
0.871025
0.048317
0.028547
0.027235
0.010320
14
0.413747
0.029345
0.794545
0.840596
30
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
53
0.855075
0.831282
0.804167
0.854319
0.724157
0.836187
0.082475
0.014860
0.053895
0.009908
15
0.412982
0.023365
0.809309
0.875609
30
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
45
0.846403
0.865909
0.781728
0.889902
0.799651
0.871016
0.100882
0.003767
0.027279
0.010319
16
0.356453
0.017163
0.770907
0.810492
30
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
58
0.836127
0.801512
0.790305
0.823723
0.686036
0.806241
0.017866
0.001166
0.062808
0.009553
17
0.327283
0.017851
0.809374
0.875595
30
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
44
0.846639
0.865898
0.782030
0.889885
0.799310
0.871000
0.023489
0.000266
0.027326
0.010318
18
0.471370
0.030900
0.743743
0.778330
30
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
68
0.811260
0.769706
0.769687
0.791036
0.650022
0.774248
0.104347
0.011570
0.068349
0.009174
19
0.374204
0.018378
0.809433
0.875575
30
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
43
0.846866
0.865882
0.782320
0.889863
0.798968
0.870979
0.100079
0.002692
0.027376
0.010315
20
0.422687
0.017088
0.838994
0.913629
40
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
34
0.893388
0.900659
0.776746
0.934486
0.846636
0.905741
0.052511
0.000454
0.047939
0.014894
21
0.391820
0.020901
0.838993
0.913629
40
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
35
0.893388
0.900659
0.776752
0.934486
0.846628
0.905742
0.017940
0.003139
0.047936
0.014894
22
0.394326
0.018664
0.839580
0.902349
40
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
31
0.901972
0.889540
0.807324
0.922949
0.809203
0.894558
0.008615
0.001477
0.044210
0.014710
23
0.555692
0.021570
0.839295
0.913605
40
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
33
0.893781
0.900643
0.777989
0.934465
0.845904
0.905706
0.109273
0.003300
0.047517
0.014894
24
0.544452
0.020041
0.823114
0.877083
40
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
40
0.889290
0.864633
0.810719
0.897107
0.769076
0.869511
0.090279
0.003231
0.049868
0.014298
25
0.782726
0.034152
0.839551
0.913540
40
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
32
0.894114
0.900597
0.779152
0.934409
0.845175
0.905615
0.078799
0.012731
0.047116
0.014898
26
0.393302
0.017893
0.798429
0.845672
40
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
52
0.865985
0.833668
0.799575
0.864979
0.729466
0.838368
0.011772
0.000550
0.055758
0.013786
27
0.612179
0.022823
0.839745
0.913438
40
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
30
0.894398
0.900526
0.780239
0.934319
0.844386
0.905468
0.207485
0.004806
0.046735
0.014903
28
0.587802
0.026332
0.770155
0.812114
40
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
59
0.837677
0.800586
0.780715
0.830654
0.691810
0.805103
0.127821
0.004507
0.060034
0.013239
29
0.481338
0.017140
0.839896
0.913306
40
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
29
0.894639
0.900430
0.781275
0.934204
0.843563
0.905285
0.077603
0.000420
0.046368
0.014909
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
40
0.478075
0.032999
0.853415
0.927482
60
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
20
0.899136
0.922143
0.810123
0.942018
0.850809
0.918286
0.096770
0.010064
0.036398
0.010398
41
0.648326
0.027211
0.853414
0.927467
60
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
21
0.899138
0.922081
0.810094
0.942041
0.850833
0.918279
0.179413
0.003018
0.036409
0.010421
42
0.536177
0.019969
0.850763
0.916017
60
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
22
0.903418
0.910739
0.835065
0.930423
0.813601
0.906889
0.269162
0.000273
0.038319
0.010307
43
0.341317
0.023910
0.854376
0.927269
60
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
18
0.902375
0.921839
0.809239
0.941918
0.851330
0.918049
0.011293
0.002971
0.038096
0.010474
44
0.440317
0.024053
0.832163
0.890361
60
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
37
0.888007
0.885194
0.834295
0.904371
0.773971
0.881517
0.060474
0.004023
0.046594
0.010020
45
0.434900
0.025521
0.854694
0.926801
60
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
15
0.904229
0.921223
0.808362
0.941624
0.851300
0.917556
0.113661
0.006775
0.039223
0.010588
46
0.339981
0.021282
0.805852
0.858483
60
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
50
0.862804
0.853562
0.819940
0.871961
0.734592
0.849925
0.009142
0.002437
0.053296
0.009645
47
0.353006
0.020783
0.854692
0.926292
60
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
16
0.905585
0.920714
0.807279
0.941249
0.851016
0.916913
0.018384
0.000745
0.040230
0.010689
48
0.356103
0.021036
0.776182
0.824403
60
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
56
0.833343
0.819648
0.798612
0.837368
0.696368
0.816192
0.019018
0.001249
0.058139
0.009276
49
0.367323
0.021134
0.854589
0.925700
60
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
17
0.906361
0.919997
0.806827
0.940855
0.850378
0.916249
0.018776
0.000972
0.040756
0.010825
50
0.639681
0.092506
0.850061
0.935042
70
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
23
0.900922
0.930258
0.800960
0.944855
0.848104
0.930013
0.113733
0.084075
0.040846
0.006939
51
0.508857
0.025836
0.850060
0.935048
70
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
24
0.900895
0.930245
0.800999
0.944886
0.848088
0.930013
0.066748
0.003954
0.040819
0.006957
52
0.440163
0.021908
0.848181
0.923507
70
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
28
0.903126
0.918694
0.830393
0.933239
0.810813
0.918589
0.055562
0.000734
0.039738
0.006881
53
0.376866
0.023093
0.854240
0.934038
70
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
19
0.907709
0.929190
0.804266
0.944399
0.850540
0.928527
0.009395
0.001085
0.042325
0.007331
54
0.413853
0.023551
0.830686
0.897611
70
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
39
0.886497
0.893027
0.834322
0.907043
0.771024
0.892764
0.034588
0.003281
0.047227
0.006670
55
0.360610
0.023332
0.855481
0.932635
70
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
13
0.910052
0.927679
0.805027
0.943691
0.851152
0.926535
0.017887
0.004383
0.042999
0.007832
56
0.340062
0.021215
0.804795
0.865464
70
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
51
0.860557
0.861123
0.822519
0.874540
0.731094
0.860730
0.009172
0.001065
0.054332
0.006420
57
0.340108
0.020398
0.855641
0.931295
70
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
12
0.910964
0.926157
0.804977
0.943029
0.850766
0.924698
0.009050
0.000092
0.043419
0.008319
58
0.338074
0.020315
0.775828
0.831171
70
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
57
0.830502
0.827049
0.802566
0.839878
0.694206
0.826588
0.007705
0.000054
0.058778
0.006159
59
0.342725
0.023905
0.855677
0.930082
70
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
11
0.911388
0.924790
0.805103
0.942395
0.850325
0.923062
0.002572
0.005186
0.043569
0.008735
60
0.408029
0.022035
0.848412
0.941788
80
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
26
0.906824
0.937565
0.786687
0.949542
0.851498
0.938258
0.005943
0.000109
0.049110
0.005490
61
0.415862
0.022492
0.848330
0.941807
80
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
27
0.906417
0.937626
0.787141
0.949485
0.851206
0.938309
0.014682
0.000580
0.048752
0.005436
62
0.410715
0.022365
0.849196
0.930247
80
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
25
0.908986
0.926281
0.821516
0.937890
0.816853
0.926569
0.010045
0.000183
0.042403
0.005406
63
0.413785
0.022355
0.855879
0.939211
80
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
10
0.912973
0.934805
0.801822
0.947861
0.852620
0.934968
0.025694
0.000381
0.045450
0.006117
64
0.416099
0.021978
0.831859
0.904072
80
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
38
0.891417
0.900094
0.826424
0.911544
0.777506
0.900577
0.008239
0.000017
0.046677
0.005288
65
0.409706
0.022427
0.856905
0.936481
80
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
6
0.913732
0.931790
0.804417
0.946153
0.852344
0.931500
0.015277
0.000656
0.044758
0.006840
66
0.566772
0.023306
0.806786
0.871818
80
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
49
0.865167
0.867982
0.816289
0.879014
0.738675
0.868456
0.092707
0.000932
0.052091
0.005092
67
0.559252
0.029834
0.856968
0.934368
80
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
5
0.913830
0.929456
0.805016
0.944944
0.851837
0.928703
0.172798
0.004001
0.044585
0.007485
68
0.426797
0.029178
0.778092
0.837201
80
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
55
0.834630
0.833493
0.797474
0.844151
0.701954
0.833960
0.010003
0.004489
0.055885
0.004918
69
0.490358
0.023170
0.856666
0.932673
80
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
8
0.913709
0.927536
0.805056
0.944007
0.851013
0.926476
0.073159
0.000476
0.044551
0.008026
70 rows × 19 columns
---------- P
Cross_val_score: [ 0.16529664 0.06014706 0.11388095]
Explained variance score: 0.117420405059
Mean absolute error: 0.454284009096
Mean squared error: 1.07343443002
R2 score: 0.116378859674
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_ridge__alpha
param_ridge__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.463123
0.023802
-0.051894
0.114146
20
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
41
0.018079
0.133428
0.033473
0.088777
-0.207504
0.120232
0.080228
0.000570
0.110107
0.018730
1
0.299254
0.015559
-0.051894
0.114146
20
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
40
0.018079
0.133428
0.033473
0.088777
-0.207504
0.120232
0.004040
0.001447
0.110107
0.018730
2
0.289663
0.014606
-0.029590
0.112737
20
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
28
0.023911
0.131781
0.035520
0.087681
-0.148409
0.118748
0.006058
0.000153
0.084070
0.018499
3
0.321524
0.015522
-0.051732
0.114146
20
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
39
0.018121
0.133428
0.033495
0.088777
-0.207082
0.120232
0.047015
0.001315
0.109923
0.018730
4
0.315368
0.015884
-0.014874
0.109580
20
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
23
0.027137
0.128091
0.036077
0.085226
-0.107998
0.115423
0.020736
0.001355
0.065886
0.017981
5
0.297203
0.015084
-0.051570
0.114145
20
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
38
0.018163
0.133428
0.033518
0.088777
-0.206661
0.120232
0.004298
0.000232
0.109739
0.018730
6
0.317276
0.018138
-0.004899
0.105656
20
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
11
0.028824
0.123503
0.035820
0.082174
-0.079473
0.111289
0.013648
0.002310
0.052758
0.017337
7
0.310764
0.015662
-0.051409
0.114145
20
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
37
0.018205
0.133427
0.033540
0.088777
-0.206242
0.120231
0.011408
0.001090
0.109557
0.018730
8
0.300032
0.015425
0.001992
0.101463
20
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
10
0.029576
0.118602
0.035115
0.078913
-0.058823
0.106873
0.003942
0.000435
0.043020
0.016649
9
0.330638
0.014660
-0.051249
0.114145
20
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
36
0.018247
0.133427
0.033562
0.088777
-0.205825
0.120231
0.026824
0.000033
0.109375
0.018730
10
0.322219
0.016615
-0.091704
0.143315
30
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
55
0.020377
0.169673
0.045032
0.101407
-0.340954
0.158865
0.027901
0.000959
0.176364
0.029960
11
0.312709
0.030255
-0.091704
0.143315
30
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
56
0.020377
0.169673
0.045031
0.101407
-0.340954
0.158865
0.015926
0.019150
0.176364
0.029960
12
0.324079
0.017543
-0.059435
0.141546
30
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
43
0.028045
0.167578
0.046771
0.100156
-0.253458
0.156904
0.011447
0.000741
0.137276
0.029590
13
0.329056
0.017410
-0.091029
0.143313
30
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
54
0.020603
0.169670
0.044965
0.101407
-0.339087
0.158863
0.022551
0.001070
0.175516
0.029959
14
0.333956
0.016801
-0.037743
0.137583
30
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
32
0.032363
0.162886
0.046907
0.097351
-0.192770
0.152511
0.022429
0.000680
0.109676
0.028761
15
0.325206
0.040679
-0.090367
0.143308
30
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
53
0.020822
0.169661
0.044899
0.101405
-0.337253
0.158856
0.018768
0.034836
0.174683
0.029956
16
0.318255
0.015898
-0.022721
0.132655
30
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
25
0.034695
0.157052
0.046188
0.093865
-0.149270
0.147049
0.027904
0.000186
0.089520
0.027731
17
0.307293
0.020640
-0.089718
0.143298
30
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
52
0.021033
0.169647
0.044835
0.101402
-0.335451
0.158846
0.006322
0.006606
0.173863
0.029951
18
0.323421
0.016471
-0.012078
0.127391
30
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
17
0.035813
0.150820
0.045019
0.090140
-0.117251
0.141214
0.004279
0.000533
0.074392
0.026631
19
0.335579
0.016776
-0.089081
0.143286
30
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
51
0.021238
0.169628
0.044772
0.101399
-0.333679
0.158831
0.035903
0.000310
0.173057
0.029945
20
0.401422
0.021088
-0.111019
0.182868
40
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
66
0.023843
0.206228
0.043379
0.143168
-0.400802
0.199209
0.014795
0.002737
0.204865
0.028218
21
0.397186
0.020131
-0.111036
0.182873
40
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
67
0.023835
0.206238
0.043402
0.143165
-0.400869
0.199216
0.021625
0.003144
0.204900
0.028224
22
0.382424
0.019273
-0.070683
0.180607
40
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
47
0.033327
0.203672
0.049601
0.141397
-0.295381
0.196752
0.012379
0.002154
0.158871
0.027869
23
0.402281
0.017981
-0.107682
0.182831
40
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
64
0.024314
0.206200
0.044419
0.143122
-0.392289
0.199171
0.019188
0.001186
0.201221
0.028225
24
0.434214
0.017557
-0.043603
0.175553
40
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
33
0.038697
0.197975
0.052532
0.137439
-0.222356
0.191244
0.031203
0.000493
0.126402
0.027090
25
0.396556
0.017696
-0.104612
0.182741
40
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
63
0.024743
0.206136
0.045300
0.143005
-0.384380
0.199083
0.007392
0.000345
0.197813
0.028245
26
0.398960
0.018627
-0.024889
0.169267
40
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
27
0.041628
0.190887
0.053614
0.132519
-0.170168
0.184396
0.016362
0.001101
0.102744
0.026120
27
0.406134
0.017514
-0.101854
0.182601
40
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
61
0.025133
0.206036
0.046072
0.142828
-0.377260
0.198939
0.012619
0.000414
0.194741
0.028273
28
0.400553
0.019991
-0.011639
0.162547
40
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
16
0.043069
0.183308
0.053561
0.127259
-0.131758
0.177075
0.027189
0.003595
0.084963
0.025082
29
0.391186
0.026814
-0.099286
0.182423
40
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
60
0.025501
0.205907
0.046730
0.142606
-0.370572
0.198755
0.004497
0.013293
0.191839
0.028306
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
40
0.407268
0.020905
-0.095497
0.297577
60
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
59
-0.008883
0.346128
0.061851
0.248164
-0.339794
0.298438
0.022338
0.000531
0.174978
0.039998
41
0.359204
0.023617
-0.095463
0.297779
60
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
58
-0.008986
0.346370
0.061664
0.248543
-0.339402
0.298424
0.016318
0.003647
0.174723
0.039940
42
0.357268
0.020142
-0.044762
0.293840
60
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
34
0.013365
0.341636
0.074096
0.245185
-0.221973
0.294699
0.008283
0.000434
0.127619
0.039381
43
0.357800
0.022317
-0.077774
0.295530
60
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
48
0.008255
0.343692
0.071803
0.245894
-0.313713
0.297005
0.025325
0.002895
0.168681
0.039940
44
0.365197
0.020173
-0.011092
0.285563
60
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
13
0.027852
0.332106
0.081339
0.238051
-0.142618
0.286531
0.029814
0.000668
0.095446
0.038404
45
0.346510
0.019970
-0.067165
0.291999
60
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
46
0.018554
0.339865
0.077199
0.241940
-0.297581
0.294192
0.003490
0.000572
0.164523
0.040008
46
0.354211
0.022033
0.011506
0.275278
60
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
8
0.036371
0.320193
0.084695
0.229537
-0.086643
0.276105
0.020347
0.002230
0.072089
0.037015
47
0.354348
0.023772
-0.060243
0.287883
60
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
44
0.026483
0.334212
0.079301
0.238566
-0.286849
0.290870
0.018993
0.000998
0.161526
0.039104
48
0.340787
0.023664
0.026306
0.264050
60
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
5
0.040922
0.306617
0.085994
0.220329
-0.048055
0.265202
0.010379
0.002772
0.055662
0.035236
49
0.352175
0.021683
-0.055104
0.282842
60
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
42
0.031573
0.328281
0.081635
0.232818
-0.278858
0.287426
0.011063
0.002650
0.159381
0.039107
50
0.364361
0.021073
-0.032640
0.355899
70
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
30
0.050008
0.408989
0.159468
0.298554
-0.307715
0.360153
0.017995
0.000830
0.199395
0.045185
51
0.357347
0.022076
-0.031728
0.355359
70
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
29
0.050089
0.409016
0.159346
0.298563
-0.304937
0.358499
0.004295
0.001068
0.198091
0.045147
52
0.352556
0.021990
0.014020
0.351231
70
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
7
0.072350
0.403762
0.166398
0.295042
-0.196913
0.354890
0.014623
0.001522
0.153878
0.044460
53
0.357084
0.021364
-0.016055
0.348392
70
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
24
0.057415
0.401128
0.151355
0.291837
-0.257220
0.352210
0.008120
0.000525
0.174630
0.044699
54
0.382945
0.022261
0.046098
0.341812
70
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
3
0.086909
0.392585
0.167194
0.286688
-0.115967
0.346165
0.022929
0.001178
0.119088
0.043342
55
0.428890
0.024408
-0.012950
0.337018
70
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
19
0.059077
0.388531
0.142614
0.281208
-0.240821
0.341315
0.093158
0.003321
0.164548
0.043920
56
0.375307
0.021164
0.064207
0.329080
70
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
2
0.094961
0.378661
0.164835
0.276280
-0.067294
0.332300
0.012124
0.000853
0.097179
0.041859
57
0.355606
0.021997
-0.013762
0.326542
70
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
21
0.060658
0.377034
0.135307
0.271575
-0.237540
0.331017
0.022811
0.000966
0.160994
0.043170
58
0.422915
0.023833
0.077358
0.316320
70
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
1
0.098367
0.363355
0.161537
0.265453
-0.027912
0.320151
0.063723
0.001875
0.078712
0.040060
59
0.452948
0.028068
-0.014807
0.317354
70
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
22
0.060730
0.366356
0.129141
0.262853
-0.234586
0.322853
0.048472
0.005209
0.157750
0.042433
60
0.433100
0.023195
-0.113660
0.413658
80
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
70
0.025592
0.462150
0.061356
0.372668
-0.428469
0.406156
0.002742
0.000600
0.222867
0.036914
61
0.413792
0.024766
-0.110106
0.412839
80
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
65
0.024962
0.460997
0.065508
0.372399
-0.421311
0.405123
0.006703
0.002162
0.220465
0.036579
62
0.445696
0.023671
-0.048464
0.407694
80
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
35
0.055168
0.455231
0.089744
0.367681
-0.290706
0.400171
0.019450
0.002229
0.171707
0.036136
63
0.413434
0.029514
-0.024501
0.391382
80
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
26
0.054312
0.441162
0.124071
0.346450
-0.252190
0.386535
0.015010
0.004360
0.163349
0.038818
64
0.425328
0.023289
-0.005187
0.395945
80
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
12
0.076155
0.441745
0.102565
0.356860
-0.194596
0.389231
0.016303
0.000725
0.134237
0.034978
65
0.427526
0.028726
-0.013439
0.369207
80
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
20
0.060139
0.419041
0.128376
0.321498
-0.229117
0.367083
0.013049
0.006693
0.154887
0.039850
66
0.425267
0.023734
0.023317
0.381993
80
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
6
0.086155
0.425394
0.112130
0.345108
-0.128579
0.375478
0.003574
0.002005
0.107825
0.033099
67
0.443106
0.022585
-0.011585
0.351855
80
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
15
0.061751
0.401269
0.126558
0.302654
-0.223348
0.351643
0.020479
0.000085
0.151917
0.040260
68
0.458216
0.024431
0.042636
0.367060
80
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
4
0.093534
0.409945
0.115847
0.330857
-0.081672
0.360377
0.035934
0.001135
0.088285
0.032631
69
0.423068
0.022897
-0.012851
0.338229
80
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
18
0.062320
0.386602
0.122567
0.288605
-0.223731
0.339482
0.002709
0.000268
0.150989
0.040017
70 rows × 19 columns
---------- pH
Cross_val_score: [ 0.7400251 0.77567394 0.7536067 ]
Explained variance score: 0.814266745755
Mean absolute error: 0.289874144105
Mean squared error: 0.164033969394
R2 score: 0.813738036703
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_ridge__alpha
param_ridge__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.422483
0.017436
0.684171
0.720647
20
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
65
0.646453
0.733411
0.712437
0.710280
0.693770
0.718250
0.054303
0.003305
0.027787
0.009594
1
0.297742
0.015090
0.684171
0.720647
20
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
64
0.646453
0.733411
0.712437
0.710280
0.693770
0.718250
0.006659
0.000514
0.027787
0.009594
2
0.317768
0.022736
0.679800
0.711750
20
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
66
0.659117
0.724356
0.699379
0.701511
0.680985
0.709383
0.018677
0.010535
0.016464
0.009475
3
0.299269
0.016073
0.684198
0.720647
20
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
63
0.646510
0.733411
0.712430
0.710280
0.693801
0.718250
0.015072
0.001405
0.027762
0.009594
4
0.318312
0.016976
0.663333
0.691821
20
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
67
0.654005
0.704074
0.676887
0.681869
0.659143
0.689520
0.028375
0.000809
0.009802
0.009210
5
0.305089
0.014730
0.684225
0.720647
20
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
62
0.646567
0.733410
0.712423
0.710280
0.693832
0.718250
0.006345
0.000265
0.027738
0.009594
6
0.310762
0.015523
0.641284
0.667045
20
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
69
0.639745
0.678859
0.650521
0.657449
0.633592
0.664827
0.014231
0.000499
0.006992
0.008880
7
0.312237
0.018608
0.684252
0.720646
20
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
61
0.646623
0.733410
0.712417
0.710280
0.693862
0.718249
0.031784
0.005134
0.027713
0.009594
8
0.310732
0.015301
0.617015
0.640575
20
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
70
0.620933
0.651921
0.623074
0.631360
0.607021
0.638445
0.026108
0.000223
0.007113
0.008528
9
0.305068
0.015701
0.684279
0.720646
20
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
60
0.646680
0.733410
0.712410
0.710279
0.693893
0.718249
0.016110
0.001570
0.027689
0.009594
10
0.327728
0.018435
0.733384
0.782674
30
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
47
0.714661
0.788172
0.742066
0.786209
0.743498
0.773640
0.025291
0.003054
0.013278
0.006438
11
0.369533
0.016227
0.733384
0.782674
30
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
46
0.714661
0.788172
0.742066
0.786209
0.743498
0.773640
0.044315
0.000496
0.013278
0.006438
12
0.386284
0.017968
0.729447
0.773011
30
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
48
0.723543
0.778442
0.731828
0.776503
0.732994
0.764089
0.035056
0.001901
0.004210
0.006359
13
0.398192
0.019508
0.733561
0.782668
30
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
45
0.714731
0.788167
0.742399
0.786203
0.743625
0.773635
0.023456
0.002030
0.013350
0.006438
14
0.356586
0.020661
0.712265
0.751367
30
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
52
0.714723
0.756645
0.710492
0.754761
0.711571
0.742694
0.049788
0.005743
0.001796
0.006181
15
0.330928
0.017064
0.733723
0.782653
30
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
44
0.714790
0.788154
0.742713
0.786185
0.743741
0.773621
0.013875
0.000404
0.013421
0.006437
16
0.330281
0.019730
0.688929
0.724458
30
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
57
0.696976
0.729548
0.684344
0.727731
0.685435
0.716096
0.011691
0.002463
0.005719
0.005959
17
0.349550
0.019608
0.733873
0.782629
30
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
43
0.714837
0.788132
0.743009
0.786156
0.743845
0.773598
0.033170
0.001935
0.013491
0.006437
18
0.336972
0.016839
0.663103
0.695710
30
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
68
0.674948
0.700597
0.656579
0.698852
0.657737
0.687680
0.019954
0.000959
0.008405
0.005723
19
0.330138
0.021244
0.734009
0.782595
30
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
42
0.714873
0.788102
0.743290
0.786117
0.743939
0.773566
0.014303
0.003247
0.013560
0.006436
20
0.413074
0.019672
0.759928
0.808877
40
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
30
0.747042
0.815273
0.760861
0.806182
0.771932
0.805178
0.009140
0.002249
0.010186
0.004541
21
0.451092
0.017410
0.759922
0.808874
40
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
31
0.747043
0.815273
0.760856
0.806178
0.771917
0.805171
0.009630
0.000299
0.010179
0.004543
22
0.444388
0.017784
0.755666
0.798892
40
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
35
0.756411
0.805210
0.748663
0.796230
0.761921
0.795237
0.017748
0.000306
0.005435
0.004486
23
0.412720
0.017869
0.760110
0.808848
40
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
29
0.747082
0.815241
0.761122
0.806156
0.772176
0.805146
0.011461
0.000163
0.010273
0.004539
24
0.387870
0.019705
0.737740
0.776518
40
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
40
0.747242
0.782662
0.725725
0.773933
0.740216
0.772958
0.011531
0.001585
0.008959
0.004363
25
0.417426
0.019815
0.760238
0.808773
40
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
28
0.747061
0.815160
0.761332
0.806088
0.772373
0.805070
0.009543
0.003268
0.010366
0.004536
26
0.429918
0.018489
0.713503
0.748715
40
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
51
0.728728
0.754633
0.698271
0.746221
0.713451
0.745292
0.024721
0.001357
0.012438
0.004202
27
0.396800
0.022928
0.760318
0.808655
40
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
26
0.746985
0.815034
0.761507
0.805983
0.772513
0.804948
0.007470
0.003650
0.010459
0.004530
28
0.408869
0.021285
0.686698
0.719003
40
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
58
0.705725
0.724688
0.669383
0.716605
0.684912
0.715716
0.004507
0.003565
0.014895
0.004036
29
0.409029
0.020034
0.760364
0.808503
40
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
25
0.746861
0.814867
0.761669
0.805850
0.772614
0.804793
0.009160
0.002994
0.010557
0.004521
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
40
0.411543
0.025739
0.762912
0.840117
60
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
20
0.748544
0.848740
0.786092
0.834856
0.754156
0.836754
0.062133
0.004779
0.016534
0.006147
41
0.400207
0.023301
0.762912
0.840169
60
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
21
0.748384
0.848776
0.786244
0.834933
0.754163
0.836798
0.011986
0.001044
0.016651
0.006134
42
0.374702
0.022615
0.760276
0.829705
60
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
27
0.758740
0.838185
0.773215
0.824631
0.748879
0.826300
0.006672
0.001132
0.009988
0.006034
43
0.370735
0.025204
0.768029
0.839433
60
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
16
0.755799
0.847925
0.786578
0.834124
0.761757
0.836249
0.008527
0.005640
0.013328
0.006067
44
0.383371
0.023532
0.743624
0.806459
60
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
38
0.750337
0.814664
0.749252
0.801310
0.731255
0.803404
0.016856
0.004045
0.008749
0.005864
45
0.375609
0.024867
0.770525
0.838013
60
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
14
0.758932
0.846131
0.785732
0.832774
0.766957
0.835135
0.015368
0.004141
0.011231
0.005821
46
0.392216
0.027826
0.720112
0.777796
60
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
50
0.731855
0.785731
0.721035
0.772916
0.707400
0.774740
0.036408
0.003427
0.010008
0.005660
47
0.390040
0.021862
0.771557
0.836317
60
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
13
0.760262
0.844178
0.784755
0.831233
0.769698
0.833538
0.020771
0.000695
0.010088
0.005638
48
0.386495
0.028286
0.693410
0.746744
60
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
56
0.708988
0.754412
0.690951
0.742091
0.680231
0.743730
0.018027
0.008862
0.011872
0.005463
49
0.402085
0.024489
0.772414
0.834828
60
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
12
0.760970
0.842339
0.783878
0.829858
0.772439
0.832287
0.029799
0.001730
0.009355
0.005403
50
0.421190
0.022397
0.768133
0.846805
70
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
15
0.750166
0.859275
0.791481
0.837868
0.762820
0.843273
0.024344
0.000753
0.017284
0.009089
51
0.411211
0.023270
0.767945
0.846796
70
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
17
0.749868
0.859274
0.791231
0.837800
0.762807
0.843316
0.021539
0.002201
0.017278
0.009106
52
0.414316
0.025500
0.764678
0.836346
70
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
18
0.759108
0.848570
0.776785
0.827599
0.758163
0.832869
0.017179
0.003656
0.008561
0.008908
53
0.412104
0.023697
0.773280
0.845425
70
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
11
0.760597
0.857174
0.790588
0.836907
0.768703
0.842196
0.035410
0.001388
0.012667
0.008583
54
0.443269
0.035746
0.746904
0.812931
70
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
37
0.749584
0.824868
0.750938
0.804336
0.740179
0.809588
0.029666
0.006978
0.004783
0.008709
55
0.398059
0.022197
0.775066
0.843092
70
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
10
0.764112
0.853894
0.788984
0.835167
0.772144
0.840216
0.022248
0.000549
0.010365
0.007911
56
0.398609
0.022748
0.722978
0.783909
70
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
49
0.731333
0.795529
0.721591
0.775563
0.715976
0.780635
0.006596
0.000629
0.006348
0.008473
57
0.433362
0.021875
0.775720
0.840833
70
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
9
0.765394
0.850907
0.787441
0.833397
0.774367
0.838196
0.014451
0.000296
0.009054
0.007388
58
0.416610
0.022233
0.695964
0.752679
70
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
55
0.708151
0.763748
0.690918
0.744777
0.688777
0.749511
0.015113
0.000585
0.008678
0.008062
59
0.452580
0.026330
0.775869
0.838719
70
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
8
0.765676
0.848229
0.785946
0.831638
0.776023
0.836290
0.056116
0.003576
0.008279
0.006988
60
0.490411
0.025599
0.785241
0.865563
80
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
3
0.761098
0.868815
0.816760
0.855637
0.777959
0.872235
0.028592
0.001215
0.023306
0.007156
61
0.488487
0.033497
0.785662
0.865511
80
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
2
0.762501
0.869004
0.817364
0.855777
0.777211
0.871751
0.011594
0.012642
0.023187
0.006973
62
0.484565
0.024072
0.781714
0.854630
80
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
7
0.767801
0.858188
0.798735
0.845003
0.778660
0.860697
0.014219
0.000595
0.012816
0.006883
63
0.492199
0.029475
0.786647
0.859351
80
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
1
0.768905
0.864474
0.806407
0.849923
0.784699
0.863655
0.029711
0.003336
0.015377
0.006675
64
0.464702
0.028661
0.763345
0.830844
80
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
19
0.756457
0.833617
0.769855
0.821764
0.763752
0.837151
0.010981
0.006809
0.005479
0.006581
65
0.460331
0.027887
0.785140
0.853389
80
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
4
0.770387
0.859550
0.800570
0.844740
0.784522
0.855876
0.010590
0.006172
0.012334
0.006297
66
0.465301
0.024387
0.738363
0.801597
80
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
39
0.737807
0.805717
0.737251
0.792017
0.740035
0.807058
0.015109
0.001440
0.001202
0.006797
67
0.467007
0.025170
0.783674
0.848999
80
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
5
0.770287
0.855338
0.796541
0.840913
0.784246
0.850745
0.010951
0.002499
0.010729
0.006017
68
0.472680
0.027830
0.711162
0.769322
80
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
54
0.712945
0.772240
0.705858
0.760655
0.714677
0.775072
0.014944
0.003142
0.003813
0.006237
69
0.458194
0.026129
0.782211
0.845459
80
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
6
0.769688
0.851888
0.793143
0.837792
0.783852
0.846696
0.020102
0.003292
0.009648
0.005821
70 rows × 19 columns
---------- SOC
Cross_val_score: [ 0.91615743 0.88907611 0.86478846]
Explained variance score: 0.907119040988
Mean absolute error: 0.246257375347
Mean squared error: 0.141440162243
R2 score: 0.906180411652
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_ridge__alpha
param_ridge__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.436218
0.017429
0.799626
0.812817
20
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
56
0.832424
0.798065
0.786214
0.820028
0.780114
0.820358
0.098613
0.002416
0.023369
0.010432
1
0.311620
0.015720
0.799626
0.812817
20
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
55
0.832424
0.798065
0.786214
0.820028
0.780114
0.820358
0.015939
0.000403
0.023369
0.010432
2
0.314333
0.016521
0.791002
0.802782
20
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
58
0.827441
0.788212
0.767458
0.809904
0.777965
0.810230
0.015814
0.001688
0.026170
0.010303
3
0.333721
0.016086
0.799628
0.812817
20
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
54
0.832435
0.798064
0.786190
0.820028
0.780132
0.820358
0.022242
0.000371
0.023374
0.010432
4
0.319809
0.016147
0.769641
0.780304
20
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
64
0.807698
0.766142
0.740066
0.787227
0.761011
0.787543
0.004629
0.000729
0.028284
0.010015
5
0.313778
0.015161
0.799630
0.812816
20
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
53
0.832446
0.798064
0.786166
0.820027
0.780149
0.820357
0.011658
0.000030
0.023379
0.010432
6
0.314067
0.017530
0.742602
0.752359
20
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
68
0.781125
0.738704
0.709463
0.759034
0.737069
0.759339
0.004870
0.002270
0.029525
0.009656
7
0.412016
0.017353
0.799631
0.812815
20
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
52
0.832457
0.798063
0.786141
0.820027
0.780166
0.820356
0.060097
0.002499
0.023384
0.010432
8
0.331978
0.017668
0.713496
0.722504
20
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
70
0.751818
0.709391
0.678330
0.728914
0.710192
0.729207
0.015121
0.002491
0.030102
0.009273
9
0.350113
0.017120
0.799631
0.812814
20
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
51
0.832467
0.798062
0.786117
0.820025
0.780183
0.820355
0.021054
0.001698
0.023389
0.010432
10
0.396738
0.021084
0.823940
0.846879
30
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
43
0.835253
0.843558
0.823200
0.845433
0.813325
0.851647
0.045992
0.005621
0.008970
0.003457
11
0.444506
0.019613
0.823940
0.846879
30
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
44
0.835253
0.843558
0.823200
0.845433
0.813325
0.851647
0.078927
0.001904
0.008970
0.003457
12
0.357798
0.020194
0.815409
0.836424
30
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
46
0.827575
0.833144
0.807183
0.834996
0.811422
0.841133
0.035286
0.004937
0.008791
0.003415
13
0.330392
0.018042
0.824032
0.846878
30
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
42
0.835544
0.843556
0.823086
0.845432
0.813421
0.851646
0.009840
0.001127
0.009059
0.003457
14
0.341687
0.017119
0.793620
0.813004
30
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
57
0.806103
0.809815
0.780749
0.811616
0.793958
0.817581
0.013895
0.000510
0.010357
0.003319
15
0.343653
0.020863
0.824119
0.846873
30
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
41
0.835827
0.843549
0.822972
0.845429
0.813512
0.851640
0.004093
0.004539
0.009149
0.003458
16
0.336571
0.018548
0.765898
0.783888
30
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
65
0.778400
0.780814
0.750116
0.782550
0.769129
0.788302
0.010934
0.001177
0.011774
0.003200
17
0.352776
0.016510
0.824200
0.846865
30
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
40
0.836099
0.843538
0.822857
0.845424
0.813597
0.851632
0.013513
0.000061
0.009238
0.003458
18
0.336952
0.019641
0.735994
0.752782
30
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
69
0.748347
0.749829
0.718397
0.751496
0.741191
0.757020
0.011493
0.004247
0.012770
0.003073
19
0.332671
0.017959
0.824276
0.846853
30
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
39
0.836363
0.843523
0.822742
0.845417
0.813676
0.851621
0.012869
0.001382
0.009328
0.003458
20
0.432061
0.022969
0.832804
0.862330
40
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
36
0.837140
0.857774
0.838185
0.862447
0.823070
0.866770
0.008439
0.002975
0.006889
0.003674
21
0.444260
0.023079
0.832802
0.862330
40
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
37
0.837139
0.857775
0.838186
0.862445
0.823066
0.866770
0.007853
0.006901
0.006892
0.003673
22
0.436367
0.020917
0.824725
0.851684
40
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
38
0.827575
0.847184
0.826544
0.851799
0.820045
0.856068
0.004206
0.003463
0.003333
0.003628
23
0.437776
0.018041
0.833176
0.862315
40
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
34
0.837838
0.857761
0.838073
0.862435
0.823600
0.866749
0.003209
0.000109
0.006765
0.003671
24
0.410572
0.019638
0.803037
0.827838
40
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
50
0.804894
0.823464
0.802525
0.827950
0.801686
0.832101
0.008017
0.001446
0.001359
0.003527
25
0.508245
0.022525
0.833494
0.862277
40
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
33
0.838481
0.857722
0.837942
0.862403
0.824041
0.866708
0.032281
0.004305
0.006682
0.003669
26
0.475381
0.022969
0.775229
0.798191
40
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
61
0.776406
0.793973
0.773148
0.798299
0.776128
0.802301
0.012955
0.005584
0.001474
0.003401
27
0.608488
0.023772
0.833769
0.862216
40
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
32
0.839068
0.857663
0.837797
0.862350
0.824421
0.866634
0.121857
0.001061
0.006624
0.003663
28
0.440309
0.047511
0.745135
0.766515
40
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
67
0.745830
0.762467
0.741980
0.766615
0.747592
0.770464
0.003517
0.032873
0.002342
0.003265
29
0.449943
0.019153
0.834004
0.862138
40
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
31
0.839610
0.857586
0.837639
0.862288
0.824742
0.866539
0.016934
0.001299
0.006593
0.003657
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
40
0.395426
0.023020
0.866777
0.905365
60
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
18
0.882635
0.900767
0.870639
0.906001
0.846994
0.909327
0.013325
0.001578
0.014809
0.003523
41
0.408404
0.023064
0.866895
0.905693
60
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
17
0.882089
0.900770
0.871532
0.906976
0.847004
0.909331
0.007227
0.000964
0.014698
0.003611
42
0.376071
0.022352
0.856558
0.894193
60
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
21
0.872943
0.889646
0.860500
0.894917
0.836167
0.898017
0.005618
0.000234
0.015274
0.003456
43
0.399568
0.022884
0.867625
0.904753
60
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
15
0.880510
0.900209
0.871419
0.905428
0.850897
0.908622
0.006989
0.001947
0.012387
0.003468
44
0.383357
0.025123
0.832881
0.869102
60
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
35
0.849352
0.864733
0.836637
0.869756
0.812590
0.872816
0.020733
0.003616
0.015246
0.003332
45
0.419911
0.027339
0.867734
0.903412
60
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
14
0.879240
0.898964
0.871142
0.904088
0.852774
0.907184
0.060991
0.008926
0.011073
0.003390
46
0.488417
0.028827
0.803320
0.838050
60
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
49
0.819580
0.833793
0.807079
0.838712
0.783239
0.841645
0.097427
0.008643
0.015077
0.003240
47
0.556733
0.029780
0.867037
0.901894
60
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
16
0.877761
0.897635
0.870296
0.902584
0.853013
0.905464
0.190178
0.008498
0.010365
0.003233
48
0.718313
0.028604
0.771883
0.804799
60
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
63
0.787348
0.800626
0.775374
0.805483
0.752867
0.808289
0.144030
0.004880
0.014296
0.003166
49
0.496051
0.033353
0.866223
0.900303
60
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
19
0.876408
0.896159
0.869201
0.900946
0.853022
0.903805
0.036063
0.012359
0.009779
0.003154
50
0.419189
0.035623
0.870998
0.919739
70
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
12
0.889987
0.915001
0.869335
0.925632
0.853599
0.918583
0.004249
0.003822
0.014906
0.004416
51
0.557221
0.048862
0.870853
0.919777
70
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
13
0.890081
0.915153
0.869176
0.925598
0.853227
0.918581
0.102210
0.029824
0.015097
0.004347
52
0.577577
0.026328
0.860805
0.908349
70
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
20
0.883766
0.903644
0.854224
0.914171
0.844336
0.907231
0.150262
0.001705
0.016760
0.004370
53
0.597887
0.043940
0.874335
0.917662
70
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
7
0.889970
0.912876
0.875625
0.923144
0.857349
0.916967
0.236610
0.022304
0.013353
0.004220
54
0.582774
0.031498
0.836768
0.882812
70
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
29
0.861440
0.878016
0.827707
0.888579
0.821062
0.881841
0.042324
0.005996
0.017688
0.004366
55
0.647915
0.031742
0.874234
0.914587
70
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
8
0.888018
0.909759
0.876128
0.919469
0.858504
0.914533
0.135008
0.001788
0.012127
0.003964
56
0.435086
0.027081
0.807195
0.851250
70
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
48
0.833440
0.846769
0.795889
0.856687
0.792155
0.850295
0.037516
0.002935
0.018656
0.004105
57
0.764890
0.047772
0.873064
0.911547
70
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
9
0.885640
0.906757
0.875205
0.915908
0.858298
0.911975
0.125843
0.028274
0.011268
0.003748
58
0.550643
0.025899
0.775210
0.817502
70
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
62
0.801986
0.813315
0.762781
0.822722
0.760758
0.816469
0.101130
0.000871
0.018989
0.003909
59
0.424401
0.025598
0.871700
0.908803
70
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
10
0.883682
0.904154
0.873821
0.912714
0.857549
0.909540
0.023981
0.001469
0.010777
0.003533
60
0.498131
0.028893
0.881897
0.930093
80
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
2
0.898736
0.923730
0.876665
0.938120
0.870223
0.928429
0.018005
0.001662
0.012217
0.005992
61
0.841252
0.035277
0.881692
0.930047
80
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
3
0.898538
0.923583
0.876305
0.938124
0.870168
0.928435
0.059262
0.011904
0.012194
0.006045
62
0.572404
0.029279
0.871372
0.918565
80
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
11
0.892450
0.912270
0.857650
0.926478
0.863934
0.916947
0.105349
0.004278
0.015151
0.005912
63
0.486728
0.028171
0.883353
0.925626
80
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
1
0.896993
0.919476
0.883187
0.932723
0.869825
0.924678
0.015178
0.003812
0.011095
0.005450
64
0.492371
0.028080
0.847230
0.892924
80
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
23
0.870689
0.886772
0.828193
0.900644
0.842717
0.891355
0.016785
0.002523
0.017645
0.005771
65
0.512443
0.026338
0.880944
0.920533
80
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
4
0.893089
0.914648
0.882120
0.926652
0.867575
0.920298
0.018101
0.001429
0.010452
0.004904
66
0.490478
0.026826
0.817103
0.860877
80
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
45
0.842056
0.855011
0.794907
0.868259
0.814249
0.859362
0.018104
0.001601
0.019360
0.005514
67
0.517873
0.025710
0.878414
0.916273
80
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
5
0.889715
0.910689
0.879949
0.921601
0.865535
0.916530
0.029381
0.001455
0.009934
0.004459
68
0.488117
0.028046
0.784422
0.826638
80
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
59
0.809393
0.820842
0.760852
0.833806
0.782925
0.825266
0.013982
0.001934
0.019851
0.005381
69
0.492939
0.026598
0.876143
0.912744
80
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
6
0.886923
0.907358
0.877790
0.917465
0.863673
0.913410
0.022684
0.001746
0.009566
0.004153
70 rows × 19 columns
---------- Sand
Cross_val_score: [ 0.86318998 0.82905191 0.87058466]
Explained variance score: 0.883032225871
Mean absolute error: 0.239758465292
Mean squared error: 0.115479041393
R2 score: 0.882849560337
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_ridge__alpha
param_ridge__normalize
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.342916
0.018127
0.717685
0.751043
20
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
65
0.705694
0.763371
0.724223
0.749680
0.723185
0.740078
0.026075
0.003583
0.008506
0.009558
1
0.327113
0.015242
0.717685
0.751043
20
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
64
0.705694
0.763371
0.724223
0.749680
0.723185
0.740078
0.013310
0.000142
0.008506
0.009558
2
0.303656
0.015412
0.712572
0.741771
20
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
66
0.702483
0.753947
0.721245
0.740425
0.714028
0.730941
0.004153
0.000141
0.007731
0.009440
3
0.295314
0.015306
0.717718
0.751043
20
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
63
0.705706
0.763371
0.724351
0.749680
0.723143
0.740078
0.006458
0.000221
0.008524
0.009558
4
0.296860
0.016314
0.694990
0.721001
20
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
67
0.686261
0.732836
0.704874
0.719693
0.693869
0.710475
0.004760
0.000772
0.007642
0.009176
5
0.315843
0.015508
0.717750
0.751042
20
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
62
0.705717
0.763370
0.724479
0.749679
0.723100
0.740078
0.018857
0.000294
0.008544
0.009558
6
0.313852
0.016856
0.671685
0.695180
20
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
69
0.663962
0.706591
0.682232
0.693919
0.668890
0.685031
0.031138
0.001653
0.007717
0.008847
7
0.304668
0.015630
0.717781
0.751041
20
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
61
0.705728
0.763369
0.724606
0.749678
0.723058
0.740077
0.011278
0.000532
0.008563
0.009558
8
0.299836
0.015279
0.646131
0.667594
20
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
70
0.639174
0.678552
0.657004
0.666382
0.642242
0.657847
0.006047
0.000130
0.007782
0.008496
9
0.306051
0.015208
0.717813
0.751040
20
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
60
0.705738
0.763368
0.724732
0.749676
0.723015
0.740076
0.006877
0.000069
0.008583
0.009558
10
0.327514
0.019950
0.765231
0.807077
30
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
49
0.768001
0.806540
0.738767
0.818617
0.788916
0.796075
0.003152
0.003813
0.020554
0.009211
11
0.356175
0.017636
0.765231
0.807077
30
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
50
0.768001
0.806540
0.738766
0.818617
0.788915
0.796075
0.032064
0.001043
0.020554
0.009211
12
0.321720
0.018632
0.761768
0.797113
30
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
52
0.759306
0.796583
0.743733
0.808511
0.782275
0.786247
0.010005
0.002703
0.015821
0.009097
13
0.333622
0.018065
0.765391
0.807074
30
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
48
0.767986
0.806537
0.739271
0.818614
0.788906
0.796072
0.016797
0.001215
0.020333
0.009211
14
0.325341
0.017164
0.744258
0.774794
30
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
56
0.738439
0.774278
0.731997
0.785872
0.762362
0.764232
0.005089
0.000458
0.013057
0.008842
15
0.328952
0.017030
0.765542
0.807066
30
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
47
0.767963
0.806529
0.739763
0.818605
0.788890
0.796063
0.010762
0.000573
0.020116
0.009210
16
0.408038
0.018024
0.720187
0.747047
30
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
59
0.712167
0.746549
0.712008
0.757728
0.736417
0.736863
0.057215
0.002158
0.011465
0.008525
17
0.356321
0.016590
0.765684
0.807052
30
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
46
0.767934
0.806516
0.740242
0.818589
0.788867
0.796049
0.033355
0.000135
0.019902
0.009210
18
0.352246
0.019397
0.693427
0.717402
30
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
68
0.683951
0.716924
0.688210
0.727660
0.708158
0.707622
0.045720
0.002293
0.010551
0.008187
19
0.434964
0.018961
0.765818
0.807032
30
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
45
0.767898
0.806499
0.740709
0.818569
0.788837
0.796029
0.040812
0.001014
0.019691
0.009209
20
0.549796
0.019889
0.805833
0.852250
40
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
36
0.794452
0.860237
0.785728
0.858806
0.837364
0.837706
0.091642
0.001119
0.022557
0.010300
21
0.415316
0.019559
0.805835
0.852251
40
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
35
0.794454
0.860238
0.785727
0.858807
0.837368
0.837708
0.017682
0.000545
0.022559
0.010300
22
0.449368
0.019546
0.801703
0.841727
40
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
38
0.788292
0.849617
0.788918
0.848201
0.827952
0.827363
0.024983
0.001140
0.018544
0.010173
23
0.447652
0.019398
0.806118
0.852215
40
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
34
0.794813
0.860200
0.786401
0.858765
0.837184
0.837681
0.043825
0.001736
0.022213
0.010294
24
0.447779
0.018987
0.782974
0.818165
40
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
42
0.768482
0.825839
0.775141
0.824456
0.805355
0.804200
0.019816
0.000862
0.016043
0.009891
25
0.424559
0.019177
0.806317
0.852112
40
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
33
0.795093
0.860097
0.786958
0.858655
0.836943
0.837583
0.024774
0.000260
0.021889
0.010290
26
0.439985
0.019301
0.757431
0.788859
40
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
53
0.742398
0.796252
0.753072
0.794929
0.776881
0.775396
0.018789
0.000395
0.014415
0.009535
27
0.463641
0.018253
0.806440
0.851959
40
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
32
0.795312
0.859940
0.787420
0.858489
0.836631
0.837449
0.039824
0.000775
0.021570
0.010278
28
0.438939
0.018723
0.729149
0.757555
40
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
58
0.713907
0.764649
0.727262
0.763383
0.746338
0.744632
0.027545
0.000752
0.013311
0.009152
29
0.416260
0.018764
0.806543
0.851767
40
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
31
0.795474
0.859738
0.787805
0.858275
0.836392
0.837288
0.015206
0.000741
0.021317
0.010255
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
40
0.375847
0.020538
0.825369
0.881447
60
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
19
0.822989
0.884271
0.803520
0.886425
0.849608
0.873646
0.009066
0.000211
0.018879
0.005586
41
0.381019
0.023118
0.825369
0.881438
60
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
20
0.823010
0.884259
0.803428
0.886372
0.849678
0.873684
0.006952
0.001493
0.018943
0.005550
42
0.393197
0.023138
0.821582
0.870518
60
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
21
0.815706
0.873316
0.807728
0.875420
0.841334
0.862818
0.029874
0.002078
0.014329
0.005512
43
0.387475
0.027809
0.827144
0.881030
60
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
18
0.822581
0.883842
0.807218
0.885936
0.851651
0.873313
0.029429
0.005375
0.018414
0.005524
44
0.382501
0.024263
0.802851
0.846164
60
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
37
0.794847
0.848871
0.794276
0.850868
0.819462
0.838753
0.008359
0.003479
0.011736
0.005303
45
0.393198
0.020258
0.827730
0.879978
60
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
16
0.821949
0.883153
0.809028
0.884735
0.852235
0.872045
0.028741
0.000070
0.018097
0.005646
46
0.367736
0.020918
0.776828
0.815844
60
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
43
0.767669
0.818522
0.772070
0.820378
0.790782
0.808632
0.008756
0.000573
0.010020
0.005156
47
0.400183
0.024038
0.827801
0.878802
60
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
15
0.820795
0.882106
0.810070
0.883547
0.852565
0.870755
0.023295
0.005042
0.018034
0.005721
48
0.362745
0.021705
0.747887
0.783467
60
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
55
0.737701
0.785955
0.745861
0.787846
0.760137
0.776600
0.000644
0.001943
0.009274
0.004916
49
0.376146
0.020998
0.827606
0.877499
60
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
17
0.819970
0.881134
0.810426
0.882052
0.852451
0.869310
0.016574
0.000764
0.017979
0.005803
50
0.402064
0.024904
0.834492
0.889724
70
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
10
0.833021
0.889985
0.804207
0.899799
0.866253
0.879388
0.012912
0.002065
0.025335
0.008335
51
0.400296
0.022985
0.834382
0.889694
70
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
12
0.832823
0.889941
0.804260
0.899782
0.866068
0.879358
0.027130
0.000950
0.025241
0.008340
52
0.406284
0.024797
0.830841
0.878723
70
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
14
0.825982
0.878963
0.811662
0.888651
0.854899
0.868557
0.030164
0.003596
0.017973
0.008205
53
0.375663
0.026882
0.835885
0.888368
70
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
8
0.831669
0.888989
0.811917
0.897964
0.864085
0.878150
0.004621
0.006672
0.021493
0.008101
54
0.387577
0.022224
0.811736
0.854136
70
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
30
0.804842
0.854363
0.800821
0.863815
0.829570
0.844231
0.010595
0.000702
0.012705
0.007997
55
0.409307
0.022568
0.835371
0.886280
70
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
9
0.829602
0.887433
0.814372
0.895205
0.862162
0.876200
0.020724
0.001054
0.019921
0.007801
56
0.390785
0.022313
0.785504
0.823595
70
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
41
0.777304
0.823738
0.779748
0.832939
0.799491
0.814107
0.009090
0.000411
0.009931
0.007689
57
0.393170
0.023273
0.834445
0.884274
70
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
11
0.827645
0.885921
0.815138
0.892553
0.860579
0.874347
0.020200
0.001178
0.019155
0.007523
58
0.375196
0.026777
0.756183
0.790875
70
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
54
0.747155
0.791110
0.754170
0.799787
0.767259
0.781729
0.004200
0.005763
0.008332
0.007374
59
0.415088
0.022773
0.833386
0.882307
70
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
13
0.825958
0.884434
0.815165
0.890166
0.859063
0.872321
0.031106
0.001419
0.018667
0.007439
60
0.473929
0.024130
0.844443
0.896509
80
0
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
2
0.849029
0.896258
0.817566
0.905128
0.866718
0.888142
0.020667
0.000809
0.020315
0.006937
61
0.475258
0.025253
0.844479
0.896420
80
0
False
{u'ridge__normalize': False, u'ridge__alpha': ...
1
0.849220
0.896059
0.817506
0.905084
0.866692
0.888117
0.032309
0.000883
0.020346
0.006932
62
0.471297
0.025263
0.840073
0.885383
80
0.125
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
5
0.843015
0.885094
0.823460
0.893916
0.853732
0.877140
0.020013
0.000440
0.012525
0.006852
63
0.504618
0.034408
0.843110
0.893525
80
0.125
False
{u'ridge__normalize': False, u'ridge__alpha': ...
3
0.842264
0.893776
0.819384
0.901883
0.867684
0.884915
0.016999
0.009561
0.019715
0.006929
64
0.482595
0.025755
0.820336
0.860619
80
0.25
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
25
0.822191
0.860327
0.810753
0.868913
0.828058
0.852616
0.027897
0.002226
0.007182
0.006657
65
0.471377
0.029691
0.840686
0.890135
80
0.25
False
{u'ridge__normalize': False, u'ridge__alpha': ...
4
0.837470
0.891010
0.819246
0.898053
0.865356
0.881341
0.004810
0.006196
0.018950
0.006851
66
0.481739
0.025373
0.793495
0.829778
80
0.375
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
40
0.794759
0.829500
0.788563
0.837729
0.797157
0.822105
0.010791
0.001301
0.003619
0.006381
67
0.483793
0.026728
0.838527
0.887276
80
0.375
False
{u'ridge__normalize': False, u'ridge__alpha': ...
6
0.833707
0.888668
0.818671
0.894823
0.863221
0.878337
0.029412
0.003457
0.018494
0.006802
68
0.492897
0.025333
0.763966
0.796841
80
0.5
True
{u'ridge__normalize': True, u'ridge__alpha': 0...
51
0.764461
0.796529
0.762762
0.804536
0.764673
0.789457
0.025573
0.001033
0.000855
0.006160
69
0.471107
0.028771
0.836787
0.884831
80
0.5
False
{u'ridge__normalize': False, u'ridge__alpha': ...
7
0.830931
0.886657
0.818017
0.892051
0.861436
0.875785
0.018583
0.003879
0.018194
0.006765
70 rows × 19 columns
Completed in -1093.29 sec
In [91]:
print len(y_pipelines_ridge)
print y_scores_ridge
5
[0.16877420396826684, 1.0734344300231238, 0.16403396939374118, 0.14144016224282882, 0.11547904139250162]
In [92]:
# SVR with PCA combinations
y_pipelines_svr = []
y_scores_svr = []
start = time.time()
for ind, y in enumerate(y_vars):
X_train, X_test, y_train, y_test = train_test_split(X_wDepth, y, test_size=0.33, random_state=42)
# set up the train and test data
print '\n----------', y_var_labels[ind]
pca = PCA()
svr = SVR()
steps = [('pca', pca), ('svr', svr)]
pipeline = Pipeline(steps)
parameters = dict(pca__n_components=list(range(20, 90, 10)),
svr__kernel=list(['rbf']),
svr__C=np.logspace(-2, 10, 13))
#svr__C=list([1e3]))
cv = GridSearchCV(pipeline, param_grid=parameters, verbose=0)
cv.fit(X_train, y_train)
print 'Cross_val_score: ', cross_val_score(cv, X_test, y_test)
y_predictions = cv.predict(X_test)
mse = mean_squared_error(y_test, y_predictions)
print 'Explained variance score: ', explained_variance_score(y_test, y_predictions)
print 'Mean absolute error: ', mean_absolute_error(y_test, y_predictions)
print 'Mean squared error: ', mse
print 'R2 score: ', r2_score(y_test, y_predictions)
display(pd.DataFrame.from_dict(cv.cv_results_))
# capture the best pipeline estimator and mse value
y_pipelines_svr.append(cv.best_estimator_)
y_scores_svr.append(mse)
print 'Completed in %0.2f sec' % (start-time.time())
---------- Ca
Cross_val_score: [ 0.86622329 0.72206497 0.87348425]
Explained variance score: 0.9083982785
Mean absolute error: 0.165606370284
Mean squared error: 0.156309305333
R2 score: 0.908306038347
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_svr__C
param_svr__kernel
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.449781
0.025146
-0.024150
-0.017155
20
0.01
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
91
-0.010265
-0.027662
-0.039275
-0.007873
-0.022962
-0.015929
0.098105
0.002965
0.011877
0.008125
1
0.329333
0.020562
0.231078
0.262848
20
0.1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
84
0.330549
0.221540
0.212867
0.239478
0.149433
0.327525
0.014554
0.000893
0.075074
0.046316
2
0.426314
0.024077
0.664136
0.762535
20
1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
77
0.865818
0.698649
0.654940
0.701448
0.470869
0.887508
0.155428
0.001590
0.161420
0.088377
3
0.768561
0.038767
0.770284
0.990535
20
10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
61
0.868729
0.992471
0.851927
0.990182
0.589813
0.988952
0.066672
0.004003
0.127673
0.001458
4
0.769867
0.032634
0.766036
0.991531
20
100
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
68
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.130239
0.002767
0.124904
0.001837
5
0.360069
0.020572
0.766039
0.991532
20
1000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
62
0.866430
0.993091
0.841483
0.992552
0.589813
0.988952
0.074717
0.000777
0.124906
0.001838
6
0.458154
0.038091
0.766036
0.991531
20
10000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
70
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.100855
0.023466
0.124904
0.001837
7
0.416961
0.021555
0.766036
0.991531
20
100000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
66
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.099268
0.001304
0.124904
0.001837
8
0.360876
0.023488
0.766036
0.991531
20
1e+06
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
65
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.037957
0.004783
0.124904
0.001837
9
0.386401
0.025548
0.766036
0.991531
20
1e+07
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
63
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.042221
0.006265
0.124904
0.001837
10
0.343017
0.020715
0.766036
0.991531
20
1e+08
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
67
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.023642
0.000999
0.124904
0.001837
11
0.332389
0.026842
0.766036
0.991531
20
1e+09
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
69
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.028423
0.008576
0.124904
0.001837
12
0.329450
0.020719
0.766036
0.991531
20
1e+10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
64
0.866424
0.993089
0.841483
0.992552
0.589813
0.988952
0.018440
0.000485
0.124904
0.001837
13
0.406023
0.029439
0.005118
0.009112
30
0.01
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
90
0.028027
-0.005216
-0.008096
0.014912
-0.004667
0.017641
0.068041
0.004770
0.016291
0.010193
14
0.325937
0.024519
0.302376
0.320126
30
0.1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
83
0.418871
0.272972
0.290493
0.292326
0.197311
0.395079
0.012789
0.001730
0.090869
0.053586
15
0.374772
0.023190
0.701433
0.772824
30
1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
76
0.884512
0.716919
0.707968
0.712173
0.511109
0.889378
0.080159
0.000971
0.152560
0.082439
16
0.386019
0.024238
0.809492
0.990014
30
10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
51
0.888863
0.991961
0.892312
0.988416
0.646993
0.989665
0.061120
0.001473
0.114801
0.001468
17
0.328993
0.022570
0.800224
0.992026
30
100
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
53
0.880930
0.993555
0.872430
0.992858
0.646998
0.989665
0.009523
0.000132
0.108297
0.001693
18
0.383228
0.026877
0.800218
0.992025
30
1000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
60
0.880930
0.993555
0.872418
0.992857
0.646994
0.989664
0.040282
0.005263
0.108297
0.001694
19
0.332045
0.022809
0.800224
0.992026
30
10000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
52
0.880930
0.993555
0.872435
0.992858
0.646994
0.989664
0.012007
0.000351
0.108301
0.001694
20
0.427650
0.026303
0.800218
0.992025
30
100000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
59
0.880930
0.993555
0.872418
0.992857
0.646994
0.989664
0.072877
0.003064
0.108297
0.001694
21
0.554558
0.027532
0.800218
0.992025
30
1e+06
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
58
0.880930
0.993555
0.872418
0.992857
0.646994
0.989664
0.038212
0.001810
0.108297
0.001694
22
0.475274
0.065564
0.800222
0.992026
30
1e+07
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
55
0.880930
0.993555
0.872430
0.992858
0.646993
0.989665
0.063567
0.044741
0.108300
0.001694
23
0.414809
0.023576
0.800218
0.992025
30
1e+08
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
57
0.880930
0.993555
0.872418
0.992857
0.646994
0.989664
0.129255
0.001615
0.108297
0.001694
24
0.588993
0.033377
0.800218
0.992025
30
1e+09
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
56
0.880930
0.993555
0.872418
0.992857
0.646994
0.989664
0.109186
0.009391
0.108297
0.001694
25
0.432592
0.031750
0.800222
0.992025
30
1e+10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
54
0.880930
0.993555
0.872435
0.992858
0.646990
0.989664
0.066300
0.006126
0.108303
0.001694
26
0.587252
0.030975
0.026361
0.028201
40
0.01
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
89
0.055097
0.011148
0.014695
0.031460
0.009181
0.041996
0.088539
0.001021
0.020482
0.012803
27
0.478530
0.025496
0.335628
0.345434
40
0.1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
82
0.461361
0.298345
0.323563
0.310699
0.221471
0.427259
0.092089
0.001580
0.098336
0.058078
28
0.518283
0.027771
0.720265
0.775820
40
1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
75
0.889700
0.724431
0.734778
0.717772
0.535660
0.885256
0.047062
0.003854
0.144946
0.077431
29
0.444665
0.029398
0.837239
0.989055
40
10
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
32
0.900356
0.990894
0.915473
0.986346
0.695643
0.989925
0.062767
0.003170
0.100217
0.001956
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
61
0.457089
0.040063
0.841678
0.992491
60
1e+07
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
25
0.894220
0.993719
0.871674
0.993417
0.758936
0.990338
0.052010
0.002676
0.059171
0.001528
62
0.464652
0.040826
0.841669
0.992491
60
1e+08
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
29
0.894214
0.993720
0.871666
0.993415
0.758925
0.990338
0.041436
0.007822
0.059174
0.001528
63
0.498757
0.034991
0.841666
0.992490
60
1e+09
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
31
0.894224
0.993720
0.871654
0.993416
0.758917
0.990335
0.089284
0.008300
0.059178
0.001529
64
0.451944
0.032297
0.841666
0.992492
60
1e+10
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
30
0.894209
0.993719
0.871664
0.993416
0.758923
0.990340
0.024328
0.002463
0.059172
0.001526
65
0.527143
0.049074
0.054448
0.052737
70
0.01
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
86
0.090219
0.033598
0.046621
0.054297
0.026364
0.070315
0.100247
0.010120
0.026657
0.015030
66
0.365749
0.033111
0.361435
0.361318
70
0.1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
79
0.483447
0.315699
0.356956
0.321361
0.243428
0.446895
0.016584
0.001537
0.098070
0.060556
67
0.381295
0.031279
0.726007
0.760877
70
1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
73
0.886596
0.714763
0.735198
0.703502
0.555606
0.864366
0.016502
0.001405
0.135325
0.073322
68
0.390800
0.031391
0.854853
0.979653
70
10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
2
0.913900
0.983100
0.918446
0.978686
0.731984
0.977174
0.027616
0.000687
0.086817
0.002514
69
0.421953
0.033184
0.851440
0.992138
70
100
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
5
0.894220
0.993715
0.880895
0.992050
0.779038
0.990648
0.033419
0.001275
0.051435
0.001254
70
0.406818
0.032247
0.846870
0.992561
70
1000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
19
0.894201
0.993711
0.867161
0.993321
0.779063
0.990650
0.028021
0.000831
0.049157
0.001360
71
0.407044
0.032708
0.846873
0.992561
70
10000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
17
0.894213
0.993712
0.867165
0.993319
0.779059
0.990651
0.013945
0.001491
0.049163
0.001360
72
0.394811
0.033608
0.846883
0.992562
70
100000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
15
0.894207
0.993715
0.867205
0.993320
0.779054
0.990652
0.014050
0.001264
0.049169
0.001361
73
0.412599
0.032063
0.846873
0.992563
70
1e+06
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
18
0.894208
0.993715
0.867179
0.993323
0.779047
0.990650
0.018241
0.000630
0.049168
0.001362
74
0.397391
0.033822
0.846876
0.992560
70
1e+07
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
16
0.894210
0.993713
0.867185
0.993319
0.779050
0.990646
0.007527
0.001024
0.049168
0.001362
75
0.395845
0.031706
0.846856
0.992563
70
1e+08
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
22
0.894213
0.993717
0.867116
0.993319
0.779057
0.990655
0.005385
0.000578
0.049157
0.001359
76
0.398313
0.032051
0.846859
0.992561
70
1e+09
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
21
0.894194
0.993715
0.867145
0.993316
0.779052
0.990653
0.026527
0.000677
0.049157
0.001359
77
0.400062
0.033171
0.846867
0.992563
70
1e+10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
20
0.894203
0.993717
0.867172
0.993321
0.779042
0.990651
0.010594
0.000806
0.049168
0.001361
78
0.464866
0.044034
0.058858
0.056428
80
0.01
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
85
0.096713
0.037869
0.050620
0.057140
0.029094
0.074276
0.025250
0.007226
0.028221
0.014872
79
0.435877
0.037441
0.362230
0.360391
80
0.1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
78
0.482796
0.315803
0.358652
0.319467
0.244774
0.445904
0.024011
0.003308
0.097236
0.060485
80
0.440991
0.033068
0.723698
0.753991
80
1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
74
0.885473
0.708309
0.731492
0.697748
0.553500
0.855917
0.002031
0.000224
0.135683
0.072201
81
0.471085
0.033816
0.853080
0.975501
80
10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
4
0.915689
0.980536
0.914918
0.976703
0.728391
0.969265
0.006770
0.000128
0.088084
0.004679
82
0.470255
0.040069
0.857951
0.991849
80
100
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
1
0.897413
0.993460
0.881619
0.991175
0.794669
0.990911
0.020433
0.006905
0.045167
0.001145
83
0.481353
0.035756
0.850204
0.992627
80
1000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
12
0.893551
0.993662
0.862217
0.993313
0.794676
0.990905
0.029912
0.000424
0.041261
0.001226
84
0.470426
0.034817
0.850202
0.992625
80
10000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
13
0.893557
0.993660
0.862203
0.993308
0.794678
0.990907
0.014386
0.000505
0.041261
0.001223
85
0.460048
0.035143
0.850211
0.992627
80
100000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
10
0.893569
0.993661
0.862217
0.993309
0.794680
0.990910
0.003397
0.000581
0.041265
0.001222
86
0.466133
0.037537
0.850192
0.992627
80
1e+06
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
14
0.893547
0.993660
0.862194
0.993309
0.794667
0.990913
0.010164
0.002900
0.041261
0.001221
87
0.541402
0.037421
0.850216
0.992625
80
1e+07
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
7
0.893576
0.993660
0.862223
0.993307
0.794680
0.990907
0.095271
0.000846
0.041268
0.001223
88
0.535644
0.036288
0.850204
0.992629
80
1e+08
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
11
0.893549
0.993662
0.862206
0.993312
0.794689
0.990913
0.097639
0.001476
0.041253
0.001222
89
0.475289
0.035126
0.850212
0.992627
80
1e+09
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
9
0.893564
0.993662
0.862225
0.993309
0.794680
0.990910
0.014460
0.000879
0.041264
0.001223
90
0.493106
0.036588
0.850216
0.992624
80
1e+10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
8
0.893583
0.993662
0.862206
0.993308
0.794689
0.990903
0.009631
0.001994
0.041265
0.001226
91 rows × 19 columns
---------- P
Cross_val_score: [ 0.14201859 0.29246724 0.37196137]
Explained variance score: 0.0381591115853
Mean absolute error: 0.407893856659
Mean squared error: 1.17043179738
R2 score: 0.0365333451692
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_svr__C
param_svr__kernel
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.354222
0.019937
-0.032051
-0.026674
20
0.01
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
46
-0.034373
-0.024538
-0.038577
-0.025434
-0.023195
-0.030050
0.054165
0.000204
0.006488
0.002415
1
0.296311
0.021351
0.019008
0.046164
20
0.1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
24
0.001393
0.058063
-0.001247
0.043725
0.056948
0.036703
0.010814
0.001829
0.026823
0.008889
2
0.308382
0.020468
0.126979
0.294617
20
1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
8
0.085738
0.352303
0.091328
0.267288
0.204032
0.264261
0.007972
0.000124
0.054480
0.040809
3
0.321949
0.022447
0.193295
0.872913
20
10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
3
0.152411
0.905848
0.123982
0.863215
0.303650
0.849675
0.010492
0.000761
0.078817
0.023936
4
0.319350
0.022057
0.106994
0.990436
20
100
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
14
0.128645
0.989468
0.044847
0.989379
0.147406
0.992462
0.002023
0.000374
0.044566
0.001433
5
0.322749
0.022561
0.107005
0.990437
20
1000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
10
0.128657
0.989471
0.044808
0.989380
0.147465
0.992460
0.008952
0.001207
0.044603
0.001431
6
0.334296
0.022216
0.106988
0.990438
20
10000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
18
0.128659
0.989468
0.044814
0.989382
0.147406
0.992462
0.026996
0.001220
0.044583
0.001432
7
0.405025
0.030052
0.106989
0.990438
20
100000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
16
0.128657
0.989469
0.044821
0.989382
0.147406
0.992462
0.051148
0.004054
0.044579
0.001432
8
0.333364
0.021614
0.106997
0.990436
20
1e+06
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
12
0.128657
0.989471
0.044847
0.989379
0.147405
0.992460
0.012117
0.000130
0.044567
0.001431
9
0.339214
0.022011
0.106990
0.990437
20
1e+07
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
15
0.128659
0.989468
0.044821
0.989382
0.147406
0.992462
0.016049
0.000310
0.044580
0.001432
10
0.321303
0.021526
0.106996
0.990437
20
1e+08
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
13
0.128659
0.989468
0.044808
0.989380
0.147438
0.992462
0.010607
0.000244
0.044596
0.001433
11
0.320654
0.022453
0.106999
0.990436
20
1e+09
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
11
0.128659
0.989468
0.044847
0.989379
0.147406
0.992462
0.007802
0.001020
0.044568
0.001433
12
0.318098
0.022260
0.106988
0.990435
20
1e+10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
17
0.128659
0.989468
0.044817
0.989378
0.147405
0.992460
0.009139
0.000844
0.044582
0.001432
13
0.322525
0.024039
-0.029977
-0.025399
30
0.01
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
44
-0.032428
-0.023136
-0.037590
-0.024576
-0.019905
-0.028485
0.013170
0.000934
0.007421
0.002260
14
0.327484
0.022565
0.018678
0.038496
30
0.1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
25
0.000508
0.049117
-0.001221
0.036880
0.056816
0.029492
0.024560
0.000173
0.026951
0.008093
15
0.323009
0.023812
0.115011
0.237507
30
1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
9
0.075685
0.294466
0.076663
0.207312
0.192836
0.210744
0.013997
0.000257
0.054979
0.040300
16
0.340983
0.025485
0.201219
0.772120
30
10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
1
0.145304
0.837766
0.138166
0.730180
0.320402
0.748414
0.010919
0.000624
0.084244
0.047012
17
0.371504
0.025538
0.013455
0.990340
30
100
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
27
0.098744
0.989490
-0.047207
0.989228
-0.011501
0.992301
0.030696
0.000654
0.062156
0.001391
18
0.360124
0.025247
0.013436
0.990340
30
1000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
31
0.098764
0.989489
-0.047237
0.989228
-0.011549
0.992302
0.006418
0.000245
0.062182
0.001392
19
0.417541
0.027973
0.013431
0.990340
30
10000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
32
0.098761
0.989491
-0.047217
0.989227
-0.011581
0.992300
0.060589
0.002497
0.062178
0.001391
20
0.381538
0.026328
0.013440
0.990342
30
100000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
29
0.098774
0.989494
-0.047214
0.989228
-0.011571
0.992302
0.010520
0.001201
0.062182
0.001391
21
0.415525
0.028724
0.013426
0.990342
30
1e+06
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
33
0.098756
0.989492
-0.047229
0.989233
-0.011581
0.992300
0.050314
0.002889
0.062179
0.001389
22
0.402848
0.026831
0.013439
0.990341
30
1e+07
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
30
0.098753
0.989495
-0.047242
0.989226
-0.011526
0.992303
0.023463
0.002502
0.062175
0.001392
23
0.377334
0.027003
0.013423
0.990341
30
1e+08
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
34
0.098734
0.989493
-0.047236
0.989230
-0.011560
0.992301
0.020550
0.002748
0.062169
0.001390
24
0.412278
0.028012
0.013445
0.990341
30
1e+09
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
28
0.098755
0.989491
-0.047220
0.989230
-0.011531
0.992302
0.006899
0.001645
0.062169
0.001391
25
0.350423
0.026298
0.013418
0.990341
30
1e+10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
35
0.098755
0.989491
-0.047230
0.989231
-0.011602
0.992301
0.009187
0.001538
0.062182
0.001390
26
0.393824
0.025812
-0.029272
-0.025247
40
0.01
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
40
-0.031231
-0.023163
-0.037017
-0.024186
-0.019560
-0.028394
0.007482
0.000690
0.007256
0.002263
27
0.389406
0.025173
0.016289
0.031952
40
0.1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
26
-0.001172
0.042439
-0.003099
0.030162
0.053206
0.023255
0.007453
0.000387
0.026091
0.007934
28
0.395878
0.027135
0.102083
0.196521
40
1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
19
0.064748
0.247750
0.063694
0.168021
0.177953
0.173791
0.014663
0.001640
0.053598
0.036301
29
0.424444
0.027525
0.198381
0.664516
40
10
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
2
0.137054
0.753218
0.135425
0.600677
0.322903
0.639654
0.012611
0.000482
0.087967
0.064709
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
61
0.508680
0.037455
-0.190814
0.989844
60
1e+07
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
71
0.012051
0.988714
-0.193018
0.988729
-0.392263
0.992089
0.010386
0.001379
0.165121
0.001587
62
0.513284
0.039452
-0.190780
0.989843
60
1e+08
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
68
0.012057
0.988712
-0.193033
0.988729
-0.392152
0.992088
0.031457
0.000380
0.165078
0.001587
63
0.513813
0.037411
-0.190890
0.989844
60
1e+09
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
75
0.011995
0.988715
-0.193119
0.988728
-0.392332
0.992089
0.041080
0.000965
0.165126
0.001588
64
0.529311
0.039057
-0.190834
0.989842
60
1e+10
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
72
0.012039
0.988712
-0.192982
0.988728
-0.392346
0.992086
0.041090
0.002011
0.165149
0.001587
65
0.399101
0.037425
-0.029870
-0.026848
70
0.01
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
43
-0.031567
-0.025800
-0.037350
-0.025824
-0.020688
-0.028919
0.003334
0.001181
0.006903
0.001465
66
0.463459
0.034249
0.007286
0.017985
70
0.1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
38
-0.005424
0.026559
-0.011135
0.016499
0.038465
0.010895
0.052533
0.001114
0.022149
0.006480
67
0.398980
0.033689
0.070748
0.128397
70
1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
22
0.045835
0.170077
0.034401
0.111012
0.132104
0.104101
0.016540
0.000828
0.043594
0.029607
68
0.447780
0.040220
0.161298
0.439841
70
10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
6
0.112965
0.542449
0.105275
0.374492
0.265841
0.402583
0.014509
0.003180
0.073918
0.073455
69
0.536350
0.046936
-0.101452
0.943558
70
100
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
57
0.031757
0.984303
-0.076097
0.912878
-0.260530
0.933492
0.014797
0.005422
0.120700
0.030015
70
0.561493
0.053595
-0.233272
0.989760
70
1000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
80
-0.012347
0.988717
-0.209033
0.988564
-0.479293
0.992000
0.013825
0.008176
0.191458
0.001585
71
0.574503
0.043541
-0.233305
0.989759
70
10000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
82
-0.012404
0.988712
-0.209061
0.988565
-0.479306
0.992001
0.010887
0.001749
0.191441
0.001586
72
0.600882
0.052374
-0.233340
0.989759
70
100000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
83
-0.012405
0.988713
-0.209091
0.988566
-0.479380
0.991998
0.032875
0.013567
0.191470
0.001584
73
0.583812
0.042339
-0.233262
0.989760
70
1e+06
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
78
-0.012427
0.988715
-0.209021
0.988563
-0.479194
0.992000
0.002412
0.001995
0.191386
0.001586
74
0.693127
0.047648
-0.233276
0.989759
70
1e+07
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
81
-0.012387
0.988715
-0.209043
0.988563
-0.479255
0.992000
0.052228
0.002101
0.191426
0.001585
75
0.687408
0.043708
-0.233246
0.989760
70
1e+08
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
77
-0.012363
0.988716
-0.209046
0.988566
-0.479185
0.991998
0.057697
0.001977
0.191406
0.001584
76
0.593573
0.044166
-0.233240
0.989761
70
1e+09
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
76
-0.012371
0.988716
-0.209074
0.988564
-0.479132
0.992002
0.034252
0.001150
0.191378
0.001586
77
0.585067
0.049986
-0.233271
0.989759
70
1e+10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
79
-0.012383
0.988715
-0.209093
0.988563
-0.479194
0.991999
0.015558
0.010700
0.191399
0.001585
78
0.490924
0.041961
-0.030227
-0.027433
80
0.01
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
45
-0.031882
-0.026821
-0.037681
-0.026424
-0.021113
-0.029054
0.037404
0.004109
0.006861
0.001158
79
0.452803
0.037656
0.005173
0.015149
80
0.1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
39
-0.005809
0.023934
-0.013093
0.013139
0.034464
0.008374
0.011028
0.002210
0.020904
0.006509
80
0.465698
0.038230
0.063723
0.115678
80
1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
23
0.041234
0.154065
0.026844
0.100478
0.123177
0.092490
0.020177
0.002723
0.042409
0.027339
81
0.496502
0.039512
0.149804
0.390933
80
10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
7
0.106052
0.489331
0.093944
0.332209
0.249584
0.351259
0.012804
0.001133
0.070661
0.070011
82
0.602596
0.044552
-0.075774
0.914773
80
100
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
47
0.022005
0.979421
-0.037547
0.864486
-0.212160
0.900413
0.023453
0.001302
0.099368
0.048008
83
0.655959
0.049740
-0.264639
0.989699
80
1000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
86
-0.032591
0.988635
-0.218602
0.988509
-0.543624
0.991954
0.010911
0.007762
0.211214
0.001595
84
0.654982
0.044233
-0.264641
0.989697
80
10000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
87
-0.032555
0.988634
-0.218630
0.988507
-0.543639
0.991952
0.035343
0.000435
0.211232
0.001595
85
0.648038
0.046814
-0.264619
0.989699
80
100000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
84
-0.032586
0.988634
-0.218645
0.988509
-0.543526
0.991953
0.022323
0.000769
0.211169
0.001595
86
0.668215
0.044960
-0.264708
0.989699
80
1e+06
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
90
-0.032619
0.988636
-0.218631
0.988509
-0.543773
0.991952
0.029095
0.001092
0.211267
0.001594
87
0.651821
0.046381
-0.264623
0.989698
80
1e+07
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
85
-0.032526
0.988634
-0.218602
0.988509
-0.543642
0.991951
0.027850
0.001784
0.211246
0.001594
88
0.680639
0.055185
-0.264653
0.989699
80
1e+08
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
89
-0.032591
0.988636
-0.218607
0.988509
-0.543661
0.991953
0.007565
0.010534
0.211230
0.001594
89
0.699670
0.048721
-0.264714
0.989698
80
1e+09
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
91
-0.032580
0.988636
-0.218622
0.988507
-0.543842
0.991953
0.054026
0.001109
0.211312
0.001595
90
0.683681
0.047485
-0.264644
0.989699
80
1e+10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
88
-0.032582
0.988633
-0.218594
0.988509
-0.543656
0.991953
0.013902
0.001278
0.211232
0.001595
91 rows × 19 columns
---------- pH
Cross_val_score: [ 0.66677802 0.70956204 0.69330498]
Explained variance score: 0.832716738855
Mean absolute error: 0.280477428867
Mean squared error: 0.147325879393
R2 score: 0.832710214588
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_svr__C
param_svr__kernel
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.325536
0.023699
0.034312
0.051760
20
0.01
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
91
0.047078
0.045019
0.002992
0.055638
0.052818
0.054624
0.011977
0.000658
0.022249
0.004785
1
0.297957
0.023034
0.376048
0.465049
20
0.1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
84
0.410919
0.447522
0.344460
0.481245
0.372629
0.466378
0.007063
0.000147
0.027247
0.013800
2
0.315254
0.024703
0.674016
0.899997
20
1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
77
0.696944
0.900661
0.654393
0.896538
0.670622
0.902792
0.003774
0.001980
0.017542
0.002596
3
0.331280
0.024027
0.679452
0.987554
20
10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
67
0.699907
0.989307
0.664586
0.987346
0.673784
0.986008
0.007953
0.001895
0.014970
0.001355
4
0.329431
0.023347
0.675438
0.989293
20
100
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
73
0.697889
0.989960
0.658378
0.989248
0.669959
0.988670
0.003280
0.000487
0.016593
0.000528
5
0.330428
0.023254
0.675444
0.989293
20
1000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
68
0.697889
0.989960
0.658398
0.989247
0.669959
0.988670
0.009945
0.001143
0.016586
0.000528
6
0.328780
0.023203
0.675428
0.989292
20
10000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
76
0.697858
0.989961
0.658389
0.989250
0.669951
0.988667
0.008254
0.000764
0.016576
0.000529
7
0.326396
0.022721
0.675444
0.989291
20
100000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
69
0.697889
0.989960
0.658405
0.989247
0.669951
0.988667
0.003455
0.000059
0.016585
0.000529
8
0.347085
0.025404
0.675443
0.989292
20
1e+06
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
70
0.697889
0.989960
0.658400
0.989248
0.669951
0.988667
0.012221
0.001943
0.016587
0.000529
9
0.336179
0.023550
0.675442
0.989292
20
1e+07
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
71
0.697889
0.989960
0.658398
0.989248
0.669951
0.988667
0.010502
0.000863
0.016587
0.000529
10
0.332996
0.022592
0.675436
0.989291
20
1e+08
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
74
0.697866
0.989960
0.658405
0.989247
0.669951
0.988667
0.008731
0.000075
0.016574
0.000529
11
0.331645
0.022652
0.675434
0.989291
20
1e+09
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
75
0.697858
0.989961
0.658405
0.989247
0.669951
0.988667
0.007642
0.000140
0.016571
0.000529
12
0.339117
0.022742
0.675438
0.989292
20
1e+10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
72
0.697889
0.989960
0.658386
0.989248
0.669951
0.988667
0.005697
0.000199
0.016591
0.000529
13
0.320177
0.027259
0.056859
0.072195
30
0.01
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
90
0.069535
0.062375
0.025275
0.076786
0.075719
0.077423
0.006647
0.000346
0.022454
0.006949
14
0.310094
0.027046
0.458634
0.527993
30
0.1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
83
0.501273
0.520060
0.427115
0.538896
0.447349
0.525024
0.004979
0.000768
0.031316
0.007971
15
0.331366
0.027061
0.708373
0.882041
30
1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
66
0.724132
0.883203
0.692703
0.877751
0.708224
0.885170
0.006199
0.001232
0.012835
0.003139
16
0.349154
0.025942
0.718428
0.983454
30
10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
55
0.735494
0.985734
0.713308
0.982463
0.706417
0.982166
0.003747
0.000122
0.012413
0.001616
17
0.364490
0.026050
0.708959
0.989430
30
100
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
61
0.725077
0.990044
0.700408
0.989387
0.701330
0.988859
0.015407
0.000073
0.011425
0.000485
18
0.357876
0.026356
0.708948
0.989429
30
1000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
65
0.725055
0.990045
0.700395
0.989381
0.701332
0.988862
0.009668
0.000246
0.011418
0.000484
19
0.369852
0.027280
0.708952
0.989430
30
10000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
63
0.725090
0.990044
0.700371
0.989386
0.701334
0.988858
0.009112
0.001635
0.011440
0.000485
20
0.363062
0.026721
0.708962
0.989429
30
100000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
59
0.725071
0.990044
0.700405
0.989387
0.701348
0.988856
0.012006
0.000965
0.011419
0.000486
21
0.361150
0.026767
0.708949
0.989430
30
1e+06
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
64
0.725069
0.990043
0.700389
0.989385
0.701327
0.988862
0.009734
0.000889
0.011427
0.000483
22
0.362218
0.026533
0.708968
0.989430
30
1e+07
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
57
0.725106
0.990045
0.700389
0.989385
0.701346
0.988860
0.009049
0.000417
0.011440
0.000485
23
0.357418
0.026151
0.708954
0.989430
30
1e+08
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
62
0.725059
0.990045
0.700390
0.989388
0.701352
0.988859
0.004240
0.000144
0.011416
0.000485
24
0.357830
0.026659
0.708965
0.989429
30
1e+09
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
58
0.725086
0.990046
0.700414
0.989383
0.701331
0.988859
0.005238
0.000900
0.011428
0.000486
25
0.376967
0.026322
0.708960
0.989429
30
1e+10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
60
0.725075
0.990045
0.700420
0.989385
0.701323
0.988858
0.018786
0.000309
0.011423
0.000485
26
0.398203
0.032272
0.069827
0.083487
40
0.01
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
89
0.083343
0.073051
0.037961
0.088751
0.088126
0.088660
0.005691
0.000591
0.022596
0.007380
27
0.401955
0.030492
0.500445
0.556407
40
0.1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
82
0.543169
0.549250
0.472395
0.565808
0.485607
0.554163
0.011324
0.000200
0.030745
0.006943
28
0.405975
0.030148
0.716440
0.863251
40
1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
56
0.730807
0.865124
0.700409
0.859206
0.718049
0.865423
0.007055
0.000708
0.012466
0.002863
29
0.442860
0.030587
0.738134
0.977683
40
10
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
6
0.749711
0.979462
0.742183
0.975889
0.722462
0.977699
0.007683
0.000978
0.011490
0.001459
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
61
0.484239
0.036972
0.727004
0.989665
60
1e+07
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
25
0.739166
0.990114
0.730197
0.989740
0.711602
0.989141
0.014673
0.000659
0.011480
0.000400
62
0.482657
0.037688
0.727002
0.989665
60
1e+08
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
26
0.739169
0.990114
0.730183
0.989740
0.711606
0.989142
0.014899
0.001280
0.011478
0.000400
63
0.480583
0.036072
0.726979
0.989664
60
1e+09
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
32
0.739179
0.990114
0.730131
0.989736
0.711579
0.989143
0.004900
0.000353
0.011489
0.000400
64
0.485483
0.041061
0.726990
0.989665
60
1e+10
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
27
0.739186
0.990113
0.730156
0.989740
0.711579
0.989142
0.006909
0.004774
0.011494
0.000400
65
0.406178
0.043330
0.082940
0.093122
70
0.01
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
85
0.097461
0.083518
0.052663
0.099902
0.098640
0.095948
0.008978
0.000521
0.021394
0.006981
66
0.388879
0.043011
0.533567
0.572473
70
0.1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
79
0.577929
0.574691
0.505892
0.573878
0.516708
0.568851
0.020736
0.000590
0.031737
0.002583
67
0.401299
0.041219
0.722975
0.827280
70
1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
43
0.730987
0.830676
0.715166
0.823851
0.722740
0.827313
0.012160
0.000269
0.006463
0.002787
68
0.443129
0.042971
0.761873
0.951962
70
10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
2
0.766650
0.954344
0.781170
0.948489
0.737781
0.953052
0.016038
0.003949
0.018023
0.002512
69
0.662470
0.044232
0.735028
0.988977
70
100
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
7
0.740689
0.990238
0.739831
0.988718
0.724541
0.987976
0.110207
0.004066
0.007416
0.000941
70
0.549777
0.041296
0.728856
0.989688
70
1000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
17
0.740367
0.990246
0.729763
0.989594
0.716393
0.989224
0.010405
0.000922
0.009811
0.000422
71
0.586559
0.040446
0.728836
0.989687
70
10000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
23
0.740333
0.990244
0.729755
0.989593
0.716375
0.989223
0.098239
0.000362
0.009805
0.000422
72
0.531215
0.040628
0.728828
0.989688
70
100000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
24
0.740355
0.990245
0.729711
0.989594
0.716375
0.989226
0.012202
0.000935
0.009813
0.000421
73
0.523158
0.041528
0.728838
0.989686
70
1e+06
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
21
0.740354
0.990245
0.729745
0.989592
0.716370
0.989223
0.006133
0.002136
0.009816
0.000423
74
0.523583
0.040495
0.728838
0.989688
70
1e+07
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
20
0.740368
0.990245
0.729744
0.989595
0.716359
0.989224
0.004200
0.000424
0.009826
0.000422
75
0.524846
0.039799
0.728847
0.989688
70
1e+08
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
19
0.740349
0.990247
0.729774
0.989595
0.716374
0.989223
0.008108
0.000164
0.009813
0.000423
76
0.523817
0.040203
0.728850
0.989688
70
1e+09
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
18
0.740368
0.990247
0.729762
0.989595
0.716375
0.989223
0.002055
0.000233
0.009820
0.000423
77
0.520545
0.039883
0.728836
0.989688
70
1e+10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
22
0.740338
0.990244
0.729743
0.989596
0.716382
0.989223
0.015241
0.000220
0.009804
0.000422
78
0.468656
0.047169
0.082905
0.092380
80
0.01
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
86
0.097253
0.082772
0.053091
0.099330
0.098315
0.095037
0.010465
0.000752
0.021065
0.007016
79
0.455236
0.047223
0.534448
0.569791
80
0.1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
78
0.577160
0.572625
0.507658
0.570669
0.518361
0.566079
0.007166
0.001854
0.030574
0.002743
80
0.477836
0.046769
0.723462
0.819292
80
1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
34
0.730115
0.822438
0.717304
0.815935
0.722942
0.819505
0.027159
0.002792
0.005245
0.002659
81
0.512797
0.045299
0.765505
0.943641
80
10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
1
0.766871
0.946373
0.787053
0.939682
0.742586
0.944869
0.019969
0.000884
0.018168
0.002866
82
0.612319
0.043062
0.741515
0.988309
80
100
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
5
0.748275
0.989678
0.743984
0.988297
0.732261
0.986953
0.021502
0.000168
0.006768
0.001113
83
0.622979
0.046266
0.731351
0.989712
80
1000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
11
0.740623
0.990272
0.729589
0.989603
0.723806
0.989261
0.018811
0.002076
0.006980
0.000420
84
0.622763
0.045267
0.731341
0.989712
80
10000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
14
0.740632
0.990270
0.729555
0.989603
0.723801
0.989264
0.000432
0.001862
0.006988
0.000418
85
0.617589
0.044222
0.731338
0.989713
80
100000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
15
0.740603
0.990272
0.729561
0.989602
0.723815
0.989265
0.015590
0.001114
0.006970
0.000418
86
0.613862
0.043296
0.731350
0.989712
80
1e+06
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
13
0.740628
0.990272
0.729555
0.989602
0.723831
0.989261
0.011838
0.000157
0.006976
0.000420
87
0.617235
0.043524
0.731350
0.989711
80
1e+07
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
12
0.740638
0.990269
0.729561
0.989601
0.723816
0.989262
0.001588
0.000197
0.006985
0.000418
88
0.630804
0.048653
0.731357
0.989712
80
1e+08
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
8
0.740644
0.990272
0.729592
0.989601
0.723801
0.989262
0.016246
0.005265
0.006991
0.000420
89
0.651543
0.044566
0.731352
0.989712
80
1e+09
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
10
0.740644
0.990272
0.729565
0.989604
0.723811
0.989262
0.028955
0.001285
0.006989
0.000419
90
0.629881
0.044737
0.731354
0.989712
80
1e+10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
9
0.740625
0.990270
0.729563
0.989602
0.723839
0.989264
0.018508
0.001414
0.006971
0.000418
91 rows × 19 columns
---------- SOC
Cross_val_score: [ 0.88502386 0.746843 0.85529741]
Explained variance score: 0.918039826962
Mean absolute error: 0.197678868055
Mean squared error: 0.12487316467
R2 score: 0.917169573909
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_svr__C
param_svr__kernel
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.319361
0.023102
-0.029721
-0.021360
20
0.01
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
91
-0.026112
-0.029868
-0.054350
-0.015237
-0.008716
-0.018973
0.020424
0.000441
0.018793
0.006207
1
0.299014
0.022062
0.303963
0.352870
20
0.1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
84
0.348919
0.321973
0.253888
0.376422
0.308908
0.360216
0.006960
0.001134
0.038965
0.022828
2
0.317139
0.021984
0.667930
0.838123
20
1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
77
0.779123
0.794396
0.628084
0.843248
0.596150
0.876726
0.011831
0.001250
0.079848
0.033806
3
0.315232
0.021623
0.719424
0.993472
20
10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
66
0.830118
0.993747
0.701989
0.993098
0.625737
0.993570
0.008701
0.000442
0.084369
0.000274
4
0.313111
0.021343
0.719225
0.993511
20
100
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
69
0.830118
0.993747
0.701388
0.993216
0.625739
0.993570
0.004680
0.000053
0.084410
0.000221
5
0.310461
0.021486
0.719224
0.993511
20
1000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
70
0.830118
0.993747
0.701388
0.993216
0.625737
0.993570
0.006831
0.000096
0.084411
0.000221
6
0.317728
0.021482
0.719221
0.993511
20
10000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
74
0.830118
0.993747
0.701380
0.993217
0.625737
0.993570
0.007606
0.000452
0.084411
0.000220
7
0.329430
0.021808
0.719224
0.993511
20
100000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
71
0.830118
0.993747
0.701388
0.993216
0.625737
0.993570
0.016663
0.000685
0.084411
0.000221
8
0.317954
0.022794
0.719224
0.993511
20
1e+06
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
73
0.830118
0.993747
0.701388
0.993216
0.625737
0.993570
0.008569
0.000980
0.084411
0.000221
9
0.312866
0.022011
0.719227
0.993511
20
1e+07
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
67
0.830118
0.993747
0.701388
0.993216
0.625745
0.993570
0.007873
0.001110
0.084407
0.000221
10
0.316109
0.021360
0.719224
0.993511
20
1e+08
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
72
0.830118
0.993747
0.701388
0.993216
0.625737
0.993570
0.000653
0.000192
0.084411
0.000221
11
0.312168
0.021506
0.719221
0.993511
20
1e+09
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
75
0.830118
0.993747
0.701380
0.993217
0.625737
0.993570
0.009432
0.000596
0.084411
0.000220
12
0.317920
0.021309
0.719225
0.993511
20
1e+10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
68
0.830118
0.993747
0.701388
0.993216
0.625739
0.993570
0.005347
0.000040
0.084410
0.000221
13
0.327793
0.026622
0.022485
0.029083
30
0.01
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
90
0.034283
0.021534
-0.003422
0.039353
0.036549
0.026363
0.018950
0.000358
0.018325
0.007524
14
0.314243
0.024515
0.389190
0.424722
30
0.1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
83
0.444152
0.385887
0.343162
0.450474
0.380044
0.437805
0.008344
0.000182
0.041745
0.027943
15
0.319602
0.023877
0.715710
0.845206
30
1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
76
0.828825
0.805009
0.684977
0.848281
0.632889
0.882327
0.006147
0.000515
0.082909
0.031640
16
0.332049
0.024207
0.766789
0.992618
30
10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
51
0.871854
0.993840
0.764380
0.990631
0.663727
0.993384
0.010835
0.000112
0.085012
0.001418
17
0.331257
0.024449
0.764844
0.993745
30
100
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
53
0.871868
0.994019
0.757666
0.993426
0.664583
0.993789
0.007253
0.000312
0.084803
0.000244
18
0.395287
0.027671
0.764842
0.993745
30
1000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
59
0.871868
0.994019
0.757660
0.993426
0.664583
0.993791
0.034850
0.002185
0.084803
0.000244
19
0.358599
0.026565
0.764842
0.993745
30
10000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
55
0.871868
0.994019
0.757660
0.993426
0.664583
0.993789
0.025640
0.002581
0.084803
0.000244
20
0.384401
0.027194
0.764844
0.993745
30
100000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
54
0.871873
0.994019
0.757660
0.993426
0.664583
0.993789
0.067419
0.002354
0.084805
0.000244
21
0.392593
0.046444
0.764838
0.993745
30
1e+06
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
61
0.871868
0.994019
0.757649
0.993426
0.664583
0.993791
0.045445
0.031727
0.084803
0.000244
22
0.372165
0.024865
0.764842
0.993745
30
1e+07
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
57
0.871868
0.994019
0.757660
0.993426
0.664583
0.993789
0.025361
0.000434
0.084803
0.000244
23
0.394865
0.024918
0.764842
0.993745
30
1e+08
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
60
0.871868
0.994019
0.757660
0.993426
0.664583
0.993791
0.082233
0.000688
0.084803
0.000244
24
0.382893
0.028189
0.764842
0.993745
30
1e+09
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
56
0.871868
0.994019
0.757660
0.993426
0.664583
0.993789
0.069966
0.004454
0.084803
0.000244
25
0.422780
0.027881
0.764842
0.993745
30
1e+10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
58
0.871868
0.994019
0.757660
0.993426
0.664583
0.993789
0.103532
0.004767
0.084803
0.000244
26
0.423294
0.030433
0.056051
0.061180
40
0.01
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
89
0.073512
0.054486
0.024567
0.068808
0.070006
0.060246
0.014250
0.001028
0.022287
0.005884
27
0.414515
0.027404
0.432982
0.460860
40
0.1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
82
0.492789
0.420087
0.391168
0.486517
0.414757
0.475976
0.022337
0.000116
0.043451
0.029150
28
0.416736
0.026426
0.739868
0.844910
40
1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
65
0.850813
0.806858
0.710751
0.846913
0.657609
0.880959
0.013271
0.000467
0.081538
0.030285
29
0.430651
0.026402
0.796881
0.989847
40
10
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
41
0.903805
0.989258
0.796358
0.987893
0.690068
0.992390
0.013976
0.000090
0.087287
0.001883
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
61
0.445266
0.034373
0.815253
0.994144
60
1e+07
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
24
0.907102
0.994351
0.814967
0.993763
0.723333
0.994318
0.019753
0.001208
0.075048
0.000270
62
0.413190
0.034804
0.815261
0.994145
60
1e+08
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
23
0.907098
0.994351
0.814986
0.993765
0.723342
0.994319
0.010535
0.003070
0.075042
0.000269
63
0.409186
0.032133
0.815245
0.994145
60
1e+09
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
26
0.907095
0.994351
0.814964
0.993764
0.723319
0.994319
0.006206
0.000211
0.075050
0.000269
64
0.436541
0.034709
0.815242
0.994145
60
1e+10
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
31
0.907095
0.994351
0.814954
0.993766
0.723320
0.994318
0.016125
0.000542
0.075050
0.000268
65
0.417571
0.042042
0.101794
0.104861
70
0.01
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
86
0.125934
0.098855
0.071581
0.114480
0.107774
0.101248
0.009710
0.001229
0.022595
0.006872
66
0.387120
0.038799
0.475037
0.491518
70
0.1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
79
0.535590
0.452053
0.439686
0.512430
0.449599
0.510070
0.005343
0.000705
0.043091
0.027923
67
0.407099
0.042082
0.762842
0.830983
70
1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
62
0.863533
0.796561
0.734987
0.832528
0.689615
0.863860
0.008016
0.005216
0.073700
0.027496
68
0.420138
0.034639
0.841903
0.976212
70
10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
2
0.930859
0.970874
0.839115
0.979600
0.755390
0.978163
0.025424
0.000663
0.071685
0.003820
69
0.452732
0.039206
0.823417
0.994188
70
100
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
13
0.910152
0.994383
0.824644
0.993830
0.735120
0.994352
0.010805
0.005149
0.071484
0.000254
70
0.462702
0.034957
0.823220
0.994193
70
1000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
18
0.910158
0.994383
0.824040
0.993843
0.735123
0.994354
0.025168
0.000406
0.071483
0.000248
71
0.468438
0.035142
0.823221
0.994193
70
10000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
17
0.910151
0.994384
0.824062
0.993844
0.735113
0.994352
0.032731
0.000107
0.071484
0.000248
72
0.436551
0.036658
0.823219
0.994192
70
100000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
20
0.910166
0.994384
0.824051
0.993842
0.735102
0.994351
0.002176
0.002154
0.071495
0.000248
73
0.452038
0.035281
0.823225
0.994193
70
1e+06
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
16
0.910149
0.994384
0.824074
0.993843
0.735114
0.994352
0.015298
0.000267
0.071483
0.000248
74
0.452370
0.035877
0.823219
0.994193
70
1e+07
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
19
0.910153
0.994383
0.824056
0.993845
0.735113
0.994352
0.040647
0.000828
0.071485
0.000247
75
0.435740
0.035897
0.823214
0.994193
70
1e+08
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
21
0.910152
0.994383
0.824054
0.993844
0.735099
0.994352
0.012965
0.001319
0.071490
0.000247
76
0.465869
0.036978
0.823225
0.994194
70
1e+09
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
15
0.910153
0.994385
0.824068
0.993844
0.735117
0.994352
0.016560
0.001097
0.071483
0.000248
77
0.437446
0.034849
0.823230
0.994193
70
1e+10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
14
0.910184
0.994383
0.824060
0.993844
0.735110
0.994352
0.012975
0.000346
0.071499
0.000247
78
0.485154
0.044271
0.109606
0.112016
80
0.01
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
85
0.135886
0.107903
0.078072
0.119661
0.114759
0.108483
0.019324
0.000233
0.023889
0.005411
79
0.514969
0.043568
0.479068
0.493430
80
0.1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
78
0.538532
0.454743
0.446061
0.514353
0.452380
0.511192
0.027692
0.001819
0.042208
0.027386
80
0.482135
0.038229
0.764864
0.825503
80
1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
52
0.862673
0.791446
0.737551
0.827729
0.693990
0.857334
0.022999
0.001128
0.071538
0.026945
81
0.497603
0.039715
0.849061
0.972049
80
10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
1
0.933099
0.966348
0.845913
0.976458
0.767844
0.973340
0.020723
0.003779
0.067523
0.004227
82
0.537780
0.043573
0.831659
0.993952
80
100
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
4
0.911818
0.994351
0.838216
0.993166
0.744632
0.994339
0.031930
0.009128
0.068432
0.000556
83
0.508262
0.038018
0.829566
0.994165
80
1000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
7
0.911828
0.994350
0.831914
0.993808
0.744637
0.994337
0.006489
0.001038
0.068297
0.000253
84
0.522231
0.037820
0.829572
0.994165
80
10000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
6
0.911833
0.994350
0.831913
0.993808
0.744650
0.994337
0.032772
0.001257
0.068294
0.000253
85
0.516826
0.038200
0.829559
0.994166
80
100000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
11
0.911831
0.994351
0.831901
0.993809
0.744626
0.994338
0.008254
0.001348
0.068303
0.000253
86
0.515419
0.038318
0.829559
0.994166
80
1e+06
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
12
0.911826
0.994350
0.831894
0.993809
0.744637
0.994338
0.009075
0.001011
0.068297
0.000252
87
0.509704
0.037788
0.829561
0.994165
80
1e+07
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
10
0.911832
0.994350
0.831904
0.993806
0.744629
0.994337
0.015228
0.001030
0.068302
0.000254
88
0.508652
0.038708
0.829573
0.994165
80
1e+08
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
5
0.911826
0.994350
0.831908
0.993807
0.744665
0.994339
0.015319
0.000973
0.068285
0.000253
89
0.518681
0.038601
0.829565
0.994167
80
1e+09
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
8
0.911834
0.994351
0.831890
0.993811
0.744653
0.994338
0.019671
0.001450
0.068293
0.000252
90
0.507867
0.037459
0.829563
0.994166
80
1e+10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
9
0.911839
0.994351
0.831896
0.993808
0.744635
0.994339
0.016063
0.000303
0.068302
0.000253
91 rows × 19 columns
---------- Sand
Cross_val_score: [ 0.82546639 0.82082172 0.82757977]
Explained variance score: 0.90157072917
Mean absolute error: 0.227022964183
Mean squared error: 0.0970269484613
R2 score: 0.901568721611
mean_fit_time
mean_score_time
mean_test_score
mean_train_score
param_pca__n_components
param_svr__C
param_svr__kernel
params
rank_test_score
split0_test_score
split0_train_score
split1_test_score
split1_train_score
split2_test_score
split2_train_score
std_fit_time
std_score_time
std_test_score
std_train_score
0
0.318563
0.025724
0.118858
0.137097
20
0.01
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
91
0.090111
0.143334
0.136383
0.134365
0.130193
0.133590
0.017083
0.001810
0.020523
0.004422
1
0.301952
0.023087
0.581639
0.663060
20
0.1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
84
0.527238
0.693515
0.618684
0.651743
0.599206
0.643922
0.007204
0.000154
0.039353
0.021770
2
0.312524
0.023586
0.757978
0.938373
20
1
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
34
0.716448
0.937547
0.778397
0.941578
0.779249
0.935993
0.009721
0.000329
0.029425
0.002354
3
0.330553
0.022505
0.745190
0.989688
20
10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
52
0.740263
0.991538
0.726104
0.991273
0.769223
0.986254
0.015741
0.000286
0.017935
0.002431
4
0.325961
0.022410
0.744689
0.991219
20
100
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
64
0.740286
0.991543
0.726157
0.991277
0.767639
0.990837
0.011293
0.000593
0.017209
0.000291
5
0.327313
0.023292
0.744689
0.991218
20
1000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
63
0.740279
0.991541
0.726160
0.991275
0.767644
0.990837
0.007807
0.001194
0.017211
0.000290
6
0.332757
0.021986
0.744683
0.991218
20
10000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
68
0.740265
0.991541
0.726153
0.991278
0.767648
0.990835
0.004339
0.000023
0.017216
0.000291
7
0.329486
0.022626
0.744688
0.991217
20
100000
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
65
0.740279
0.991541
0.726160
0.991275
0.767641
0.990834
0.015582
0.000413
0.017209
0.000292
8
0.324419
0.022141
0.744682
0.991218
20
1e+06
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
69
0.740265
0.991541
0.726160
0.991275
0.767639
0.990837
0.009285
0.000097
0.017210
0.000290
9
0.325052
0.022078
0.744685
0.991217
20
1e+07
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
67
0.740263
0.991538
0.726168
0.991276
0.767640
0.990837
0.012495
0.000115
0.017207
0.000289
10
0.320138
0.022113
0.744689
0.991218
20
1e+08
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
61
0.740286
0.991541
0.726160
0.991275
0.767640
0.990837
0.000962
0.000197
0.017208
0.000290
11
0.325326
0.022241
0.744688
0.991217
20
1e+09
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
66
0.740263
0.991538
0.726168
0.991276
0.767648
0.990835
0.006366
0.000238
0.017211
0.000290
12
0.328419
0.022105
0.744689
0.991218
20
1e+10
rbf
{u'pca__n_components': 20, u'svr__kernel': u'r...
62
0.740263
0.991538
0.726172
0.991277
0.767649
0.990837
0.013505
0.000080
0.017210
0.000289
13
0.324977
0.027842
0.187116
0.205464
30
0.01
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
90
0.151833
0.212370
0.208143
0.200345
0.201508
0.203676
0.012056
0.000650
0.025143
0.005069
14
0.313031
0.026631
0.635731
0.698361
30
0.1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
83
0.575396
0.726695
0.680403
0.685465
0.651627
0.682924
0.004866
0.000155
0.044329
0.020062
15
0.329455
0.026857
0.782492
0.920930
30
1
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
11
0.742854
0.916409
0.805405
0.925240
0.799370
0.921141
0.004328
0.001702
0.028190
0.003608
16
0.362648
0.025331
0.766190
0.988574
30
10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
13
0.769148
0.990867
0.732031
0.991039
0.797378
0.983817
0.016546
0.000204
0.026743
0.003365
17
0.364651
0.025453
0.758976
0.991398
30
100
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
24
0.762594
0.991649
0.733021
0.991565
0.781300
0.990980
0.017260
0.000115
0.019863
0.000297
18
0.350731
0.025775
0.758958
0.991398
30
1000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
31
0.762592
0.991648
0.732991
0.991567
0.781278
0.990979
0.005312
0.000396
0.019868
0.000298
19
0.359580
0.025449
0.758952
0.991398
30
10000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
32
0.762586
0.991648
0.732984
0.991566
0.781271
0.990979
0.012276
0.000223
0.019868
0.000298
20
0.352435
0.025617
0.758976
0.991399
30
100000
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
25
0.762615
0.991649
0.733003
0.991567
0.781295
0.990980
0.013550
0.000389
0.019870
0.000298
21
0.356453
0.025789
0.758964
0.991397
30
1e+06
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
28
0.762592
0.991648
0.732991
0.991565
0.781295
0.990980
0.007914
0.000662
0.019874
0.000297
22
0.355836
0.025325
0.758960
0.991397
30
1e+07
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
30
0.762592
0.991648
0.732991
0.991565
0.781283
0.990980
0.005285
0.000100
0.019870
0.000297
23
0.352883
0.025362
0.758966
0.991398
30
1e+08
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
27
0.762586
0.991648
0.733016
0.991566
0.781284
0.990979
0.008676
0.000194
0.019859
0.000298
24
0.347018
0.025534
0.758964
0.991397
30
1e+09
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
29
0.762592
0.991648
0.733003
0.991567
0.781282
0.990977
0.006812
0.000367
0.019864
0.000299
25
0.357681
0.026838
0.758967
0.991398
30
1e+10
rbf
{u'pca__n_components': 30, u'svr__kernel': u'r...
26
0.762594
0.991649
0.732991
0.991566
0.781300
0.990980
0.015987
0.001104
0.019876
0.000298
26
0.396140
0.031295
0.243559
0.259445
40
0.01
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
89
0.209011
0.271276
0.265250
0.249207
0.256551
0.257853
0.010521
0.000436
0.024733
0.009079
27
0.400184
0.029970
0.652430
0.705137
40
0.1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
82
0.593844
0.731674
0.696771
0.688973
0.666900
0.694765
0.011205
0.000409
0.043258
0.018913
28
0.402341
0.028353
0.790778
0.904785
40
1
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
7
0.753791
0.899235
0.812992
0.909582
0.805694
0.905537
0.011483
0.000120
0.026373
0.004257
29
0.440193
0.028393
0.790004
0.984065
40
10
rbf
{u'pca__n_components': 40, u'svr__kernel': u'r...
8
0.807582
0.983953
0.747323
0.988624
0.815039
0.979618
0.014743
0.000276
0.030304
0.003678
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
61
0.473304
0.036391
0.750218
0.991407
60
1e+07
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
46
0.762331
0.991684
0.728834
0.991404
0.759443
0.991133
0.013409
0.000729
0.015152
0.000225
62
0.488738
0.037964
0.750215
0.991406
60
1e+08
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
49
0.762322
0.991682
0.728830
0.991404
0.759447
0.991130
0.035002
0.002677
0.015153
0.000225
63
0.525531
0.038178
0.750211
0.991407
60
1e+09
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
51
0.762295
0.991684
0.728816
0.991404
0.759476
0.991133
0.053100
0.002872
0.015158
0.000225
64
0.536934
0.035995
0.750212
0.991406
60
1e+10
rbf
{u'pca__n_components': 60, u'svr__kernel': u'r...
50
0.762315
0.991682
0.728822
0.991403
0.759452
0.991131
0.048491
0.000366
0.015155
0.000225
65
0.437614
0.045999
0.334549
0.348440
70
0.01
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
86
0.296151
0.360576
0.361817
0.335517
0.345828
0.349228
0.049847
0.002120
0.027975
0.010245
66
0.423468
0.043432
0.660998
0.697282
70
0.1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
78
0.600542
0.719456
0.711716
0.680382
0.670971
0.692009
0.029876
0.000879
0.045944
0.016382
67
0.401922
0.038847
0.788841
0.864587
70
1
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
9
0.751726
0.865107
0.816075
0.866602
0.798867
0.862050
0.003726
0.000230
0.027217
0.001894
68
0.429829
0.038470
0.824732
0.964468
70
10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
2
0.841169
0.959503
0.797839
0.972183
0.835124
0.961718
0.015831
0.000268
0.019157
0.005530
69
0.511426
0.039115
0.760662
0.989549
70
100
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
14
0.761556
0.991596
0.725972
0.991426
0.794453
0.985623
0.019034
0.000145
0.027947
0.002776
70
0.513852
0.040760
0.744711
0.991390
70
1000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
56
0.758343
0.991640
0.726122
0.991427
0.749616
0.991103
0.004249
0.000321
0.013607
0.000221
71
0.511978
0.039761
0.744696
0.991390
70
10000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
59
0.758344
0.991638
0.726105
0.991427
0.749588
0.991104
0.004416
0.000441
0.013611
0.000220
72
0.532539
0.041482
0.744719
0.991390
70
100000
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
54
0.758366
0.991639
0.726091
0.991427
0.749648
0.991104
0.031702
0.001904
0.013633
0.000220
73
0.517734
0.039825
0.744711
0.991391
70
1e+06
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
55
0.758360
0.991640
0.726111
0.991429
0.749610
0.991104
0.005850
0.000636
0.013617
0.000220
74
0.511984
0.041205
0.744723
0.991388
70
1e+07
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
53
0.758375
0.991637
0.726112
0.991425
0.749631
0.991103
0.011921
0.001922
0.013624
0.000219
75
0.541039
0.042497
0.744697
0.991390
70
1e+08
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
58
0.758332
0.991641
0.726105
0.991426
0.749601
0.991102
0.047089
0.003834
0.013609
0.000221
76
0.561987
0.040932
0.744704
0.991390
70
1e+09
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
57
0.758333
0.991640
0.726115
0.991427
0.749612
0.991104
0.017181
0.001222
0.013606
0.000220
77
0.540473
0.042352
0.744694
0.991391
70
1e+10
rbf
{u'pca__n_components': 70, u'svr__kernel': u'r...
60
0.758314
0.991641
0.726095
0.991429
0.749620
0.991103
0.005172
0.002187
0.013610
0.000221
78
0.571590
0.048486
0.350695
0.363814
80
0.01
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
85
0.312105
0.376121
0.379440
0.350234
0.360690
0.365088
0.152670
0.000132
0.028390
0.010607
79
0.458749
0.045663
0.660707
0.694093
80
0.1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
79
0.601674
0.715605
0.712488
0.677379
0.668186
0.689295
0.009611
0.000313
0.045561
0.015970
80
0.464768
0.043861
0.786385
0.854897
80
1
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
10
0.750357
0.857433
0.811376
0.854170
0.797562
0.853088
0.014199
0.001834
0.026139
0.001847
81
0.493957
0.044799
0.829148
0.958342
80
10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
1
0.842780
0.951861
0.806106
0.966557
0.838506
0.956609
0.008514
0.005109
0.016371
0.006123
82
0.587251
0.042409
0.767804
0.988141
80
100
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
12
0.777542
0.990775
0.726114
0.991174
0.799718
0.982475
0.010468
0.000576
0.030812
0.004010
83
0.614917
0.043139
0.738744
0.991337
80
1000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
75
0.753062
0.991593
0.723257
0.991472
0.739858
0.990945
0.019897
0.000223
0.012197
0.000281
84
0.626726
0.042994
0.738732
0.991337
80
10000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
77
0.753015
0.991594
0.723230
0.991473
0.739898
0.990945
0.011102
0.000541
0.012191
0.000282
85
0.622219
0.045425
0.738759
0.991337
80
100000
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
70
0.753087
0.991592
0.723260
0.991473
0.739874
0.990946
0.017669
0.001972
0.012206
0.000281
86
0.614988
0.048456
0.738740
0.991337
80
1e+06
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
76
0.753031
0.991595
0.723252
0.991472
0.739882
0.990946
0.019633
0.002962
0.012188
0.000281
87
0.614861
0.043437
0.738745
0.991337
80
1e+07
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
74
0.753062
0.991592
0.723240
0.991474
0.739877
0.990945
0.002419
0.001253
0.012205
0.000282
88
0.628104
0.042915
0.738746
0.991337
80
1e+08
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
73
0.753040
0.991594
0.723245
0.991472
0.739898
0.990947
0.017771
0.000337
0.012195
0.000281
89
0.617600
0.044125
0.738753
0.991336
80
1e+09
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
71
0.753063
0.991590
0.723259
0.991474
0.739881
0.990945
0.015480
0.000374
0.012197
0.000281
90
0.611163
0.042932
0.738750
0.991337
80
1e+10
rbf
{u'pca__n_components': 80, u'svr__kernel': u'r...
72
0.753039
0.991593
0.723283
0.991474
0.739872
0.990945
0.010227
0.000315
0.012178
0.000281
91 rows × 19 columns
Completed in -1490.37 sec
In [93]:
print len(y_pipelines_svr)
print y_scores_svr
5
[0.15630930533293441, 1.1704317973796174, 0.14732587939348427, 0.12487316466964694, 0.097026948461285234]
In [94]:
# Pick out the best performing models/pipelines based on mse for each predictor
# combine results lists from modeling cells
y_vars_pipelines = [y_pipelines_lin, y_pipelines_linsel, y_pipelines_ridge, y_pipelines_svr]
y_vars_scores = [y_scores_lin, y_scores_linsel, y_scores_ridge, y_scores_svr]
pipelines = np.array(y_vars_pipelines)
scores = np.array(y_vars_scores)
pipeline_winners = []
print pipelines.shape
# P sucks
print scores
for ind, y in enumerate(y_vars):
# get index of best score
best_ind = np.argmin(scores[:,ind])
print(best_ind, ind)
# fit the pipeline for all X
pipelines[best_ind, ind].fit(X_wDepth,y)
# capture the pipeline
pipeline_winners.append(pipelines[best_ind, ind])
print len(pipeline_winners)
(4, 5)
[[ 0.16877459 1.10647336 0.1767721 0.15434747 0.11549188]
[ 0.16842672 1.10076421 0.17877103 0.15445713 0.1096961 ]
[ 0.1687742 1.07343443 0.16403397 0.14144016 0.11547904]
[ 0.15630931 1.1704318 0.14732588 0.12487316 0.09702695]]
(3, 0)
(2, 1)
(3, 2)
(3, 3)
(3, 4)
5
In [95]:
# Iterate through test samples
allPredictions = []
pipeline_winners = y_pipelines_lin
for s_ind in range(len(test_x)):
sampleId = test_ids[s_ind]
sample = test_x_wdepth[s_ind]
currentSamplePredictions = []
# Use the winning model to estimate the outcome variables
for ind in range(0, 5):
pred = pipeline_winners[ind].predict(sample.reshape(1,-1))[0]
currentSamplePredictions.append(pred)
allPredictions.append(currentSamplePredictions)
#print len(allPredictions)
#print allPredictions
print 'Predictions calculated.'
Predictions calculated.
In [96]:
# Generate csv for AfricaSoil Kaggle
filename = 'jc_20170422_1.csv'
# Clean file
open(filename, 'w').close()
with open(filename, 'w') as f:
f.write('PIDN,Ca,P,pH,SOC,Sand\n') # python will convert \n to os.linesep
# Iterate through test samples
for i in range(len(allPredictions)):
pred = allPredictions[i]
testId = test_ids[i]
text = testId + ',' + str(pred[0]) + ',' + str(pred[1]) + ',' + str(pred[2]) + ',' + str(pred[3]) + ',' + str(pred[4]) + '\n'
f.write(text)
f.close()
In [97]:
# Check where jupyter may drop the csv if it can't be found where expected
import os
fileDir = os.path.dirname(os.path.realpath('__file__'))
print fileDir
/Users/jcasper/Documents/Education/UCBerkeley-DS/2017_Spring/DATASCIW207_ML/kaggle_africa_soil/notebook
NOTES
Options for high dimensional data where large number of features and fewer number of observations: can choose random sets of variables and asses their importance using cross-validation; ridge regression, the lasso or elastic net for regularization (process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting); choose a technique, such as a support vector machine or random forest that deals well with a large number of predictors.
LASSO (least absolute shrinkage and selection operator) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the statistical model it produces.
When considering ML methods, consider:
Always start simple: first algorithm to try would be naive Bayes, logistic regression, k-nearest neighbour (First start with one neighbour) and Fisher's linear discriminant before anything else. For advanced machine learning, ensemble methods are the ones that produces the best results as is shown by winners in kaggale competition and XGBOOST has been very popular among the kaggale winners. Neural Networks may be useful for predicting values but number of observations is low.
Subject: dirt quality for agriculture, Predictor variables: 3593 features (see feature_names), Response variables: 'Ca', 'P', 'pH', 'Soc', 'Sand
A continuous predictor variable is sometimes called a covariate and a categorical predictor variable is sometimes called a factor. In the cake experiment, a covariate could be various oven temperatures and a factor could be different ovens. Usually, you create a plot of predictor variables on the x-axis and response variables on the y-axis.
For continuous variables such as income, it is customary to do a log transformation to get it as close to a normal distribution as possible. You can then employ OLS and run some diagnostics to check your model fit. For other types of continuous variables, get a histogram and check the distribution. If it is somewhat normal, you can run an OLS and check the diagnostics and model fit.
References:
Content source: carlosscastro/kaggle_africa_soil
Similar notebooks: