Predicting XGBoost score with (1):
1183747 samples
205 features
{'num_round': 200, 'colsample_bytree': 0.7453702907128406, 'silent': 1, 'eval_metric': 'auc', 'min_child_weight': 2, 'subsample': 0.8392938371195724, 'eta': 0.1, 'objective': 'binary:logistic', 'seed': 1712, 'max_depth': 7, 'early_stopping': 30, 'booster': 'gbtree'} parameters
[0] train-auc:0.867207 eval-auc:0.873051
Multiple eval metrics have been passed: 'eval-auc' will be used for early stopping.
Will train until eval-auc hasn't improved in 30 rounds.
[5] train-auc:0.898239 eval-auc:0.900304
[10] train-auc:0.905708 eval-auc:0.909996
[15] train-auc:0.910038 eval-auc:0.912381
[20] train-auc:0.916738 eval-auc:0.915035
[25] train-auc:0.919084 eval-auc:0.915347
[30] train-auc:0.92289 eval-auc:0.917575
[35] train-auc:0.924875 eval-auc:0.917585
[40] train-auc:0.930593 eval-auc:0.918583
[45] train-auc:0.933461 eval-auc:0.919082
[50] train-auc:0.937429 eval-auc:0.919758
[55] train-auc:0.941318 eval-auc:0.918685
[60] train-auc:0.946065 eval-auc:0.919393
[65] train-auc:0.9496 eval-auc:0.920822
[70] train-auc:0.953826 eval-auc:0.920526
[75] train-auc:0.957756 eval-auc:0.921945
[80] train-auc:0.960299 eval-auc:0.922854
[85] train-auc:0.962387 eval-auc:0.92339
[90] train-auc:0.965019 eval-auc:0.923267
[95] train-auc:0.966425 eval-auc:0.922979
[100] train-auc:0.967711 eval-auc:0.922816
[105] train-auc:0.969176 eval-auc:0.923066
[110] train-auc:0.970298 eval-auc:0.923205
Stopping. Best iteration:
[82] train-auc:0.960947 eval-auc:0.923542
0.960947
0.923542
0.43
0.467485658843
Predicting XGBoost score with (2):
1183747 samples
205 features
{'num_round': 200, 'colsample_bytree': 0.8980266639177703, 'silent': 1, 'eval_metric': 'auc', 'min_child_weight': 17, 'subsample': 0.7212129811909465, 'eta': 0.1, 'objective': 'binary:logistic', 'seed': 1712, 'max_depth': 7, 'early_stopping': 30, 'booster': 'gbtree'} parameters
[0] train-auc:0.864343 eval-auc:0.869565
Multiple eval metrics have been passed: 'eval-auc' will be used for early stopping.
Will train until eval-auc hasn't improved in 30 rounds.
[5] train-auc:0.889238 eval-auc:0.889885
[10] train-auc:0.892675 eval-auc:0.893925
[15] train-auc:0.911287 eval-auc:0.91405
[20] train-auc:0.915807 eval-auc:0.915146
[25] train-auc:0.919535 eval-auc:0.915715
[30] train-auc:0.923535 eval-auc:0.919075
[35] train-auc:0.928089 eval-auc:0.919679
[40] train-auc:0.931808 eval-auc:0.920687
[45] train-auc:0.933352 eval-auc:0.920361
[50] train-auc:0.935335 eval-auc:0.919503
[55] train-auc:0.937538 eval-auc:0.920457
[60] train-auc:0.939877 eval-auc:0.921004
[65] train-auc:0.941501 eval-auc:0.922674
[70] train-auc:0.94331 eval-auc:0.923136
[75] train-auc:0.944683 eval-auc:0.922491
[80] train-auc:0.947022 eval-auc:0.922225
[85] train-auc:0.948472 eval-auc:0.92271
[90] train-auc:0.949981 eval-auc:0.922523
[95] train-auc:0.951713 eval-auc:0.921723
Stopping. Best iteration:
[68] train-auc:0.94286 eval-auc:0.923329
0.94286
0.923329
0.35
0.462257549847
Predicting XGBoost score with (3):
1183747 samples
205 features
{'num_round': 200, 'colsample_bytree': 0.739144317250818, 'silent': 1, 'eval_metric': 'auc', 'min_child_weight': 34, 'subsample': 0.801345202299914, 'eta': 0.1, 'objective': 'binary:logistic', 'seed': 1712, 'max_depth': 7, 'early_stopping': 30, 'booster': 'gbtree'} parameters
[0] train-auc:0.858663 eval-auc:0.863485
Multiple eval metrics have been passed: 'eval-auc' will be used for early stopping.
Will train until eval-auc hasn't improved in 30 rounds.
[5] train-auc:0.891567 eval-auc:0.893424
[10] train-auc:0.90638 eval-auc:0.910631
[15] train-auc:0.909735 eval-auc:0.912267
[20] train-auc:0.916526 eval-auc:0.917744
[25] train-auc:0.917784 eval-auc:0.91753
[30] train-auc:0.921465 eval-auc:0.919432
[35] train-auc:0.923342 eval-auc:0.919033
[40] train-auc:0.928687 eval-auc:0.919672
[45] train-auc:0.930837 eval-auc:0.919901
[50] train-auc:0.933015 eval-auc:0.921099
[55] train-auc:0.934305 eval-auc:0.921451
[60] train-auc:0.936361 eval-auc:0.921169
[65] train-auc:0.938001 eval-auc:0.921948
[70] train-auc:0.939736 eval-auc:0.922118
[75] train-auc:0.941142 eval-auc:0.921947
[80] train-auc:0.942057 eval-auc:0.922473
[85] train-auc:0.943369 eval-auc:0.922307
[90] train-auc:0.944584 eval-auc:0.92295
[95] train-auc:0.945345 eval-auc:0.922934
[100] train-auc:0.946451 eval-auc:0.922676
[105] train-auc:0.947304 eval-auc:0.922459
[110] train-auc:0.948216 eval-auc:0.922615
[115] train-auc:0.948894 eval-auc:0.922701
[120] train-auc:0.950044 eval-auc:0.922428
Stopping. Best iteration:
[94] train-auc:0.945158 eval-auc:0.923018
0.945158
0.923018
0.25
0.468943862623
Predicting XGBoost score with (4):
1183747 samples
205 features
{'num_round': 200, 'colsample_bytree': 0.7410143734327053, 'silent': 1, 'eval_metric': 'auc', 'min_child_weight': 10, 'subsample': 0.721034425443535, 'eta': 0.1, 'objective': 'binary:logistic', 'seed': 1712, 'max_depth': 7, 'early_stopping': 30, 'booster': 'gbtree'} parameters
[0] train-auc:0.861042 eval-auc:0.866097
Multiple eval metrics have been passed: 'eval-auc' will be used for early stopping.
Will train until eval-auc hasn't improved in 30 rounds.
[5] train-auc:0.874991 eval-auc:0.88022
[10] train-auc:0.902645 eval-auc:0.907294
[15] train-auc:0.91004 eval-auc:0.912556
[20] train-auc:0.917408 eval-auc:0.915894
[25] train-auc:0.918707 eval-auc:0.915874
[30] train-auc:0.922567 eval-auc:0.917761
[35] train-auc:0.924465 eval-auc:0.917982
[40] train-auc:0.929008 eval-auc:0.920142
[45] train-auc:0.931855 eval-auc:0.919786
[50] train-auc:0.935537 eval-auc:0.920682
[55] train-auc:0.938028 eval-auc:0.920825
[60] train-auc:0.94116 eval-auc:0.921698
[65] train-auc:0.943445 eval-auc:0.922397
[70] train-auc:0.945271 eval-auc:0.92203
[75] train-auc:0.94793 eval-auc:0.922402
[80] train-auc:0.949659 eval-auc:0.922743
[85] train-auc:0.95143 eval-auc:0.923158
[90] train-auc:0.95282 eval-auc:0.922941
[95] train-auc:0.954532 eval-auc:0.922561
[100] train-auc:0.955805 eval-auc:0.922697
[105] train-auc:0.956831 eval-auc:0.922748
[110] train-auc:0.958109 eval-auc:0.922656
[115] train-auc:0.959173 eval-auc:0.922332
Stopping. Best iteration:
[85] train-auc:0.95143 eval-auc:0.923158
0.95143
0.923158
0.45
0.466437997444
Predicting XGBoost score with (5):
1183747 samples
205 features
{'num_round': 200, 'colsample_bytree': 0.8189064535289485, 'silent': 1, 'eval_metric': 'auc', 'min_child_weight': 34, 'subsample': 0.8047666639909352, 'eta': 0.1, 'objective': 'binary:logistic', 'seed': 1712, 'max_depth': 7, 'early_stopping': 30, 'booster': 'gbtree'} parameters
[0] train-auc:0.859828 eval-auc:0.864589
Multiple eval metrics have been passed: 'eval-auc' will be used for early stopping.
Will train until eval-auc hasn't improved in 30 rounds.
[5] train-auc:0.887744 eval-auc:0.890307
[10] train-auc:0.896932 eval-auc:0.898908
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
<ipython-input-214-7bd170faee54> in <module>()
6 algo=tpe.suggest,
7 max_evals=200,
----> 8 trials=trials)
/Users/joostbloom/anaconda/lib/python2.7/site-packages/hyperopt/fmin.pyc in fmin(fn, space, algo, max_evals, trials, rseed)
332
333 rval = FMinIter(algo, domain, trials, max_evals=max_evals)
--> 334 rval.exhaust()
335 return trials.argmin
336
/Users/joostbloom/anaconda/lib/python2.7/site-packages/hyperopt/fmin.pyc in exhaust(self)
292 def exhaust(self):
293 n_done = len(self.trials)
--> 294 self.run(self.max_evals - n_done, block_until_done=self.async)
295 self.trials.refresh()
296 return self
/Users/joostbloom/anaconda/lib/python2.7/site-packages/hyperopt/fmin.pyc in run(self, N, block_until_done)
266 else:
267 # -- loop over trials and do the jobs directly
--> 268 self.serial_evaluate()
269
270 if stopped:
/Users/joostbloom/anaconda/lib/python2.7/site-packages/hyperopt/fmin.pyc in serial_evaluate(self, N)
185 ctrl = base.Ctrl(self.trials, current_trial=trial)
186 try:
--> 187 result = self.domain.evaluate(spec, ctrl)
188 except Exception, e:
189 logger.info('job exception: %s' % str(e))
/Users/joostbloom/anaconda/lib/python2.7/site-packages/hyperopt/fmin.pyc in evaluate(self, config, ctrl, attach_attachments)
112 pyll_rval = pyll.rec_eval(self.expr, memo=memo,
113 print_node_on_error=self.rec_eval_print_node_on_error)
--> 114 rval = self.fn(pyll_rval)
115
116 if isinstance(rval, (float, int, np.number)):
<ipython-input-212-540c28882d45> in score_xgboost(params)
27 evals_result=eval_result,
28 early_stopping_rounds=params['early_stopping'],
---> 29 verbose_eval=5)
30
31 #print('\t score: {}'.format(roc_auc_score(y_val, y_pred_val)))
/Users/joostbloom/anaconda/lib/python2.7/site-packages/xgboost-0.6-py2.7.egg/xgboost/training.pyc in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, learning_rates, xgb_model, callbacks)
201 evals=evals,
202 obj=obj, feval=feval,
--> 203 xgb_model=xgb_model, callbacks=callbacks)
204
205
/Users/joostbloom/anaconda/lib/python2.7/site-packages/xgboost-0.6-py2.7.egg/xgboost/training.pyc in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks)
72 # Skip the first update if it is a recovery step.
73 if version % 2 == 0:
---> 74 bst.update(dtrain, i, obj)
75 bst.save_rabit_checkpoint()
76 version += 1
/Users/joostbloom/anaconda/lib/python2.7/site-packages/xgboost-0.6-py2.7.egg/xgboost/core.pyc in update(self, dtrain, iteration, fobj)
804
805 if fobj is None:
--> 806 _check_call(_LIB.XGBoosterUpdateOneIter(self.handle, iteration, dtrain.handle))
807 else:
808 pred = self.predict(dtrain)
KeyboardInterrupt: