Iterative Construction of a Penalised Vine Structure

This notebook iteratively estimate the quantile.

Libraries


In [1]:
import openturns as ot
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

%load_ext autoreload
%autoreload 2

random_state = 123
np.random.seed(random_state)

Model function

This example consider the simple additive example.


In [2]:
from depimpact.tests import func_overflow, margins_overflow, var_names_overflow, func_sum
from depimpact.tests.test_functions import func_prod
from dependence import iterative_vine_minimize

test_func = func_sum

Dimension and margins

We first define the problem dimension and the margins


In [3]:
if test_func == func_overflow:
    margins = margins_overflow
    dim = len(margins)
else:
    dim = 2
    margins = [ot.Normal()]*dim

We chose the coefficients of the variables throught the additive function.


In [126]:
if test_func in [func_sum, func_prod]:
    #coeficients = np.logspace(0., 3., dim+1, endpoint=False)[1:]
    #coeficients /= coeficients.max()
    coeficients = np.asarray(range(1, dim+1))
    new_test_func = lambda x: test_func(x, coeficients)

    n_plot = 10000
    x = np.asarray(ot.ComposedDistribution(margins).getSample(n_plot))
    y = new_test_func(x)    

    fig, axes = plt.subplots(dim, 1, sharex=False, sharey=True, figsize=(4, 2*dim))
    for i in range(dim):
        ax = axes[i]
        ax.plot(x[:, i], y, '.')
        ax.set_xlabel(r'$X_{%d}$' % (i+1), fontsize=12)
        ax.set_ylabel(r'$y$', fontsize=12)
    fig.tight_layout()
else:
    new_test_func = test_func


Copula families

We now consider only Gaussian dependencies for this example


In [127]:
families = np.zeros((dim, dim), dtype=int)
for i in range(1, dim):
    for j in range(i):
        families[i, j] = 1

Estimations

We create an instance of the main class for conservative estimate, and we define a q_func object for the quantile as a quantity of interest


In [128]:
from depimpact import ConservativeEstimate, quantile_func

alpha = 0.95

if alpha > 0.5: # Maximizing the quantile
    def q_func(x, axis=1):
        return - quantile_func(alpha)(x, axis=axis)
else: # Minimizing
    q_func = quantile_func(alpha)

quant_estimate = ConservativeEstimate(model_func=new_test_func, margins=margins, families=families)

In [129]:
f = lambda x: quant_estimate.stochastic_function(x, return_input_sample=False)

In [130]:
from skopt import gp_minimize

res = gp_minimize(f,                  # the function to minimize
                  [(-0.99999, 0.99999)],      # the bounds on each dimension of x
                  acq_func="LCB",      # the acquisition function
                  n_calls=310,         # the number of evaluations of f 
                  n_random_starts=300,  # the number of random initialization points
                  noise=0.5,       # the noise level (optional)
                  random_state=123,
                  verbose=True,
                  n_jobs=7)   # the random seed


Iteration No: 1 started. Evaluating function at random point.
Iteration No: 1 ended. Evaluation done at random point.
Time taken: 0.0149
Function value obtained: -3.4731
Current minimum: -3.4731
Iteration No: 2 started. Evaluating function at random point.
Iteration No: 2 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -0.9695
Current minimum: -3.4731
Iteration No: 3 started. Evaluating function at random point.
Iteration No: 3 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -2.8367
Current minimum: -3.4731
Iteration No: 4 started. Evaluating function at random point.
Iteration No: 4 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -0.6676
Current minimum: -3.4731
Iteration No: 5 started. Evaluating function at random point.
Iteration No: 5 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 4.5796
Current minimum: -3.4731
Iteration No: 6 started. Evaluating function at random point.
Iteration No: 6 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 3.1790
Current minimum: -3.4731
Iteration No: 7 started. Evaluating function at random point.
Iteration No: 7 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -4.2072
Current minimum: -4.2072
Iteration No: 8 started. Evaluating function at random point.
Iteration No: 8 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 0.5375
Current minimum: -4.2072
Iteration No: 9 started. Evaluating function at random point.
Iteration No: 9 ended. Evaluation done at random point.
Time taken: 0.0167
Function value obtained: 1.5527
Current minimum: -4.2072
Iteration No: 10 started. Evaluating function at random point.
Iteration No: 10 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -2.2889
Current minimum: -4.2072
Iteration No: 11 started. Evaluating function at random point.
Iteration No: 11 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.9077
Current minimum: -4.2072
Iteration No: 12 started. Evaluating function at random point.
Iteration No: 12 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.5051
Current minimum: -4.2072
Iteration No: 13 started. Evaluating function at random point.
Iteration No: 13 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.2617
Current minimum: -4.2072
Iteration No: 14 started. Evaluating function at random point.
Iteration No: 14 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -4.5318
Current minimum: -4.5318
Iteration No: 15 started. Evaluating function at random point.
Iteration No: 15 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.4282
Current minimum: -4.5318
Iteration No: 16 started. Evaluating function at random point.
Iteration No: 16 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.3595
Current minimum: -4.5318
Iteration No: 17 started. Evaluating function at random point.
Iteration No: 17 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 1.0908
Current minimum: -4.5318
Iteration No: 18 started. Evaluating function at random point.
Iteration No: 18 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -1.1984
Current minimum: -4.5318
Iteration No: 19 started. Evaluating function at random point.
Iteration No: 19 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -1.0468
Current minimum: -4.5318
Iteration No: 20 started. Evaluating function at random point.
Iteration No: 20 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.7690
Current minimum: -4.5318
Iteration No: 21 started. Evaluating function at random point.
Iteration No: 21 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -3.1645
Current minimum: -4.5318
Iteration No: 22 started. Evaluating function at random point.
Iteration No: 22 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 0.9159
Current minimum: -4.5318
Iteration No: 23 started. Evaluating function at random point.
Iteration No: 23 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 2.2651
Current minimum: -4.5318
Iteration No: 24 started. Evaluating function at random point.
Iteration No: 24 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.4506
Current minimum: -4.5318
Iteration No: 25 started. Evaluating function at random point.
Iteration No: 25 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.5321
Current minimum: -4.5318
Iteration No: 26 started. Evaluating function at random point.
Iteration No: 26 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 3.0533
Current minimum: -4.5318
Iteration No: 27 started. Evaluating function at random point.
Iteration No: 27 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: 1.5357
Current minimum: -4.5318
Iteration No: 28 started. Evaluating function at random point.
Iteration No: 28 ended. Evaluation done at random point.
Time taken: 0.0139
Function value obtained: 0.5695
Current minimum: -4.5318
Iteration No: 29 started. Evaluating function at random point.
Iteration No: 29 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: 0.9533
Current minimum: -4.5318
Iteration No: 30 started. Evaluating function at random point.
Iteration No: 30 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: 1.5378
Current minimum: -4.5318
Iteration No: 31 started. Evaluating function at random point.
Iteration No: 31 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: -0.3019
Current minimum: -4.5318
Iteration No: 32 started. Evaluating function at random point.
Iteration No: 32 ended. Evaluation done at random point.
Time taken: 0.0141
Function value obtained: 0.1379
Current minimum: -4.5318
Iteration No: 33 started. Evaluating function at random point.
Iteration No: 33 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: -0.4171
Current minimum: -4.5318
Iteration No: 34 started. Evaluating function at random point.
Iteration No: 34 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -2.5906
Current minimum: -4.5318
Iteration No: 35 started. Evaluating function at random point.
Iteration No: 35 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.5453
Current minimum: -4.5318
Iteration No: 36 started. Evaluating function at random point.
Iteration No: 36 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.8242
Current minimum: -4.5318
Iteration No: 37 started. Evaluating function at random point.
Iteration No: 37 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -1.2442
Current minimum: -4.5318
Iteration No: 38 started. Evaluating function at random point.
Iteration No: 38 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -2.0669
Current minimum: -4.5318
Iteration No: 39 started. Evaluating function at random point.
Iteration No: 39 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 1.1164
Current minimum: -4.5318
Iteration No: 40 started. Evaluating function at random point.
Iteration No: 40 ended. Evaluation done at random point.
Time taken: 0.0166
Function value obtained: 1.1536
Current minimum: -4.5318
Iteration No: 41 started. Evaluating function at random point.
Iteration No: 41 ended. Evaluation done at random point.
Time taken: 0.0168
Function value obtained: -0.5120
Current minimum: -4.5318
Iteration No: 42 started. Evaluating function at random point.
Iteration No: 42 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 2.2490
Current minimum: -4.5318
Iteration No: 43 started. Evaluating function at random point.
Iteration No: 43 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 1.3429
Current minimum: -4.5318
Iteration No: 44 started. Evaluating function at random point.
Iteration No: 44 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.9935
Current minimum: -4.5318
Iteration No: 45 started. Evaluating function at random point.
Iteration No: 45 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -4.7034
Current minimum: -4.7034
Iteration No: 46 started. Evaluating function at random point.
Iteration No: 46 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -0.5386
Current minimum: -4.7034
Iteration No: 47 started. Evaluating function at random point.
Iteration No: 47 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -8.7767
Current minimum: -8.7767
Iteration No: 48 started. Evaluating function at random point.
Iteration No: 48 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 2.1533
Current minimum: -8.7767
Iteration No: 49 started. Evaluating function at random point.
Iteration No: 49 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -2.8465
Current minimum: -8.7767
Iteration No: 50 started. Evaluating function at random point.
Iteration No: 50 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.6838
Current minimum: -8.7767
Iteration No: 51 started. Evaluating function at random point.
Iteration No: 51 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -1.5860
Current minimum: -8.7767
Iteration No: 52 started. Evaluating function at random point.
Iteration No: 52 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 4.4705
Current minimum: -8.7767
Iteration No: 53 started. Evaluating function at random point.
Iteration No: 53 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.7389
Current minimum: -8.7767
Iteration No: 54 started. Evaluating function at random point.
Iteration No: 54 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 3.2528
Current minimum: -8.7767
Iteration No: 55 started. Evaluating function at random point.
Iteration No: 55 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 0.9602
Current minimum: -8.7767
Iteration No: 56 started. Evaluating function at random point.
Iteration No: 56 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.3594
Current minimum: -8.7767
Iteration No: 57 started. Evaluating function at random point.
Iteration No: 57 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.3825
Current minimum: -8.7767
Iteration No: 58 started. Evaluating function at random point.
Iteration No: 58 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -0.6397
Current minimum: -8.7767
Iteration No: 59 started. Evaluating function at random point.
Iteration No: 59 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -1.4202
Current minimum: -8.7767
Iteration No: 60 started. Evaluating function at random point.
Iteration No: 60 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -0.5457
Current minimum: -8.7767
Iteration No: 61 started. Evaluating function at random point.
Iteration No: 61 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -1.2592
Current minimum: -8.7767
Iteration No: 62 started. Evaluating function at random point.
Iteration No: 62 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 1.7213
Current minimum: -8.7767
Iteration No: 63 started. Evaluating function at random point.
Iteration No: 63 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 0.2218
Current minimum: -8.7767
Iteration No: 64 started. Evaluating function at random point.
Iteration No: 64 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: -2.6094
Current minimum: -8.7767
Iteration No: 65 started. Evaluating function at random point.
Iteration No: 65 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 0.9944
Current minimum: -8.7767
Iteration No: 66 started. Evaluating function at random point.
Iteration No: 66 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -3.1239
Current minimum: -8.7767
Iteration No: 67 started. Evaluating function at random point.
Iteration No: 67 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.9384
Current minimum: -8.7767
Iteration No: 68 started. Evaluating function at random point.
Iteration No: 68 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.6504
Current minimum: -8.7767
Iteration No: 69 started. Evaluating function at random point.
Iteration No: 69 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.9419
Current minimum: -8.7767
Iteration No: 70 started. Evaluating function at random point.
Iteration No: 70 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.3719
Current minimum: -8.7767
Iteration No: 71 started. Evaluating function at random point.
Iteration No: 71 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -2.4758
Current minimum: -8.7767
Iteration No: 72 started. Evaluating function at random point.
Iteration No: 72 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -0.6933
Current minimum: -8.7767
Iteration No: 73 started. Evaluating function at random point.
Iteration No: 73 ended. Evaluation done at random point.
Time taken: 0.0168
Function value obtained: -2.5782
Current minimum: -8.7767
Iteration No: 74 started. Evaluating function at random point.
Iteration No: 74 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.1849
Current minimum: -8.7767
Iteration No: 75 started. Evaluating function at random point.
Iteration No: 75 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.6119
Current minimum: -8.7767
Iteration No: 76 started. Evaluating function at random point.
Iteration No: 76 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.5877
Current minimum: -8.7767
Iteration No: 77 started. Evaluating function at random point.
Iteration No: 77 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -1.9117
Current minimum: -8.7767
Iteration No: 78 started. Evaluating function at random point.
Iteration No: 78 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.3185
Current minimum: -8.7767
Iteration No: 79 started. Evaluating function at random point.
Iteration No: 79 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 0.5898
Current minimum: -8.7767
Iteration No: 80 started. Evaluating function at random point.
Iteration No: 80 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.4693
Current minimum: -8.7767
Iteration No: 81 started. Evaluating function at random point.
Iteration No: 81 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.1574
Current minimum: -8.7767
Iteration No: 82 started. Evaluating function at random point.
Iteration No: 82 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 1.1901
Current minimum: -8.7767
Iteration No: 83 started. Evaluating function at random point.
Iteration No: 83 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.9858
Current minimum: -8.7767
Iteration No: 84 started. Evaluating function at random point.
Iteration No: 84 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 2.3110
Current minimum: -8.7767
Iteration No: 85 started. Evaluating function at random point.
Iteration No: 85 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.8984
Current minimum: -8.7767
Iteration No: 86 started. Evaluating function at random point.
Iteration No: 86 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 4.2343
Current minimum: -8.7767
Iteration No: 87 started. Evaluating function at random point.
Iteration No: 87 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -1.1794
Current minimum: -8.7767
Iteration No: 88 started. Evaluating function at random point.
Iteration No: 88 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.4800
Current minimum: -8.7767
Iteration No: 89 started. Evaluating function at random point.
Iteration No: 89 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -1.9563
Current minimum: -8.7767
Iteration No: 90 started. Evaluating function at random point.
Iteration No: 90 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.8473
Current minimum: -8.7767
Iteration No: 91 started. Evaluating function at random point.
Iteration No: 91 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 3.1977
Current minimum: -8.7767
Iteration No: 92 started. Evaluating function at random point.
Iteration No: 92 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: -0.8568
Current minimum: -8.7767
Iteration No: 93 started. Evaluating function at random point.
Iteration No: 93 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 2.3072
Current minimum: -8.7767
Iteration No: 94 started. Evaluating function at random point.
Iteration No: 94 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 1.4240
Current minimum: -8.7767
Iteration No: 95 started. Evaluating function at random point.
Iteration No: 95 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 0.2283
Current minimum: -8.7767
Iteration No: 96 started. Evaluating function at random point.
Iteration No: 96 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: -1.2339
Current minimum: -8.7767
Iteration No: 97 started. Evaluating function at random point.
Iteration No: 97 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 1.2758
Current minimum: -8.7767
Iteration No: 98 started. Evaluating function at random point.
Iteration No: 98 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -1.1568
Current minimum: -8.7767
Iteration No: 99 started. Evaluating function at random point.
Iteration No: 99 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.8590
Current minimum: -8.7767
Iteration No: 100 started. Evaluating function at random point.
Iteration No: 100 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -5.1229
Current minimum: -8.7767
Iteration No: 101 started. Evaluating function at random point.
Iteration No: 101 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.1123
Current minimum: -8.7767
Iteration No: 102 started. Evaluating function at random point.
Iteration No: 102 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 1.2772
Current minimum: -8.7767
Iteration No: 103 started. Evaluating function at random point.
Iteration No: 103 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 1.1615
Current minimum: -8.7767
Iteration No: 104 started. Evaluating function at random point.
Iteration No: 104 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -4.4957
Current minimum: -8.7767
Iteration No: 105 started. Evaluating function at random point.
Iteration No: 105 ended. Evaluation done at random point.
Time taken: 0.0169
Function value obtained: -0.3733
Current minimum: -8.7767
Iteration No: 106 started. Evaluating function at random point.
Iteration No: 106 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 2.0832
Current minimum: -8.7767
Iteration No: 107 started. Evaluating function at random point.
Iteration No: 107 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 0.7656
Current minimum: -8.7767
Iteration No: 108 started. Evaluating function at random point.
Iteration No: 108 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -3.2269
Current minimum: -8.7767
Iteration No: 109 started. Evaluating function at random point.
Iteration No: 109 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.3660
Current minimum: -8.7767
Iteration No: 110 started. Evaluating function at random point.
Iteration No: 110 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.1743
Current minimum: -8.7767
Iteration No: 111 started. Evaluating function at random point.
Iteration No: 111 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.6058
Current minimum: -8.7767
Iteration No: 112 started. Evaluating function at random point.
Iteration No: 112 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 0.4511
Current minimum: -8.7767
Iteration No: 113 started. Evaluating function at random point.
Iteration No: 113 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -4.6663
Current minimum: -8.7767
Iteration No: 114 started. Evaluating function at random point.
Iteration No: 114 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 0.6667
Current minimum: -8.7767
Iteration No: 115 started. Evaluating function at random point.
Iteration No: 115 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.3163
Current minimum: -8.7767
Iteration No: 116 started. Evaluating function at random point.
Iteration No: 116 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -4.0814
Current minimum: -8.7767
Iteration No: 117 started. Evaluating function at random point.
Iteration No: 117 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.8841
Current minimum: -8.7767
Iteration No: 118 started. Evaluating function at random point.
Iteration No: 118 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 0.1154
Current minimum: -8.7767
Iteration No: 119 started. Evaluating function at random point.
Iteration No: 119 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 2.6091
Current minimum: -8.7767
Iteration No: 120 started. Evaluating function at random point.
Iteration No: 120 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.2155
Current minimum: -8.7767
Iteration No: 121 started. Evaluating function at random point.
Iteration No: 121 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -3.7099
Current minimum: -8.7767
Iteration No: 122 started. Evaluating function at random point.
Iteration No: 122 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.2686
Current minimum: -8.7767
Iteration No: 123 started. Evaluating function at random point.
Iteration No: 123 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 2.3220
Current minimum: -8.7767
Iteration No: 124 started. Evaluating function at random point.
Iteration No: 124 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 0.1104
Current minimum: -8.7767
Iteration No: 125 started. Evaluating function at random point.
Iteration No: 125 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -0.3050
Current minimum: -8.7767
Iteration No: 126 started. Evaluating function at random point.
Iteration No: 126 ended. Evaluation done at random point.
Time taken: 0.0138
Function value obtained: -0.2920
Current minimum: -8.7767
Iteration No: 127 started. Evaluating function at random point.
Iteration No: 127 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -1.3967
Current minimum: -8.7767
Iteration No: 128 started. Evaluating function at random point.
Iteration No: 128 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 0.3202
Current minimum: -8.7767
Iteration No: 129 started. Evaluating function at random point.
Iteration No: 129 ended. Evaluation done at random point.
Time taken: 0.0145
Function value obtained: -1.0513
Current minimum: -8.7767
Iteration No: 130 started. Evaluating function at random point.
Iteration No: 130 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.8720
Current minimum: -8.7767
Iteration No: 131 started. Evaluating function at random point.
Iteration No: 131 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -1.9627
Current minimum: -8.7767
Iteration No: 132 started. Evaluating function at random point.
Iteration No: 132 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 3.3019
Current minimum: -8.7767
Iteration No: 133 started. Evaluating function at random point.
Iteration No: 133 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.3342
Current minimum: -8.7767
Iteration No: 134 started. Evaluating function at random point.
Iteration No: 134 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -3.2985
Current minimum: -8.7767
Iteration No: 135 started. Evaluating function at random point.
Iteration No: 135 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 1.5372
Current minimum: -8.7767
Iteration No: 136 started. Evaluating function at random point.
Iteration No: 136 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.7801
Current minimum: -8.7767
Iteration No: 137 started. Evaluating function at random point.
Iteration No: 137 ended. Evaluation done at random point.
Time taken: 0.0167
Function value obtained: -0.9657
Current minimum: -8.7767
Iteration No: 138 started. Evaluating function at random point.
Iteration No: 138 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -3.5721
Current minimum: -8.7767
Iteration No: 139 started. Evaluating function at random point.
Iteration No: 139 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -3.8083
Current minimum: -8.7767
Iteration No: 140 started. Evaluating function at random point.
Iteration No: 140 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -1.1141
Current minimum: -8.7767
Iteration No: 141 started. Evaluating function at random point.
Iteration No: 141 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.2741
Current minimum: -8.7767
Iteration No: 142 started. Evaluating function at random point.
Iteration No: 142 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.4214
Current minimum: -8.7767
Iteration No: 143 started. Evaluating function at random point.
Iteration No: 143 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.4945
Current minimum: -8.7767
Iteration No: 144 started. Evaluating function at random point.
Iteration No: 144 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.2283
Current minimum: -8.7767
Iteration No: 145 started. Evaluating function at random point.
Iteration No: 145 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -1.1969
Current minimum: -8.7767
Iteration No: 146 started. Evaluating function at random point.
Iteration No: 146 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 2.3289
Current minimum: -8.7767
Iteration No: 147 started. Evaluating function at random point.
Iteration No: 147 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 2.5413
Current minimum: -8.7767
Iteration No: 148 started. Evaluating function at random point.
Iteration No: 148 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 0.7378
Current minimum: -8.7767
Iteration No: 149 started. Evaluating function at random point.
Iteration No: 149 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 2.2476
Current minimum: -8.7767
Iteration No: 150 started. Evaluating function at random point.
Iteration No: 150 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -2.0076
Current minimum: -8.7767
Iteration No: 151 started. Evaluating function at random point.
Iteration No: 151 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -2.1379
Current minimum: -8.7767
Iteration No: 152 started. Evaluating function at random point.
Iteration No: 152 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -2.6198
Current minimum: -8.7767
Iteration No: 153 started. Evaluating function at random point.
Iteration No: 153 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.0083
Current minimum: -8.7767
Iteration No: 154 started. Evaluating function at random point.
Iteration No: 154 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.2610
Current minimum: -8.7767
Iteration No: 155 started. Evaluating function at random point.
Iteration No: 155 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.9386
Current minimum: -8.7767
Iteration No: 156 started. Evaluating function at random point.
Iteration No: 156 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 1.5270
Current minimum: -8.7767
Iteration No: 157 started. Evaluating function at random point.
Iteration No: 157 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 2.6669
Current minimum: -8.7767
Iteration No: 158 started. Evaluating function at random point.
Iteration No: 158 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: 5.0729
Current minimum: -8.7767
Iteration No: 159 started. Evaluating function at random point.
Iteration No: 159 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -0.3721
Current minimum: -8.7767
Iteration No: 160 started. Evaluating function at random point.
Iteration No: 160 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 2.3810
Current minimum: -8.7767
Iteration No: 161 started. Evaluating function at random point.
Iteration No: 161 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 2.3236
Current minimum: -8.7767
Iteration No: 162 started. Evaluating function at random point.
Iteration No: 162 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.3906
Current minimum: -8.7767
Iteration No: 163 started. Evaluating function at random point.
Iteration No: 163 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.4017
Current minimum: -8.7767
Iteration No: 164 started. Evaluating function at random point.
Iteration No: 164 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 6.4008
Current minimum: -8.7767
Iteration No: 165 started. Evaluating function at random point.
Iteration No: 165 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.5855
Current minimum: -8.7767
Iteration No: 166 started. Evaluating function at random point.
Iteration No: 166 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -3.0498
Current minimum: -8.7767
Iteration No: 167 started. Evaluating function at random point.
Iteration No: 167 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.2268
Current minimum: -8.7767
Iteration No: 168 started. Evaluating function at random point.
Iteration No: 168 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 1.0634
Current minimum: -8.7767
Iteration No: 169 started. Evaluating function at random point.
Iteration No: 169 ended. Evaluation done at random point.
Time taken: 0.0168
Function value obtained: 0.4231
Current minimum: -8.7767
Iteration No: 170 started. Evaluating function at random point.
Iteration No: 170 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 1.1512
Current minimum: -8.7767
Iteration No: 171 started. Evaluating function at random point.
Iteration No: 171 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.6058
Current minimum: -8.7767
Iteration No: 172 started. Evaluating function at random point.
Iteration No: 172 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 5.0800
Current minimum: -8.7767
Iteration No: 173 started. Evaluating function at random point.
Iteration No: 173 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -1.7456
Current minimum: -8.7767
Iteration No: 174 started. Evaluating function at random point.
Iteration No: 174 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -2.8320
Current minimum: -8.7767
Iteration No: 175 started. Evaluating function at random point.
Iteration No: 175 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -0.1558
Current minimum: -8.7767
Iteration No: 176 started. Evaluating function at random point.
Iteration No: 176 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.2838
Current minimum: -8.7767
Iteration No: 177 started. Evaluating function at random point.
Iteration No: 177 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.0302
Current minimum: -8.7767
Iteration No: 178 started. Evaluating function at random point.
Iteration No: 178 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -2.3863
Current minimum: -8.7767
Iteration No: 179 started. Evaluating function at random point.
Iteration No: 179 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -1.1302
Current minimum: -8.7767
Iteration No: 180 started. Evaluating function at random point.
Iteration No: 180 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.5701
Current minimum: -8.7767
Iteration No: 181 started. Evaluating function at random point.
Iteration No: 181 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.5813
Current minimum: -8.7767
Iteration No: 182 started. Evaluating function at random point.
Iteration No: 182 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -3.0446
Current minimum: -8.7767
Iteration No: 183 started. Evaluating function at random point.
Iteration No: 183 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 0.9733
Current minimum: -8.7767
Iteration No: 184 started. Evaluating function at random point.
Iteration No: 184 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.2877
Current minimum: -8.7767
Iteration No: 185 started. Evaluating function at random point.
Iteration No: 185 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 2.5627
Current minimum: -8.7767
Iteration No: 186 started. Evaluating function at random point.
Iteration No: 186 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 1.0220
Current minimum: -8.7767
Iteration No: 187 started. Evaluating function at random point.
Iteration No: 187 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -1.6017
Current minimum: -8.7767
Iteration No: 188 started. Evaluating function at random point.
Iteration No: 188 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: -0.3373
Current minimum: -8.7767
Iteration No: 189 started. Evaluating function at random point.
Iteration No: 189 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: -1.3690
Current minimum: -8.7767
Iteration No: 190 started. Evaluating function at random point.
Iteration No: 190 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -1.2694
Current minimum: -8.7767
Iteration No: 191 started. Evaluating function at random point.
Iteration No: 191 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 1.2471
Current minimum: -8.7767
Iteration No: 192 started. Evaluating function at random point.
Iteration No: 192 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -1.8605
Current minimum: -8.7767
Iteration No: 193 started. Evaluating function at random point.
Iteration No: 193 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -0.9671
Current minimum: -8.7767
Iteration No: 194 started. Evaluating function at random point.
Iteration No: 194 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 3.5562
Current minimum: -8.7767
Iteration No: 195 started. Evaluating function at random point.
Iteration No: 195 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -2.3147
Current minimum: -8.7767
Iteration No: 196 started. Evaluating function at random point.
Iteration No: 196 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.9197
Current minimum: -8.7767
Iteration No: 197 started. Evaluating function at random point.
Iteration No: 197 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.0848
Current minimum: -8.7767
Iteration No: 198 started. Evaluating function at random point.
Iteration No: 198 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.7669
Current minimum: -8.7767
Iteration No: 199 started. Evaluating function at random point.
Iteration No: 199 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -2.0889
Current minimum: -8.7767
Iteration No: 200 started. Evaluating function at random point.
Iteration No: 200 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.9039
Current minimum: -8.7767
Iteration No: 201 started. Evaluating function at random point.
Iteration No: 201 ended. Evaluation done at random point.
Time taken: 0.0181
Function value obtained: -1.4096
Current minimum: -8.7767
Iteration No: 202 started. Evaluating function at random point.
Iteration No: 202 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.2629
Current minimum: -8.7767
Iteration No: 203 started. Evaluating function at random point.
Iteration No: 203 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -2.7960
Current minimum: -8.7767
Iteration No: 204 started. Evaluating function at random point.
Iteration No: 204 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -6.9594
Current minimum: -8.7767
Iteration No: 205 started. Evaluating function at random point.
Iteration No: 205 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.8248
Current minimum: -8.7767
Iteration No: 206 started. Evaluating function at random point.
Iteration No: 206 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -5.8917
Current minimum: -8.7767
Iteration No: 207 started. Evaluating function at random point.
Iteration No: 207 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -2.6135
Current minimum: -8.7767
Iteration No: 208 started. Evaluating function at random point.
Iteration No: 208 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 2.2294
Current minimum: -8.7767
Iteration No: 209 started. Evaluating function at random point.
Iteration No: 209 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -3.3663
Current minimum: -8.7767
Iteration No: 210 started. Evaluating function at random point.
Iteration No: 210 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -1.5338
Current minimum: -8.7767
Iteration No: 211 started. Evaluating function at random point.
Iteration No: 211 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 1.5339
Current minimum: -8.7767
Iteration No: 212 started. Evaluating function at random point.
Iteration No: 212 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -1.2708
Current minimum: -8.7767
Iteration No: 213 started. Evaluating function at random point.
Iteration No: 213 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 2.6962
Current minimum: -8.7767
Iteration No: 214 started. Evaluating function at random point.
Iteration No: 214 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -2.3170
Current minimum: -8.7767
Iteration No: 215 started. Evaluating function at random point.
Iteration No: 215 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -3.2340
Current minimum: -8.7767
Iteration No: 216 started. Evaluating function at random point.
Iteration No: 216 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 1.8171
Current minimum: -8.7767
Iteration No: 217 started. Evaluating function at random point.
Iteration No: 217 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.5949
Current minimum: -8.7767
Iteration No: 218 started. Evaluating function at random point.
Iteration No: 218 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -3.7606
Current minimum: -8.7767
Iteration No: 219 started. Evaluating function at random point.
Iteration No: 219 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 4.2026
Current minimum: -8.7767
Iteration No: 220 started. Evaluating function at random point.
Iteration No: 220 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: 0.2677
Current minimum: -8.7767
Iteration No: 221 started. Evaluating function at random point.
Iteration No: 221 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -2.4770
Current minimum: -8.7767
Iteration No: 222 started. Evaluating function at random point.
Iteration No: 222 ended. Evaluation done at random point.
Time taken: 0.0139
Function value obtained: 1.0679
Current minimum: -8.7767
Iteration No: 223 started. Evaluating function at random point.
Iteration No: 223 ended. Evaluation done at random point.
Time taken: 0.0148
Function value obtained: -1.0024
Current minimum: -8.7767
Iteration No: 224 started. Evaluating function at random point.
Iteration No: 224 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -0.3767
Current minimum: -8.7767
Iteration No: 225 started. Evaluating function at random point.
Iteration No: 225 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: 0.9953
Current minimum: -8.7767
Iteration No: 226 started. Evaluating function at random point.
Iteration No: 226 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -0.4704
Current minimum: -8.7767
Iteration No: 227 started. Evaluating function at random point.
Iteration No: 227 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 1.1358
Current minimum: -8.7767
Iteration No: 228 started. Evaluating function at random point.
Iteration No: 228 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 0.6568
Current minimum: -8.7767
Iteration No: 229 started. Evaluating function at random point.
Iteration No: 229 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -2.1712
Current minimum: -8.7767
Iteration No: 230 started. Evaluating function at random point.
Iteration No: 230 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.4526
Current minimum: -8.7767
Iteration No: 231 started. Evaluating function at random point.
Iteration No: 231 ended. Evaluation done at random point.
Time taken: 0.0137
Function value obtained: -1.0064
Current minimum: -8.7767
Iteration No: 232 started. Evaluating function at random point.
Iteration No: 232 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: -1.1793
Current minimum: -8.7767
Iteration No: 233 started. Evaluating function at random point.
Iteration No: 233 ended. Evaluation done at random point.
Time taken: 0.0161
Function value obtained: 0.4251
Current minimum: -8.7767
Iteration No: 234 started. Evaluating function at random point.
Iteration No: 234 ended. Evaluation done at random point.
Time taken: 0.0138
Function value obtained: 1.6837
Current minimum: -8.7767
Iteration No: 235 started. Evaluating function at random point.
Iteration No: 235 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 0.1729
Current minimum: -8.7767
Iteration No: 236 started. Evaluating function at random point.
Iteration No: 236 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 0.0853
Current minimum: -8.7767
Iteration No: 237 started. Evaluating function at random point.
Iteration No: 237 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 2.2437
Current minimum: -8.7767
Iteration No: 238 started. Evaluating function at random point.
Iteration No: 238 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -2.4314
Current minimum: -8.7767
Iteration No: 239 started. Evaluating function at random point.
Iteration No: 239 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 2.8191
Current minimum: -8.7767
Iteration No: 240 started. Evaluating function at random point.
Iteration No: 240 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.0670
Current minimum: -8.7767
Iteration No: 241 started. Evaluating function at random point.
Iteration No: 241 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.6931
Current minimum: -8.7767
Iteration No: 242 started. Evaluating function at random point.
Iteration No: 242 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -2.8500
Current minimum: -8.7767
Iteration No: 243 started. Evaluating function at random point.
Iteration No: 243 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -2.5360
Current minimum: -8.7767
Iteration No: 244 started. Evaluating function at random point.
Iteration No: 244 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -0.8070
Current minimum: -8.7767
Iteration No: 245 started. Evaluating function at random point.
Iteration No: 245 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 1.7746
Current minimum: -8.7767
Iteration No: 246 started. Evaluating function at random point.
Iteration No: 246 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.0792
Current minimum: -8.7767
Iteration No: 247 started. Evaluating function at random point.
Iteration No: 247 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 3.2688
Current minimum: -8.7767
Iteration No: 248 started. Evaluating function at random point.
Iteration No: 248 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -0.7932
Current minimum: -8.7767
Iteration No: 249 started. Evaluating function at random point.
Iteration No: 249 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.7724
Current minimum: -8.7767
Iteration No: 250 started. Evaluating function at random point.
Iteration No: 250 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: 0.5986
Current minimum: -8.7767
Iteration No: 251 started. Evaluating function at random point.
Iteration No: 251 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 2.1747
Current minimum: -8.7767
Iteration No: 252 started. Evaluating function at random point.
Iteration No: 252 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: -2.0975
Current minimum: -8.7767
Iteration No: 253 started. Evaluating function at random point.
Iteration No: 253 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 0.8911
Current minimum: -8.7767
Iteration No: 254 started. Evaluating function at random point.
Iteration No: 254 ended. Evaluation done at random point.
Time taken: 0.0136
Function value obtained: -0.8113
Current minimum: -8.7767
Iteration No: 255 started. Evaluating function at random point.
Iteration No: 255 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -0.1630
Current minimum: -8.7767
Iteration No: 256 started. Evaluating function at random point.
Iteration No: 256 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -0.5761
Current minimum: -8.7767
Iteration No: 257 started. Evaluating function at random point.
Iteration No: 257 ended. Evaluation done at random point.
Time taken: 0.0140
Function value obtained: -1.1710
Current minimum: -8.7767
Iteration No: 258 started. Evaluating function at random point.
Iteration No: 258 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.5606
Current minimum: -8.7767
Iteration No: 259 started. Evaluating function at random point.
Iteration No: 259 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 2.8369
Current minimum: -8.7767
Iteration No: 260 started. Evaluating function at random point.
Iteration No: 260 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -1.4375
Current minimum: -8.7767
Iteration No: 261 started. Evaluating function at random point.
Iteration No: 261 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 4.0409
Current minimum: -8.7767
Iteration No: 262 started. Evaluating function at random point.
Iteration No: 262 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.2575
Current minimum: -8.7767
Iteration No: 263 started. Evaluating function at random point.
Iteration No: 263 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -1.0480
Current minimum: -8.7767
Iteration No: 264 started. Evaluating function at random point.
Iteration No: 264 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -1.7864
Current minimum: -8.7767
Iteration No: 265 started. Evaluating function at random point.
Iteration No: 265 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -0.3915
Current minimum: -8.7767
Iteration No: 266 started. Evaluating function at random point.
Iteration No: 266 ended. Evaluation done at random point.
Time taken: 0.0173
Function value obtained: 2.7219
Current minimum: -8.7767
Iteration No: 267 started. Evaluating function at random point.
Iteration No: 267 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: 0.0810
Current minimum: -8.7767
Iteration No: 268 started. Evaluating function at random point.
Iteration No: 268 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.6559
Current minimum: -8.7767
Iteration No: 269 started. Evaluating function at random point.
Iteration No: 269 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -0.2995
Current minimum: -8.7767
Iteration No: 270 started. Evaluating function at random point.
Iteration No: 270 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: -4.0883
Current minimum: -8.7767
Iteration No: 271 started. Evaluating function at random point.
Iteration No: 271 ended. Evaluation done at random point.
Time taken: 0.0126
Function value obtained: 2.2365
Current minimum: -8.7767
Iteration No: 272 started. Evaluating function at random point.
Iteration No: 272 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.3391
Current minimum: -8.7767
Iteration No: 273 started. Evaluating function at random point.
Iteration No: 273 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -1.0476
Current minimum: -8.7767
Iteration No: 274 started. Evaluating function at random point.
Iteration No: 274 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 1.3155
Current minimum: -8.7767
Iteration No: 275 started. Evaluating function at random point.
Iteration No: 275 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 2.7933
Current minimum: -8.7767
Iteration No: 276 started. Evaluating function at random point.
Iteration No: 276 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.9564
Current minimum: -8.7767
Iteration No: 277 started. Evaluating function at random point.
Iteration No: 277 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 1.4282
Current minimum: -8.7767
Iteration No: 278 started. Evaluating function at random point.
Iteration No: 278 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: -2.3996
Current minimum: -8.7767
Iteration No: 279 started. Evaluating function at random point.
Iteration No: 279 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 1.9969
Current minimum: -8.7767
Iteration No: 280 started. Evaluating function at random point.
Iteration No: 280 ended. Evaluation done at random point.
Time taken: 0.0132
Function value obtained: 4.8617
Current minimum: -8.7767
Iteration No: 281 started. Evaluating function at random point.
Iteration No: 281 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: 1.6679
Current minimum: -8.7767
Iteration No: 282 started. Evaluating function at random point.
Iteration No: 282 ended. Evaluation done at random point.
Time taken: 0.0127
Function value obtained: 4.3280
Current minimum: -8.7767
Iteration No: 283 started. Evaluating function at random point.
Iteration No: 283 ended. Evaluation done at random point.
Time taken: 0.0128
Function value obtained: -2.3997
Current minimum: -8.7767
Iteration No: 284 started. Evaluating function at random point.
Iteration No: 284 ended. Evaluation done at random point.
Time taken: 0.0131
Function value obtained: -2.4931
Current minimum: -8.7767
Iteration No: 285 started. Evaluating function at random point.
Iteration No: 285 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: 5.8897
Current minimum: -8.7767
Iteration No: 286 started. Evaluating function at random point.
Iteration No: 286 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -0.4081
Current minimum: -8.7767
Iteration No: 287 started. Evaluating function at random point.
Iteration No: 287 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: -6.5036
Current minimum: -8.7767
Iteration No: 288 started. Evaluating function at random point.
Iteration No: 288 ended. Evaluation done at random point.
Time taken: 0.0133
Function value obtained: -1.5661
Current minimum: -8.7767
Iteration No: 289 started. Evaluating function at random point.
Iteration No: 289 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 2.7053
Current minimum: -8.7767
Iteration No: 290 started. Evaluating function at random point.
Iteration No: 290 ended. Evaluation done at random point.
Time taken: 0.0134
Function value obtained: 3.1597
Current minimum: -8.7767
Iteration No: 291 started. Evaluating function at random point.
Iteration No: 291 ended. Evaluation done at random point.
Time taken: 0.0176
Function value obtained: -0.7195
Current minimum: -8.7767
Iteration No: 292 started. Evaluating function at random point.
Iteration No: 292 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 2.0569
Current minimum: -8.7767
Iteration No: 293 started. Evaluating function at random point.
Iteration No: 293 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: 2.3604
Current minimum: -8.7767
Iteration No: 294 started. Evaluating function at random point.
Iteration No: 294 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -0.5338
Current minimum: -8.7767
Iteration No: 295 started. Evaluating function at random point.
Iteration No: 295 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -4.2057
Current minimum: -8.7767
Iteration No: 296 started. Evaluating function at random point.
Iteration No: 296 ended. Evaluation done at random point.
Time taken: 0.0129
Function value obtained: -1.1835
Current minimum: -8.7767
Iteration No: 297 started. Evaluating function at random point.
Iteration No: 297 ended. Evaluation done at random point.
Time taken: 0.0130
Function value obtained: -1.2808
Current minimum: -8.7767
Iteration No: 298 started. Evaluating function at random point.
Iteration No: 298 ended. Evaluation done at random point.
Time taken: 0.0170
Function value obtained: 1.3746
Current minimum: -8.7767
Iteration No: 299 started. Evaluating function at random point.
Iteration No: 299 ended. Evaluation done at random point.
Time taken: 0.0135
Function value obtained: 2.6856
Current minimum: -8.7767
Iteration No: 300 started. Evaluating function at random point.
Iteration No: 300 ended. Evaluation done at random point.
Time taken: 5.0082
Function value obtained: 0.2106
Current minimum: -8.7767
Iteration No: 301 started. Searching for the next optimal point.
Iteration No: 301 ended. Search finished for the next optimal point.
Time taken: 4.8650
Function value obtained: 0.0547
Current minimum: -8.7767
Iteration No: 302 started. Searching for the next optimal point.
Iteration No: 302 ended. Search finished for the next optimal point.
Time taken: 4.7914
Function value obtained: 2.7636
Current minimum: -8.7767
Iteration No: 303 started. Searching for the next optimal point.
Iteration No: 303 ended. Search finished for the next optimal point.
Time taken: 4.9635
Function value obtained: -0.8986
Current minimum: -8.7767
Iteration No: 304 started. Searching for the next optimal point.
Iteration No: 304 ended. Search finished for the next optimal point.
Time taken: 4.8729
Function value obtained: -2.5383
Current minimum: -8.7767
Iteration No: 305 started. Searching for the next optimal point.
Iteration No: 305 ended. Search finished for the next optimal point.
Time taken: 5.0367
Function value obtained: -6.0096
Current minimum: -8.7767
Iteration No: 306 started. Searching for the next optimal point.
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-130-89ecf5482cfb> in <module>()
      9                   random_state=123,
     10                   verbose=True,
---> 11                   n_jobs=7)   # the random seed

/local00/home/D58174/git-repo/scikit-optimize/skopt/optimizer/gp.py in gp_minimize(func, dimensions, base_estimator, n_calls, n_random_starts, acq_func, acq_optimizer, x0, y0, random_state, verbose, callback, n_points, n_restarts_optimizer, xi, kappa, noise, n_jobs)
    219         n_restarts_optimizer=n_restarts_optimizer,
    220         x0=x0, y0=y0, random_state=rng, verbose=verbose,
--> 221         callback=callback, n_jobs=n_jobs)

/local00/home/D58174/git-repo/scikit-optimize/skopt/optimizer/base.py in base_minimize(func, dimensions, base_estimator, n_calls, n_random_starts, acq_func, acq_optimizer, x0, y0, random_state, verbose, callback, n_points, n_restarts_optimizer, xi, kappa, n_jobs)
    247 
    248         next_y = func(next_x)
--> 249         result = optimizer.tell(next_x, next_y)
    250         result.specs = specs
    251 

/local00/home/D58174/git-repo/scikit-optimize/skopt/optimizer/optimizer.py in tell(self, x, y, fit)
    465                     X=X, model=est, y_opt=np.min(self.yi),
    466                     acq_func=cand_acq_func,
--> 467                     acq_func_kwargs=self.acq_func_kwargs)
    468                 # Find the minimum of the acquisition function by randomly
    469                 # sampling points from the space

/local00/home/D58174/git-repo/scikit-optimize/skopt/acquisition.py in _gaussian_acquisition(X, model, y_opt, acq_func, return_grad, acq_func_kwargs)
     40 
     41     if acq_func == "LCB":
---> 42         func_and_grad = gaussian_lcb(X, model, kappa, return_grad)
     43         if return_grad:
     44             acq_vals, acq_grad = func_and_grad

/local00/home/D58174/git-repo/scikit-optimize/skopt/acquisition.py in gaussian_lcb(X, model, kappa, return_grad)
    140 
    141         else:
--> 142             mu, std = model.predict(X, return_std=True)
    143             if kappa == "inf":
    144                 return -std

/local00/home/D58174/git-repo/scikit-optimize/skopt/learning/gaussian_process/gpr.py in predict(self, X, return_std, return_cov, return_mean_grad, return_std_grad)
    322                 # Compute variance of predictive distribution
    323                 y_var = self.kernel_.diag(X)
--> 324                 y_var -= np.einsum("ki,kj,ij->k", K_trans, K_trans, K_inv)
    325 
    326                 # Check if any of the variances is negative because of

~/anaconda/lib/python3.6/site-packages/numpy/core/einsumfunc.py in einsum(*operands, **kwargs)
    946     # If no optimization, run pure einsum
    947     if optimize_arg is False:
--> 948         return c_einsum(*operands, **kwargs)
    949 
    950     valid_einsum_kwargs = ['out', 'dtype', 'order', 'casting']

KeyboardInterrupt: 

In [131]:
"x^*=%.4f, f(x^*)=%.4f" % (res.x[0], res.fun)


Out[131]:
'x^*=0.6403, f(x^*)=-2.5847'

In [81]:
from skopt.acquisition import gaussian_ei

fig, ax = plt.subplots()
noise_level = 0.

x = np.linspace(-0.999, 0.999, 400).reshape(-1, 1)
x_gp = res.space.transform(x.tolist())
fx = np.array([f(x_i) for x_i in x])

# Plot the 5 iterations following the 5 random points
for n_iter in range(5):
    gp = res.models[n_iter]
    curr_x_iters = res.x_iters[:5+n_iter]
    curr_func_vals = res.func_vals[:5+n_iter]

    # Plot true function.
    plt.subplot(5, 2, 2*n_iter+1)
    #plt.plot(x, fx, "r--", label="True (unknown)")
    plt.fill(np.concatenate([x, x[::-1]]),
             np.concatenate([fx - 1.9600 * noise_level, 
                             fx[::-1] + 1.9600 * noise_level]),
             alpha=.2, fc="r", ec="None")

    # Plot GP(x) + contours
    y_pred, sigma = gp.predict(x_gp, return_std=True)
    plt.plot(x, y_pred, "g--", label=r"$\mu_{GP}(x)$")
    plt.fill(np.concatenate([x, x[::-1]]),
             np.concatenate([y_pred - 1.9600 * sigma, 
                             (y_pred + 1.9600 * sigma)[::-1]]),
             alpha=.2, fc="g", ec="None")

    # Plot sampled points
    plt.plot(curr_x_iters, curr_func_vals,
             "r.", markersize=8, label="Observations")
    
    # Adjust plot layout
    plt.grid()

    if n_iter == 0:
        plt.legend(loc="best", prop={'size': 6}, numpoints=1)
        
    if n_iter != 4:
        plt.tick_params(axis='x', which='both', bottom='off', 
                        top='off', labelbottom='off') 

    # Plot EI(x)
    plt.subplot(5, 2, 2*n_iter+2)
    acq = gaussian_ei(x_gp, gp, y_opt=np.min(curr_func_vals))
    plt.plot(x, acq, "b", label="EI(x)")
    plt.fill_between(x.ravel(), -2.0, acq.ravel(), alpha=0.3, color='blue')
    
    next_x = res.x_iters[5+n_iter]
    next_acq = gaussian_ei(res.space.transform([next_x]), gp, y_opt=np.min(curr_func_vals))
    plt.plot(next_x, next_acq, "bo", markersize=6, label="Next query point")
    
    # Adjust plot layout
    #plt.ylim(0, 0.1)
    plt.grid()
    
    if n_iter == 0:
        plt.legend(loc="best", prop={'size': 6}, numpoints=1)
        
    if n_iter != 4:
        plt.tick_params(axis='x', which='both', bottom='off', 
                        top='off', labelbottom='off') 

plt.show()



In [132]:
from bayes_opt import BayesianOptimization

In [133]:
bo = BayesianOptimization(f,{'x': (-0.999, 0.999)})
bo.maximize(init_points=200, n_iter=2, kappa=2)


Initialization
-----------------------------------------
 Step |   Time |      Value |         x | 
    1 | 00m00s |    0.29787 |   -0.0635 | 
    2 | 00m00s |    2.52044 |    0.5934 | 
    3 | 00m00s |    0.18079 |   -0.5970 | 
    4 | 00m00s |    0.38684 |   -0.4398 | 
    5 | 00m00s |   -5.01662 |    0.7260 | 
    6 | 00m00s |   -0.45977 |    0.8992 | 
    7 | 00m00s |   -3.68628 |   -0.4265 | 
    8 | 00m00s |    0.93125 |    0.2443 | 
    9 | 00m00s |   -1.85152 |    0.8917 | 
   10 | 00m00s |    0.93541 |    0.3559 | 
   11 | 00m00s |    2.29849 |   -0.7084 | 
   12 | 00m00s |   -1.19243 |   -0.0936 | 
   13 | 00m00s |    0.06835 |   -0.4205 | 
   14 | 00m00s |    2.30190 |    0.3985 | 
   15 | 00m00s |   -3.13619 |    0.3280 | 
   16 | 00m00s |    0.87002 |    0.7338 | 
   17 | 00m00s |    1.01293 |   -0.3159 | 
   18 | 00m00s |    0.56705 |    0.5853 | 
   19 | 00m00s |    1.28552 |   -0.7176 | 
   20 | 00m00s |    0.22036 |   -0.6169 | 
   21 | 00m00s |    2.53259 |   -0.4154 | 
   22 | 00m00s |   -1.24683 |   -0.6706 | 
   23 | 00m00s |   -1.59939 |   -0.7221 | 
   24 | 00m00s |    1.62373 |    0.6639 | 
   25 | 00m00s |   -0.18785 |    0.3857 | 
   26 | 00m00s |    2.32508 |   -0.4162 | 
   27 | 00m00s |    4.60386 |    0.2954 | 
   28 | 00m00s |    2.40087 |    0.3053 | 
   29 | 00m00s |    0.37289 |    0.9497 | 
   30 | 00m00s |    2.56877 |   -0.8320 | 
   31 | 00m00s |   -0.63034 |   -0.7685 | 
   32 | 00m00s |    1.37997 |   -0.5358 | 
   33 | 00m00s |    0.66575 |   -0.9533 | 
   34 | 00m00s |    0.65055 |    0.3898 | 
   35 | 00m00s |   -2.69205 |    0.0578 | 
   36 | 00m00s |    1.07618 |    0.6224 | 
   37 | 00m00s |    1.34343 |    0.8343 | 
   38 | 00m00s |   -4.42247 |    0.2154 | 
   39 | 00m00s |    0.02188 |   -0.4918 | 
   40 | 00m00s |    1.85265 |    0.5490 | 
   41 | 00m00s |   -1.19532 |    0.7429 | 
   42 | 00m00s |   -0.56922 |   -0.4834 | 
   43 | 00m00s |    0.57931 |   -0.9104 | 
   44 | 00m00s |    2.06707 |    0.1398 | 
   45 | 00m00s |    4.48244 |   -0.0128 | 
   46 | 00m00s |    1.21945 |    0.8172 | 
   47 | 00m00s |    1.17967 |   -0.0005 | 
   48 | 00m00s |    1.04767 |    0.7769 | 
   49 | 00m00s |    0.26998 |    0.6470 | 
   50 | 00m00s |    1.97589 |   -0.5895 | 
   51 | 00m00s |    3.87961 |   -0.6768 | 
   52 | 00m00s |   -0.23493 |   -0.5940 | 
   53 | 00m00s |    2.66619 |   -0.7992 | 
   54 | 00m00s |    3.91743 |    0.8558 | 
   55 | 00m00s |   -0.45323 |    0.1366 | 
   56 | 00m00s |    1.24676 |   -0.6969 | 
   57 | 00m00s |   -0.61455 |   -0.5159 | 
   58 | 00m00s |    0.23769 |   -0.8758 | 
   59 | 00m00s |    2.10498 |   -0.7111 | 
   60 | 00m00s |   -0.23516 |    0.0065 | 
   61 | 00m00s |   -1.29357 |   -0.2972 | 
   62 | 00m00s |    1.77719 |    0.5621 | 
   63 | 00m00s |   -1.91319 |   -0.0144 | 
   64 | 00m00s |   -2.12472 |   -0.0916 | 
   65 | 00m00s |   -3.31230 |   -0.0793 | 
   66 | 00m00s |   -1.04458 |    0.5556 | 
   67 | 00m00s |    0.03202 |   -0.9076 | 
   68 | 00m00s |   -0.66320 |   -0.4977 | 
   69 | 00m00s |    2.86557 |    0.4877 | 
   70 | 00m00s |    2.19081 |   -0.1212 | 
   71 | 00m00s |    0.49678 |    0.8572 | 
   72 | 00m00s |    5.84984 |    0.2691 | 
   73 | 00m00s |    1.14568 |   -0.0291 | 
   74 | 00m00s |   -2.00610 |    0.6691 | 
   75 | 00m00s |   -0.17072 |   -0.4706 | 
   76 | 00m00s |   -1.82818 |   -0.8718 | 
   77 | 00m00s |    1.65964 |    0.5296 | 
   78 | 00m00s |   -1.39628 |   -0.6357 | 
   79 | 00m00s |   -1.09441 |    0.4175 | 
   80 | 00m00s |   -0.72023 |   -0.7857 | 
   81 | 00m00s |    2.08426 |    0.5593 | 
   82 | 00m00s |    4.21832 |    0.2715 | 
   83 | 00m00s |   -1.27825 |   -0.5072 | 
   84 | 00m00s |    1.08377 |   -0.5291 | 
   85 | 00m00s |    0.46650 |   -0.4906 | 
   86 | 00m00s |   -0.93583 |    0.2992 | 
   87 | 00m00s |    2.04761 |    0.0625 | 
   88 | 00m00s |   -1.80647 |   -0.6437 | 
   89 | 00m00s |    0.16505 |    0.8477 | 
   90 | 00m00s |    2.56042 |    0.6623 | 
   91 | 00m00s |    3.06389 |   -0.3075 | 
   92 | 00m00s |    0.58889 |    0.2506 | 
   93 | 00m00s |   -1.74998 |   -0.9410 | 
   94 | 00m00s |   -1.41392 |    0.0737 | 
   95 | 00m00s |    1.65658 |   -0.0253 | 
   96 | 00m00s |   -1.32104 |    0.9200 | 
   97 | 00m00s |    4.21428 |    0.0985 | 
   98 | 00m00s |    1.76528 |    0.0778 | 
   99 | 00m00s |   -0.46372 |    0.2679 | 
  100 | 00m00s |    0.15807 |   -0.6272 | 
  101 | 00m00s |    3.06252 |   -0.2225 | 
  102 | 00m00s |    0.59878 |   -0.6258 | 
  103 | 00m00s |   -1.62989 |   -0.9094 | 
  104 | 00m00s |    2.38012 |    0.5811 | 
  105 | 00m00s |    1.34722 |    0.9597 | 
  106 | 00m00s |   -2.64980 |    0.8314 | 
  107 | 00m00s |    0.71996 |    0.3954 | 
  108 | 00m00s |    1.64390 |   -0.3063 | 
  109 | 00m00s |    0.98947 |   -0.2365 | 
  110 | 00m00s |   -0.64693 |    0.1618 | 
  111 | 00m00s |   -0.13334 |   -0.5304 | 
  112 | 00m00s |    0.12628 |   -0.0660 | 
  113 | 00m00s |    0.08597 |    0.1834 | 
  114 | 00m00s |    0.53596 |   -0.5729 | 
  115 | 00m00s |   -1.80686 |    0.1943 | 
  116 | 00m00s |    0.37169 |    0.0276 | 
  117 | 00m00s |   -1.18330 |    0.8577 | 
  118 | 00m00s |   -2.21853 |    0.5197 | 
  119 | 00m00s |   -0.37995 |    0.2660 | 
  120 | 00m00s |    2.03271 |   -0.2523 | 
  121 | 00m00s |   -0.71076 |   -0.6761 | 
  122 | 00m00s |   -1.01262 |    0.7329 | 
  123 | 00m00s |   -1.03709 |    0.7095 | 
  124 | 00m00s |    0.54570 |    0.3193 | 
  125 | 00m00s |   -1.13992 |    0.7087 | 
  126 | 00m00s |   -3.41371 |   -0.0557 | 
  127 | 00m00s |    0.95061 |   -0.3909 | 
  128 | 00m00s |    0.67616 |   -0.9662 | 
  129 | 00m00s |   -0.28318 |    0.5747 | 
  130 | 00m00s |   -1.92706 |   -0.2758 | 
  131 | 00m00s |    3.97100 |    0.0126 | 
  132 | 00m00s |   -3.05985 |   -0.5786 | 
  133 | 00m00s |   -0.93395 |    0.4620 | 
  134 | 00m00s |    2.43760 |   -0.1428 | 
  135 | 00m00s |   -1.19761 |    0.1985 | 
  136 | 00m00s |   -0.35285 |    0.0197 | 
  137 | 00m00s |   -1.74753 |   -0.3548 | 
  138 | 00m00s |    3.28758 |   -0.8226 | 
  139 | 00m00s |    1.17585 |    0.6103 | 
  140 | 00m00s |   -0.45660 |   -0.7804 | 
  141 | 00m00s |    2.86783 |    0.6440 | 
  142 | 00m00s |   -1.84910 |    0.8621 | 
  143 | 00m00s |    1.44255 |   -0.4593 | 
  144 | 00m00s |   -0.69594 |   -0.0729 | 
  145 | 00m00s |   -1.02494 |   -0.3561 | 
  146 | 00m00s |    0.86847 |    0.2194 | 
  147 | 00m00s |    1.75716 |    0.4261 | 
  148 | 00m00s |    2.79950 |   -0.2876 | 
  149 | 00m00s |    0.66661 |   -0.2666 | 
  150 | 00m00s |    2.13952 |   -0.3994 | 
  151 | 00m00s |   -3.57985 |   -0.1460 | 
  152 | 00m00s |    2.37730 |    0.1232 | 
  153 | 00m00s |   -4.62085 |    0.9737 | 
  154 | 00m00s |    4.20855 |    0.3059 | 
  155 | 00m00s |    1.53648 |    0.9530 | 
  156 | 00m00s |   -2.69178 |    0.6880 | 
  157 | 00m00s |    1.13780 |    0.8729 | 
  158 | 00m00s |   -0.43233 |   -0.7872 | 
  159 | 00m00s |   -0.14325 |   -0.3958 | 
  160 | 00m00s |   -4.40021 |   -0.0077 | 
  161 | 00m00s |   -0.04378 |   -0.1826 | 
  162 | 00m00s |    0.25018 |   -0.6844 | 
  163 | 00m00s |   -1.45361 |   -0.2018 | 
  164 | 00m00s |    3.46772 |    0.9159 | 
  165 | 00m00s |   -1.78893 |    0.2342 | 
  166 | 00m00s |    5.49416 |    0.7953 | 
  167 | 00m00s |   -5.64079 |    0.7929 | 
  168 | 00m00s |    1.33535 |   -0.5061 | 
  169 | 00m00s |    3.74247 |    0.8827 | 
  170 | 00m00s |   -0.03111 |   -0.1351 | 
  171 | 00m00s |    1.21118 |    0.7266 | 
  172 | 00m00s |   -2.93237 |    0.1715 | 
  173 | 00m00s |    0.00485 |   -0.8181 | 
  174 | 00m00s |   -1.77405 |    0.9892 | 
  175 | 00m00s |    0.64400 |   -0.1809 | 
  176 | 00m00s |   -1.01465 |   -0.7256 | 
  177 | 00m00s |    3.33638 |    0.3078 | 
  178 | 00m00s |    5.74471 |    0.7242 | 
  179 | 00m00s |    0.33093 |   -0.4748 | 
  180 | 00m00s |   -2.91998 |    0.2271 | 
  181 | 00m00s |    0.78682 |   -0.5821 | 
  182 | 00m00s |    0.55245 |   -0.2140 | 
  183 | 00m00s |    4.73763 |    0.5169 | 
  184 | 00m00s |   -0.80745 |   -0.5216 | 
  185 | 00m00s |    0.20675 |    0.3617 | 
  186 | 00m00s |   -2.62386 |   -0.8427 | 
  187 | 00m00s |   -0.65785 |    0.5870 | 
  188 | 00m00s |   -1.53076 |   -0.4091 | 
  189 | 00m00s |   -1.00318 |    0.7548 | 
  190 | 00m00s |    0.87370 |   -0.6654 | 
  191 | 00m00s |   -4.05973 |   -0.0239 | 
  192 | 00m00s |   -2.41043 |   -0.2583 | 
  193 | 00m00s |   -0.15260 |   -0.2688 | 
  194 | 00m00s |   -1.90120 |   -0.3767 | 
  195 | 00m00s |    3.32751 |   -0.2645 | 
  196 | 00m00s |    0.67149 |    0.1880 | 
  197 | 00m00s |    0.77341 |   -0.2074 | 
  198 | 00m00s |   -4.15746 |    0.6320 | 
  199 | 00m00s |   -2.50489 |    0.2296 | 
  200 | 00m00s |   -2.03068 |    0.4629 | 
Bayesian Optimization
-----------------------------------------
 Step |   Time |      Value |         x | 
  201 | 00m14s |   -2.51361 |    0.2691 | 
  202 | 00m03s |   -3.54191 |    0.7242 | 

In [134]:
from skgarden import RandomForestQuantileRegressor
rfq = RandomForestQuantileRegressor()
x_plot = np.linspace(-0.99, 0.99, 100)[:, np.newaxis]
rfq.fit(bo.X, bo.Y)
q_pred = rfq.predict(x_plot, quantile=95)
plt.plot(bo.X, bo.Y, '.')
plt.plot(x_plot, q_pred)


Out[134]:
[<matplotlib.lines.Line2D at 0x7f1e5c825748>]

First, we compute the quantile at independence


In [66]:
n = 10000
indep_result = quant_estimate.independence(n_input_sample=n, q_func=q_func, random_state=random_state)

In [67]:
indep_result.compute_bootstrap(1000)
boot_std = indep_result.bootstrap_sample.std()
boot_mean = indep_result.bootstrap_sample.mean()
print('Quantile at independence: %.2f with a C.O.V at %.1f %%' % (boot_mean, abs(boot_std/boot_mean)*100.))


Quantile at independence: -1.63 with a C.O.V at 1.3 %

In [68]:
algorithm_parameters = {
    "n_input_sample": 10000,
    "n_dep_param_init": 10,
    "max_n_pairs": 1,
    "grid_type": 'lhs',
    "q_func": q_func,
    "n_add_pairs": 1,
    "n_remove_pairs": 0,
    "adapt_vine_structure": True,
    "with_bootstrap": False,
    "verbose": False,
    "iterative_save": False,
    "iterative_load": False,
    "load_input_samples": False,
    "keep_input_samples": False
}

quant_estimate = ConservativeEstimate(model_func=new_test_func, margins=margins, families=families)

iterative_results = iterative_vine_minimize(estimate_object=quant_estimate, **algorithm_parameters)


---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-68-2a678ea611a8> in <module>()
     18 quant_estimate = ConservativeEstimate(model_func=new_test_func, margins=margins, families=families)
     19 
---> 20 iterative_results = iterative_vine_minimize(estimate_object=quant_estimate, **algorithm_parameters)

/local00/home/D58174/git-repo/dep-impact/dependence/iterative_vines.py in iterative_vine_minimize(estimate_object, n_input_sample, n_dep_param_init, max_n_pairs, grid_type, q_func, n_add_pairs, n_remove_pairs, adapt_vine_structure, delta, with_bootstrap, verbose, **kwargs)
     32     assert grid_type in GRIDS, "Unknow Grid type {0}".format(grid_type)
     33     assert 0 < max_n_pairs <= corr_dim, "Maximum number of pairs must be positive"
---> 34     assert 1 <= n_add_pairs < corr_dim, "Must add at least one pair at each iteration"
     35     assert 0 <= n_remove_pairs < corr_dim, "This cannot be negative"
     36     assert callable(q_func), "Quantity function must be callable"

AssertionError: Must add at least one pair at each iteration

In [69]:
from depimpact.dependence_plot import matrix_plot_quantities
results = iterative_results[0]
matrix_plot_quantities(results, figsize=(18, 15))
# plt.savefig('output/matrix_plot.png')


---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-69-087191c2d7ab> in <module>()
      1 from dependence.dependence_plot import matrix_plot_quantities
----> 2 results = iterative_results[0]
      3 matrix_plot_quantities(results, figsize=(18, 15))
      4 # plt.savefig('output/matrix_plot.png')

NameError: name 'iterative_results' is not defined

Iterative Approach

Now lets see how good we can be with the iterative appraoch.


In [20]:
algorithm_parameters = {
    "n_input_sample": 10000,
    "n_dep_param_init": 20,
    "max_n_pairs": 3,
    "grid_type": 'vertices',
    "q_func": q_func,
    "n_add_pairs": 2,
    "n_remove_pairs": 7,
    "adapt_vine_structure": False,
    "with_bootstrap": False,
    "verbose": True,
    "iterative_save": False,
    "iterative_load": False,
    "load_input_samples": False,
    "keep_input_samples": False,
    "input_names": var_names_overflow
}

quant_estimate = ConservativeEstimate(model_func=new_test_func, margins=margins, families=families)

iterative_results = iterative_vine_minimize(estimate_object=quant_estimate, **algorithm_parameters)


n=10000. Worst quantile of [(1, 0)] at -1.8365542554541638
The variables are: K_s-Q
n=10000. Worst quantile of [(2, 0)] at -1.8245302264434484
The variables are: Z_v-Q
n=10000. Worst quantile of [(2, 1)] at -1.8658454406628129
The variables are: Z_v-K_s
n=10000. Worst quantile of [(3, 0)] at -1.8948356176617809
The variables are: Z_m-Q
n=10000. Worst quantile of [(3, 1)] at -1.8897021464935606
The variables are: Z_m-K_s
n=10000. Worst quantile of [(3, 2)] at -1.8746482861346154
The variables are: Z_m-Z_v
n=10000. Worst quantile of [(4, 0)] at -1.8692452915370747
The variables are: H_d-Q
n=10000. Worst quantile of [(4, 1)] at -1.84057992389644
The variables are: H_d-K_s
n=10000. Worst quantile of [(4, 2)] at -1.8764223127288653
The variables are: H_d-Z_v
n=10000. Worst quantile of [(4, 3)] at -1.868190935451335
The variables are: H_d-Z_m
n=10000. Worst quantile of [(5, 0)] at -1.8704323637793827
The variables are: C_b-Q
n=10000. Worst quantile of [(5, 1)] at -1.8522220859479948
The variables are: C_b-K_s
n=10000. Worst quantile of [(5, 2)] at -1.84928237797955
The variables are: C_b-Z_v
n=10000. Worst quantile of [(5, 3)] at -1.8890715254725463
The variables are: C_b-Z_m
n=10000. Worst quantile of [(5, 4)] at -1.8637271095359986
The variables are: C_b-H_d
n=10000. Worst quantile of [(6, 0)] at -1.861409348298271
The variables are: L-Q
n=10000. Worst quantile of [(6, 1)] at -1.8604451090990728
The variables are: L-K_s
n=10000. Worst quantile of [(6, 2)] at -1.8503879155918188
The variables are: L-Z_v
n=10000. Worst quantile of [(6, 3)] at -1.8486847236426747
The variables are: L-Z_m
n=10000. Worst quantile of [(6, 4)] at -1.9094855799761237
The variables are: L-H_d
n=10000. Worst quantile of [(6, 5)] at -1.9808537754349351
The variables are: L-C_b
n=10000. Worst quantile of [(7, 0)] at -1.878663628554278
The variables are: B-Q
n=10000. Worst quantile of [(7, 1)] at -1.875981171189049
The variables are: B-K_s
n=10000. Worst quantile of [(7, 2)] at -1.8965733382685335
The variables are: B-Z_v
n=10000. Worst quantile of [(7, 3)] at -1.9068084355700579
The variables are: B-Z_m
n=10000. Worst quantile of [(7, 4)] at -1.9703615137391826
The variables are: B-H_d
n=10000. Worst quantile of [(7, 5)] at -2.13048224348371
The variables are: B-C_b
n=10000. Worst quantile of [(7, 6)] at -2.4138564474178485
The variables are: B-L

Iteration 1: selected pair: (7, 6) (B-L)
Total number of evaluations = 560000. Minimum quantity at -2.41.

n=10000. Worst quantile of [(7, 6), (7, 5), (2, 1)] at -2.474806169711615
The variables are: B-L B-C_b Z_v-K_s
n=10000. Worst quantile of [(7, 6), (7, 5), (3, 0)] at -2.462795413315042
The variables are: B-L B-C_b Z_m-Q
n=10000. Worst quantile of [(7, 6), (7, 5), (3, 1)] at -2.480081021380042
The variables are: B-L B-C_b Z_m-K_s
n=10000. Worst quantile of [(7, 6), (7, 5), (3, 2)] at -2.477767701020804
The variables are: B-L B-C_b Z_m-Z_v
n=10000. Worst quantile of [(7, 6), (7, 5), (4, 0)] at -2.478156972240032
The variables are: B-L B-C_b H_d-Q
n=10000. Worst quantile of [(7, 6), (7, 5), (4, 2)] at -2.4660472987594337
The variables are: B-L B-C_b H_d-Z_v
n=10000. Worst quantile of [(7, 6), (7, 5), (4, 3)] at -2.5048094803586483
The variables are: B-L B-C_b H_d-Z_m
n=10000. Worst quantile of [(7, 6), (7, 5), (5, 0)] at -2.46712591410516
The variables are: B-L B-C_b C_b-Q
n=10000. Worst quantile of [(7, 6), (7, 5), (5, 3)] at -2.468606060961924
The variables are: B-L B-C_b C_b-Z_m
n=10000. Worst quantile of [(7, 6), (7, 5), (5, 4)] at -2.5104141202520553
The variables are: B-L B-C_b C_b-H_d
n=10000. Worst quantile of [(7, 6), (7, 5), (6, 0)] at -2.4758211615921533
The variables are: B-L B-C_b L-Q
n=10000. Worst quantile of [(7, 6), (7, 5), (6, 1)] at -2.4942856875408634
The variables are: B-L B-C_b L-K_s
n=10000. Worst quantile of [(7, 6), (7, 5), (6, 4)] at -2.6243529404675123
The variables are: B-L B-C_b L-H_d
n=10000. Worst quantile of [(7, 6), (7, 5), (6, 5)] at -2.7807825068452185
The variables are: B-L B-C_b L-C_b
n=10000. Worst quantile of [(7, 6), (7, 5), (7, 0)] at -2.477226480842869
The variables are: B-L B-C_b B-Q
n=10000. Worst quantile of [(7, 6), (7, 5), (7, 1)] at -2.4941181705281967
The variables are: B-L B-C_b B-K_s
n=10000. Worst quantile of [(7, 6), (7, 5), (7, 2)] at -2.452656724485727
The variables are: B-L B-C_b B-Z_v
n=10000. Worst quantile of [(7, 6), (7, 5), (7, 3)] at -2.474326707161017
The variables are: B-L B-C_b B-Z_m
n=10000. Worst quantile of [(7, 6), (7, 5), (7, 4)] at -2.4776504180517076
The variables are: B-L B-C_b B-H_d

Iteration 2: selected pair: (6, 5) (L-C_b)
Total number of evaluations = 4360000. Minimum quantity at -2.78.

Max number of pairs reached

In [17]:
from depimpact.dependence_plot import plot_iterative_results, matrix_plot_input

plot_iterative_results(iterative_results, indep_result=indep_result, q_func=q_func)



In [18]:
n = 10000
K = 300
quant_estimate.vine_structure = None
grid_results = quant_estimate.gridsearch(n_dep_param=K, n_input_sample=n, q_func=q_func, grid_type='vertices')

In [19]:
min_result = grid_results.min_result
print(min_result.quantity)


-3.0746765451805227

In [ ]:
from depimpact.dependence_plot import matrix_plot_input
matrix_plot_input(min_result, margins=margins)

In [46]:
from depimpact.utils import get_pair_id, get_pairs_by_levels, get_possible_structures
pairs_iter = [[1, 0]]
pairs_iter_id = [get_pair_id(dim, pair, with_plus=False) for pair in pairs_iter]
pairs_by_levels = get_pairs_by_levels(dim, pairs_iter_id)
quant_estimate.vine_structure = get_possible_structures(dim, pairs_by_levels)[0]
quant_estimate.vine_structure


Out[46]:
array([[2, 0, 0, 0, 0, 0, 0, 0],
       [8, 1, 0, 0, 0, 0, 0, 0],
       [7, 8, 3, 0, 0, 0, 0, 0],
       [6, 7, 8, 4, 0, 0, 0, 0],
       [5, 6, 7, 8, 5, 0, 0, 0],
       [4, 5, 6, 7, 8, 6, 0, 0],
       [3, 4, 5, 6, 7, 8, 7, 0],
       [1, 3, 4, 5, 6, 7, 8, 8]])

In [47]:
grid_results = quant_estimate.gridsearch(n_dep_param=K, n_input_sample=n, q_func=q_func)

In [48]:
min_result = grid_results.min_result
print(min_result.quantity)


-0.5385555538636204

In [49]:
from depimpact.dependence_plot import matrix_plot_input
matrix_plot_input(min_result, margins=margins)


Out[49]:
<seaborn.axisgrid.PairGrid at 0x7fff521bf4e0>

In [ ]: