notebook.community

Edit and run

VarAnneal Tutorial

Introduction

VarAnneal is a Python package for state and parameter estimation in partially observed dynamical systems and neural networks. It uses variational annealing (VA), a variational data assimilation method. VarAnneal is hosted on GitHub at github.com/paulrozdeba/varanneal.

In variational data assimilation, statistical path estimates conditioned on observational data are calculated using high-dimensional integrals of the form: $$ \mathrm{E}\left[G(X) \lvert Y\right] = \frac{\int dX \: G(X)\: e^{-A(X,Y)}}{\int dX \: e^{-A(X,Y)}} \equiv \frac{1}{\mathcal{Z}(Y)} \int dX \: G(X)\: e^{-A(X,Y)} $$ where $X$ is a vector of model states and parameters, and $Y$ is a vector of observational data. These integrals may be approximated using Laplace's method around the dominant peaks of $P(X\lvert Y)$. VA uses an annealing-like, numerical continuation method for finding these peaks using numerical optimization. This optimization is carried out using one of a variety of methods, such as L-BFGS-B, NCG, IPOPT (future), ... These methods require derivatives of $A$, which are computed with automatic differentiation (AD) using the PYADOLC wrapper around ADOL-C.

In dynamical systems, this amounts to estimating statistics for model parameters, as well as trajectories of model states, like the mode, mean, variance, ... The data consists of time series of partial observations of the model variables.

In neural networks, this is used as a method of training the network weights on labeled data sets.

ODE Example 1: Partially Observed Lorenz 96 with Uknown Parameter





In [1]:



    


# Necessary package imports
import time
import numpy as np
%matplotlib nbagg
import matplotlib.pyplot as plt



    











In [2]:



    


from varanneal import va_ode  # The ODE version of VarAnneal

Data

Time series of $L$-dimensional observations $Y$: $$ Y = \left\{ \mathbf{y}^1, \mathbf{y}^2, \ldots, \mathbf{y}^N \right\} $$ where $\mathbf{y}^n = \mathbf{y}(t = t^n)$, $\mathbf{y}^n \in \mathbb{R}^L$ represents an observation at time $t^n$, and $t^1$ and $t^N$ are the times at the beginning and end of the observation window, respectively. $\Delta t = t^{n} - t^{n-1}$ is the (constant) time interval between observations.

Model

$D$-dimensional Lorenz 96 ($D > L$): $$ \frac{dx_i}{dt} = F_i(\mathbf{x}) \equiv x_{i-1} \left(x_{i+1} - x_{i-2}\right) - x_i + k $$ where $\mathbf{x} \in \mathbb{R}^D$ is the state vector of the model system, and $k$ is a static parameter.

Our assumption is that the $\mathbf{y}^n$ is a noisy observation of $L$ out of $D$ Lorenz 96 system variables. This means the observation function $\mathbf{h}$ has the following form: $$ \mathbf{y}^n = \mathbf{h}(\mathbf{x}^n) = \mathbf{H}\mathbf{x}^n + \mathbf{\xi}^n $$ where $\mathbf{H}: \mathbb{R}^D \to \mathbb{R}^L$ is a projection matrix with unit entries, i.e. $\sum_{j}H_{ij}x_j = x_i$, and $\mathbf{\xi}^n \sim N(0, R_m^{-1})$ is i.i.d. Gaussian noise with zero mean and a variance of $1/R_m$.

Discretization

The model system must be discretized in time: $$ \mathbf{x}^{n_f+1} = \mathbf{f}(\mathbf{x}^{n_f}) = \mathbf{x}^{n_f} + \int\limits_{t^n}^{t^{n+1}} dt\: \mathbf{F}(\mathbf{x}(t)) $$ Define a grid of state variables $X$: $$ X = \left\{ \mathbf{x}^1, \mathbf{x}^2, \ldots, \mathbf{x}^{N_f} \right\} $$ where $\Delta t_f = t^{n_f} - t^{n_f - 1}$ is distinct from the data sampling time interval, and in general $\Delta t_f \leq \Delta t$. However, $\Delta t / \Delta t_f$ should be rational so that there are state variables at all the observation times.

You also need to choose the numerical approximation method for discretization of $\mathbf{f}$. For example, using Simpson-Hermite the model error is: $$ \begin{align} \frac{1}{N} \sum_{n\text{ odd}}^{N-2} \sum_{i=1}^D R_{f,i} &\left\{x^{n+2}_i - x^n_i - \frac{\delta t}{6} \left[F_i(t^n, x^n) + 4 F_i(t^{n+1}, x^{n+1}) + F_i(t^{n+2}, x^{n+2})\right]\right\}^2 \nonumber \\ &+ \left\{ x^{n+1}_i - \frac{1}{2}\left(x^n_i + x^{n+2}_i\right) - \frac{\delta t}{8}\left[F_i(t^n, x^n) - F_i(t^{n+2}, x^{n+2})\right]\right\}^2 \end{align} $$ VarAnneal lets you choose between Simpson-Hermite, RK2, RK4, and the trapezoid rule. These are the built-in methods, but in princple any discretization scheme is definable in the code.

Define the ODE system





In [3]:



    


def l96(t, x, k):
    # Define this as you would any ODE system in Python, when x is a *time series* 
    # of states.
    # Note that using loops will not cause any noticeable slowdown with PYADOLC 
    # (as opposed to vectorized Numpy operations on vectors), and in fact you 
    # should be careful which Numpy vector functions you use (all of the "allowed" 
    # Numpy functions are defined in the PYADOLC source code).
    
    # Define a vector to store the elements of the ODE vector field.
    # Note that using numpy.zeros_like avoids the need to overload this function for 
    # object vs. numerical types (important when function is being taped vs. evaluated).
    dxdt = np.zeros_like(x)  # will have shape (N, D)
    D = x.shape[1]
    for i in range(D):
        dxdt[:, i] = x[:, (i-1)%D] * (x[:, (i+1)%D] - x[:, (i-2)%D]) - x[:, i] + k
    return dxdt
    
    # fancy footwork version below
    #return np.roll(x,1,1) * (np.roll(x,-1,1) - np.roll(x,2,1)) - x + k

Action/annealing (hyper)parameters





In [4]:



    


# Model system dimension
D = 20

# Measured variable indices
Lidx = [0, 2, 4, 6, 8, 10, 14, 16]

# RM, RF0
RM = 1.0 / (0.5**2)
RF0 = 4.0e-6

# alpha, and beta ladder
alpha = 1.5
beta_array = np.linspace(0, 100, 101)

g0 = RF0/RM
gammas_all = g0 * alpha**beta_array

Load observed data





In [5]:



    


# The data file included with this example contains observations of all D=20 
# components.  We need to load it and extract just the measured components.
data = np.load("Lorenz96/data/l96_D20_k8p17_data_dt0p05_N161_sm0p5.npy")

# Extract observation times
times_data = data[:, 0]
dt_data = times_data[1] - times_data[0]
N_data = len(times_data)

# Get out just the data
data = data[:, 1:]
data = data[:, Lidx]

Set $\Delta t_f$ based on $\Delta t$.





In [6]:



    


# model state discretization
freq_mod = 1.0  # how often to put down a state variable
dt_model = dt_data / freq_mod
if freq_mod == 1.0:
    N_model = N_data
else:
    N_model = int(N_data * freq_mod) - 1

Initial path/parameter guesses

Later in the notebook, we'll have the option of setting the initial guesses for the observed variables equal to the observations themselves.





In [7]:



    


# State variables
# This should be an array with N_f elements, where element n_f is a D-dimensional 
# vector. In other words, this is an array of state vectors at all "model times".
X0 = (20.0*np.random.rand(N_model * D) - 10.0).reshape((N_model, D))

# Parameters
Pidx = [0]  # indices of estimated parameters
# Initial guess
P0 = np.array([4.0 * np.random.rand() + 6.0])  # Static parameter
#Pinit = 4.0 * np.random.rand(N_model, 1) + 6.0  # Time-dependent parameter

Use VA to estimate states and parameters

First we need to initialize an Annealer object, which stores information about the model, data, annealing hyperparameters, and the action. It also executes the VA algorithm, then is used to save the state and parameter estimates to file.





In [8]:



    


# Initialize Annealer
anneal_l96 = va_ode.Annealer()

# Set the Lorenz 96 model
anneal_l96.set_model(l96, D)

# Load the data into the Annealer object
anneal_l96.set_data(data, t=times_data)

Run VA





In [ ]:



    


# First set some options for the optimization.
# The full list of options can be found in the scipy.optimization package documentation.
BFGS_options = {'gtol':1.0e-8, 'ftol':1.0e-8, 'maxfun':1000000, 'maxiter':1000000}

tstart = time.time()  # time how long VA takes

# Annealer.anneal() executes VA for all beta values (defined above)
# Note the init_to_data option: this initializes the measured variables to the data.
anneal_l96.anneal(X0, P0, alpha, beta_array, RM, RF0, Lidx, Pidx, dt_model=dt_model,
                  init_to_data=True, disc='SimpsonHermite', method='L-BFGS-B',
                  opt_args=BFGS_options, adolcID=0)

print("\nADOL-C annealing completed in %f s."%(time.time() - tstart))

Save action, constituent errors, and state/parameter estimates to file.





In [11]:



    


anneal_l96.save_paths("Lorenz96/results/paths_D20_L8_N161.npy")
anneal_l96.save_params("Lorenz96/results/params_D20_L8_N161.npy")
anneal_l96.save_action_errors("Lorenz96/results/aerr_D20_L8_N161.npy")

Plot the results

One measured, one unmeasured state variable





In [110]:



    


# Load path & parameter estimates, and action curves
allpaths = np.load("Lorenz96/results/paths_D20_L8_N161.npy")
allparam = np.load("Lorenz96/results/params_D20_L8_N161.npy")
aerr = np.load("Lorenz96/results/aerr_D20_L8_N161.npy")

# Load the true solution
true_soln = np.load("Lorenz96/data/l96_D20_k8p17_truesoln_dt0p05_N161.npy")



    











In [107]:



    


beta_show = 50  # plot estimates at this beta value

plot_idx_meas = 1  # measured component index (time has index 0)
plot_idx_meas_dataidx = 0  # measured component index in data array
plot_idx_unmeas = 2  # unmeasured component index (time has index 0)

# plot all path estimates at this beta simultaneously
fig,ax = plt.subplots(2, 1, figsize=(5, 4.5), sharex=True)
fig.set_tight_layout(True)

tplot = allpaths[beta_show, :, 0]

# Observed variable
# plot the data
ax[0].plot(tplot, data[:, plot_idx_meas_dataidx], color="dodgerblue", label="Data", marker=".", ms=3, lw=0)#, alpha=0.7)#, lw=1.5)
# plot the estimate
ax[0].plot(tplot, allpaths[beta_show, :, plot_idx_meas], color="C1", label="Estimate")#, alpha=1.0, lw=2.0)
# plot the true solution
ax[0].plot(tplot, true_soln[:, plot_idx_meas], color="black", label="True", ls="--")#, lw=1.5, alpha=0.7)
ax[0].set_xlim(tplot[0], tplot[-1])
ax[0].set_ylabel(r"$x_%d$ (observed)"%(plot_idx_meas))
ax[0].set_title(r"L96 state estimates ($L=8$, $R_f/R_m = %.1E$)"%(RF0/RM * alpha**beta_show))

h,l = ax[0].get_legend_handles_labels()
ax[0].legend(h,l)

# Unobserved variable
# plot the estimate
ax[1].plot(tplot, allpaths[beta_show, :, plot_idx_unmeas], color="C1")#, alpha=1.0, lw=2.0)
# plot the true solution
ax[1].plot(tplot, true_soln[:, plot_idx_unmeas], color="black", ls="--")#, alpha=0.7)#, lw=1.5)
ax[1].set_xlim(tplot[0], tplot[-1])
ax[1].set_ylabel(r"$x_%d$ (unobserved)"%(plot_idx_unmeas))
ax[1].set_xlabel("Time")

plt.show()
#fig.savefig("/home/prozdeba/CloudStation/write-ups/dynamical_reg_L96_20170123/figs/L96_estimates_L7.pdf")

Plot parameter estimate over course of annealing





In [123]:



    


fig,ax = plt.subplots(1, 1, figsize=(3,3))
fig.set_tight_layout(True)

ax.plot(gammas_all, allparam, color="C1", label="Estimate")#, lw=0, marker='.', ms=3)
ax.axhline(y=8.17, color="black", label="True", ls="--")
ax.set_xscale('log')
ax.set_xlim(gammas_all[0], gammas_all[-1])
ax.set_xlabel(r"$R_f/R_m$")
ax.set_ylabel("$k$")
ax.set_title("L96 parameter")

h,l = ax.get_legend_handles_labels()
ax.legend(h,l)

plt.show()

Plot the action





In [109]:



    


fig,ax = plt.subplots(1, 3, figsize=(9, 3), sharey=True)
fig.set_tight_layout(True)

ymin = 1.0e20
ymax = 0.0

plotlw = 1.0
plotalpha = 1
#plotcolors = ["C0", "C0", "C0"]
plotcolors = ["black", "black", "black"]

action_vals = aerr[:, 1]
ax[0].plot(gammas_all[:], action_vals, lw=plotlw, color=plotcolors[0], alpha=plotalpha)
ax[0].set_xlabel(r"$R_f/R_m$")
ax[0].set_ylabel("Action")
ax[0].axhline(y=1, lw=1, ls="--", color="C3")#, alpha=.7)

measerr_vals = aerr[:, 2]
ax[1].plot(gammas_all[:], measerr_vals, lw=plotlw, color=plotcolors[1], alpha=plotalpha)
ax[1].set_xlabel(r"$R_f/R_m$")
ax[1].set_ylabel("Meas. error")

moderr_vals = aerr[:, 3]
ax[2].plot(gammas_all[:], moderr_vals, lw=plotlw, color=plotcolors[2], alpha=plotalpha)
ax[2].set_xlabel(r"$R_f/R_m$")
ax[2].set_ylabel("Model error")

fig.suptitle("L96 Action & Errors ($L=8$, single data set)", y=1.0)

for i in range(3):
    ax[i].set_yscale('log')
    ax[i].set_xscale('log')
    ax[i].set_xlim(gammas_all[0], gammas_all[-1])
    #ax[i].set_ylim(.001, 1.0e2)

plt.show()

ODE Example 2: Voltage Observations of a Neuron, Modeled with "Simple" HH





In [1]:



    


# Necessary package imports
import time
import numpy as np
%matplotlib nbagg
import matplotlib.pyplot as plt



    











In [2]:



    


from varanneal import va_ode  # The ODE version of VarAnneal

In this example, observations of the voltage trace of a neuron stimulated with a known input current (known as a current clamp protocol) are simulated in a twin experiment. A "simple" Hodgkin-Huxley model (called NaKL) is integrated forward in time with a known time-dependent stimulus, and the solution is sampled at regular intervals with some random noise to simulate noise in a real electrode.

The NaKL model is: $$ \begin{align} \frac{dV}{dt} &= g_{Na} m^3 h \left(E_{Na} - V\right) + g_K n^4 \left(E_K - V\right) + g_L \left(E_L - V\right) + I(t) \\ \frac{dz}{dt} &= \frac{z_\infty(V) - z}{\tau_z(V)} \quad \left(z = m, h, n\right) \\ z_\infty(V) &= \frac{1}{2} \left[1 + \tanh\left(\frac{V - V_z}{\sigma_z}\right) \right], \quad \tau_z(V) = \tau_{1,z} + \tau_{2,z} \left[1 - \tanh\left(\frac{V - V_z}{\sigma_z}\right) \right] \end{align} $$ The various model parameters are assumed to be unknown, or only known to within some finite range. Additionally, only the voltage is actually measured. We will thus use VarAnneal to estimate the gating variable states, and all of the model parameters, from voltage data and the known stimulus current $I(t)$.

New considerations (compared to Lorenz 96)

  • Now there is an input current. We need to make sure we feed VarAnneal this input current correctly, in addition to defining the ODE system properly to take in multiple parameters and the input current (see function definition below).
  • Voltage has units, while gating variables do not; and voltage operates on a different "dynamical scale". That is, over similar amounts of time, voltage will vary between (roughly) -100 and 100 mV, while the gating variables are bound to the range (0, 1). This is problematic if $R_f$ is a scalar, because the voltage model error terms will be weighted more strongly than the gating variables for nothing more than dimensional reasons. Two possible solutions are (1) set a different $R_f$ for each model component, or (2) rescale the voltage data so that it also lies within the interval (0, 1). Here we're going to choose option (1), setting $R_f$ for voltage to be $10^4$ times smaller than for the gating variables.

Define the ODE system





In [3]:



    


# rate functions
def x_inf(V, Vt, Vs):
    return 0.5*(1.0 + np.tanh((V - Vt)/Vs))

def x_tau(V, Vt, Vs, t1, t2):
    return t1 + t2*(1.0 - np.tanh((V - Vt)/Vs)**2)

# ODE system
def nakl(t, x, pstim):
    # Define this as you would any ODE system in Python, when x is a *time series* 
    # of states.
    
    # Now define a vector to store the elements of the ODE vector field.
    # Note that using numpy.zeros_like avoids the need to overload this function for 
    # object vs. numerical types (important when function is being taped vs. evaluated).
    dxdt = np.zeros_like(x)  # will have shape (N, D)
    # If there is a stimulus, pstim (third argument) is assumed to be a tuple like
    #    (parameters, stimlulus time series)
    p,Iext = pstim
    
    # name the states and parameters
    V, m, h, n = (x[:, 0], x[:, 1], x[:, 2], x[:, 3])
    gNa, gK, gL = (p[0], p[1], p[2])
    ENa, EK, EL = (p[3], p[4], p[5])
    Vtm, Vsm, t1m, t2m = (p[6], p[7], p[8], p[9])
    Vth, Vsh, t1h, t2h = (p[10], p[11], p[12], p[13])
    Vtn, Vsn, t1n, t2n = (p[14], p[15], p[16], p[17])

    dxdt[:, 0] = gNa * m**3 * h * (ENa - V) + gK * n**4 * (EK - V) + gL * (EL - V) + Iext
    dxdt[:, 1] = (x_inf(V, Vtm, Vsm) - m) / x_tau(V, Vtm, Vsm, t1m, t2m)
    dxdt[:, 2] = (x_inf(V, Vth, Vsh) - h) / x_tau(V, Vth, Vsh, t1h, t2h)
    dxdt[:, 3] = (x_inf(V, Vtn, Vsn) - n) / x_tau(V, Vtn, Vsn, t1n, t2n)
    
    return dxdt

Action/annealing (hyper)parameters





In [4]:



    


# Model system dimension
D = 4

# Measured variable indices
Lidx = [0]

# RM and RF0
RM = 1.0
# Note that we're considering a *different* RF for each model component.
# Definining it this way treats RF like a diagonal matrix.
RF0 = [1.0E-8, 1.0E-4, 1.0E-4, 1.0E-4]

# alpha, and beta ladder
alpha = 1.1
beta_all = np.linspace(0, 400, 401)

gamma0 = RF0[0]/RM
gamma_all = gamma0 * alpha**beta_all

Load observation & stimulus data

Note that VarAnneal also contains convenience functions for loading in data from files without needing to manually manipulate arrays like this. I'll show an example a few cells down, after the Annealer object is initialized.





In [5]:



    


# The data file included with this example contains observations of all D=20 
# components.  We need to load it and extract just the measured components.
data = np.load("NaKL/data/NaKL_Vdata_dt0p02_N6001_sm1p0.npy")

# Extract observation times
times_data = data[:, 0]
dt_data = times_data[1] - times_data[0]
N_data = len(times_data)

# Get out just the data
data = data[:, 1:]
data = data[:, Lidx]

###################################
# Stimulus
stim = np.load("NaKL/data/NaKL_stim_dt0p02_N6001.npy")
stim = stim[:, 1]

Set $\Delta t_f$ based on $\Delta t$.





In [6]:



    


# model state discretization
freq_mod = 1.0  # how often to put down a state variable
dt_model = dt_data / freq_mod
if freq_mod == 1.0:
    N_model = N_data
else:
    N_model = int(N_data * freq_mod) - 1

Initial path/parameter guesses





In [7]:



    


# State variables
# This should be an array with N_f elements, where element n_f is a D-dimensional 
# vector. In other words, this is an array of state vectors at all "model times".
Xinit = 0.2*np.random.rand(N_model, D) + 0.4
# Note we didn't rescale V init, to reflect its wider dynamical range of -100 to +100.
# Later we're going to set it equal to the data anyway!

##############################
# Parameters
Pidx = range(18)  # indices of estimated parameters
#Pidx = []  # leave empty to estimate no parameters

# Initial guess
# Start by defining bounds for initial guesses (later used for optimization bounds)
Pb = []
Pb.append([60.0, 180.0])  # gNa_b
Pb.append([10.0, 30.0])  # gK_b
Pb.append([0.15, 0.45])  # gL_b
Pb.append([47.5, 52.5])  # ENa_b
Pb.append([-80.85, -73.15])  # EK_b
Pb.append([-56.7, -51.3])  # EL_b
Pb.append([-42.0, -38.0])  # Vtm_b
Pb.append([14.25, 15.75])  # Vsm_b
Pb.append([0.095, 0.105])  # t1m_b
Pb.append([0.38, 0.42])  # t2m_b
Pb.append([-63.0, -57.0])  # Vth_b
Pb.append([-15.75, -14.25])  # Vsh_b
Pb.append([0.95, 1.05])  # t1h_b
Pb.append([6.65, 7.35])  # t2h_b
Pb.append([-57.75, -52.25])  # Vtn_b
Pb.append([28.5, 31.5])  # Vsn_b
Pb.append([0.95, 1.05])  # t1n_b
Pb.append([4.75, 5.25])  # t2n_b

# Draw initial guesses from these intervals
Pinit = np.zeros(len(Pidx))
for i,b in enumerate(Pb):
    r = b[1] - b[0]
    Pinit[i]= r*np.random.rand() + b[0]
Pinit = np.array(Pinit)
#Pinit = np.load("NaKL/NaKL_trueparam_dt0p02_N6001.npy")  # use this to load in true parameters

Initialize VarAnneal object to estimate states and parameters





In [8]:



    


# Initialize Annealer
anneal_nakl = va_ode.Annealer()

# Set the NaKL model
anneal_nakl.set_model(nakl, D)

# Load the data and stimulus into the Annealer object
anneal_nakl.set_data(data, stim=stim, t=times_data)

Run VA

Note that the tolerance on f and g for termination of BFGS has been turned up here, compared to the Lorenz 96 example. This example still runs fine with 10^-8 but will take much longer, and using 10^-4 was found to produce good accuracy anyway.





In [9]:



    


# First set some options for the optimization and annealing loop
# Bounds
bounds = [[-100.0, 100.0], [0.0, 1.0], [0.0, 1.0], [0.0, 1.0]]
for i in xrange(len(Pidx)):
    bounds.append(Pb[i])
    #bounds.append([None, None])

# The full list of BFGS options can be found in the scipy.optimization package documentation.
BFGS_options = {'gtol':1.0e-8, 'ftol':1.0e-8, 'maxfun':1000000, 'maxiter':1000000}

# Set up dictionaries with necessary info for tracking progress
# Don't pass these as arguments to anneal() if you don't want to track progress
track_paths = {'filename':"NaKL/results/inprog_paths.npy"}
track_params = {'filename':"NaKL/results/inprog_params.npy"}
track_action_errors = {'filename':"NaKL/results/inprog_aerr.npy"}



    











In [ ]:



    


tstart = time.time()  # time how long VA takes

# Annealer.anneal() executes VA for all beta values (defined above)
# Note the init_to_data option: this initializes the measured variables to the data.
anneal_nakl.anneal(Xinit, Pinit, alpha, beta_all, RM, RF0, Lidx, Pidx, dt_model=dt_model,
                   init_to_data=True, disc='SimpsonHermite', method='L-BFGS-B',
                   opt_args=BFGS_options, adolcID=0, bounds=bounds, track_paths=track_paths, 
                   track_params=track_params, track_action_errors=track_action_errors)

print("\nADOL-C annealing completed in %f s."%(time.time() - tstart))

Save action, constituent errors, and state/parameter estimates to file.





In [31]:



    


anneal_nakl.save_paths("NaKL/results/paths.npy")
anneal_nakl.save_params("NaKL/results/params.npy")
anneal_nakl.save_action_errors("NaKL/results/aerr.npy")

Plot the results





In [10]:



    


# Load path & parameter estimates, and action curves
allpaths = np.load("NaKL/results/paths.npy")
allparam = np.load("NaKL/results/params.npy")
aerr = np.load("NaKL/results/aerr.npy")

# Load the true solution and parameters
true_soln = np.load("NaKL/data/NaKL_truesoln_dt0p02_N6001.npy")
true_soln_allg = np.resize(true_soln, allpaths.shape)
true_param = np.load("NaKL/data/NaKL_trueparam_dt0p02_N6001.npy")

Plot all state variable estimates





In [ ]:



    


beta_show = 180  # plot estimates at this beta value

# plot all path estimates at this beta simultaneously
fig,ax = plt.subplots(4, 1, figsize=(6, 8), sharex=True)
fig.set_tight_layout(True)

tplot = allpaths[beta_show, :, 0]

# Observed variable (voltage)
# plot the data
ax[0].plot(tplot, data[:, 0], color="dodgerblue", label="Data", marker=".", ms=3, lw=0)#, alpha=0.7)#, lw=1.5)
# plot the estimate
ax[0].plot(tplot, allpaths[beta_show, :, 1], color="C1", label="Estimate")#, alpha=1.0, lw=2.0)
# plot the true solution
ax[0].plot(tplot, true_soln[:, 1], color="black", label="True", ls="--")#, lw=1.5, alpha=0.7)
ax[0].set_xlim(tplot[0], tplot[-1])
ax[0].set_ylabel(r"$V$ (observed)")
ax[0].set_title(r"NaKL state estimates ($R_f/R_m = %.1E$)"%(RF0[0]/RM * alpha**beta_show))

h,l = ax[0].get_legend_handles_labels()
ax[0].legend(h,l)

# Unobserved variables (gating variables)
gvnames = ["m", "h", "n"]
for i in [1, 2, 3]:
    # plot the estimate
    ax[i].plot(tplot, allpaths[beta_show, :, i], color="C1")#, alpha=1.0, lw=2.0)
    # plot the true solution
    ax[i].plot(tplot, true_soln[:, i], color="black", ls="--")#, alpha=0.7)#, lw=1.5)
    ax[i].set_xlim(tplot[0], tplot[-1])
    ax[i].set_ylabel(r"$%s$ (unobserved)"%(gvnames[i-1]))
    #ax[1].set_xlabel("Time")

ax[3].set_xlabel("Time")

plt.show()
#fig.savefig("/home/prozdeba/CloudStation/write-ups/dynamical_reg_L96_20170123/figs/L96_estimates_L7.pdf")

Plot the action





In [ ]:



    


fig,ax = plt.subplots(1, 3, figsize=(9, 3), sharey=True)
fig.set_tight_layout(True)

ymin = 1.0e20
ymax = 0.0

plotlw = 1.0
plotalpha = 1
#plotcolors = ["C0", "C0", "C0"]
plotcolors = ["black", "black", "black"]

action_vals = aerr[:, 1]
ax[0].plot(gammas_all[:], action_vals, lw=plotlw, color=plotcolors[0], alpha=plotalpha)
ax[0].set_xlabel(r"$R_f/R_m$")
ax[0].set_ylabel("Action")
ax[0].axhline(y=1, lw=1, ls="--", color="C3")#, alpha=.7)

measerr_vals = aerr[:, 2]
ax[1].plot(gammas_all[:], measerr_vals, lw=plotlw, color=plotcolors[1], alpha=plotalpha)
ax[1].set_xlabel(r"$R_f/R_m$")
ax[1].set_ylabel("Meas. error")

moderr_vals = aerr[:, 3]
ax[2].plot(gammas_all[:], moderr_vals, lw=plotlw, color=plotcolors[2], alpha=plotalpha)
ax[2].set_xlabel(r"$R_f/R_m$")
ax[2].set_ylabel("Model error")

fig.suptitle("NaKL Action & Errors (single data set)", y=1.0)

for i in range(3):
    ax[i].set_yscale('log')
    ax[i].set_xscale('log')
    ax[i].set_xlim(gammas_all[0], gammas_all[-1])
    #ax[i].set_ylim(.001, 1.0e2)

plt.show()

Neural Network Example 1: All-to-All Feed-Forward Network for MNIST Image Recognition





In [ ]:



    


# Necessary package imports
import time
import numpy as np
%matplotlib nbagg
import matplotlib.pyplot as plt



    











In [3]:



    


from varanneal import va_nnet

Data

$M$ pairs of data, each consisting of a handwritten image (the input) and a label for the image, which is which digit a human decides the image is of (the output), forming the data $Y$.

Model

Here we use an all-to-all feed-forward neural network with 3 layers: one input, one output, and one hidden layer. The input and output layer sizes are fixed by the number of pixels in each image and the number of possible labels, respectively.

The state of the network, $X$, is a collection of neuron activations in each layer: $$ X = \left\{ \mathbf{x}^{l_1}, \mathbf{x}^{l_2}, \ldots, \mathbf{x}^{l_F} \right\} $$

The state of the network in layer $l_n$ is transferred to layer $l_{n+1}$ according to the activation function: $$ \mathbf{x}^{n+1} = \sigma(\mathbf{x}^{n}, \mathbf{W}^{n+1}) = \frac{1}{1 + \exp \left[-\mathbf{W}^{n+1} \cdot \mathbf{x}^n \right]} $$ where $\mathbf{W}^n$ is the matrix of "synaptic" weights between layer $n-1$ and $n$.

Define the activation function





In [34]:



    


def sigmoid(x, W, b):
    linpart = np.dot(W, x) + b
    return 1.0 / (1.0 + np.exp(-linpart))

Network structure





In [35]:



    


N = 3  # Total umber of layers
D_in = 784  # Number of neurons in the input layer
D_out = 10  # Number of neurons in the output layer
D_hidden = 30  # Number of neurons in the hidden layers

# Network structure is stored in a list, where element i is the number 
# of neurons in layer i.
structure = np.zeros(N, dtype='int')
structure[0] = D_in  # 3 neurons in the input layer
structure[N-1] = D_out  # 2 neurons in the output layer
for i in range(1, N-1):
    structure[i] = D_hidden  # 5 neurons in the hidden layers

Lidx = [np.linspace(0, D_in-1, D_in, dtype='int'), np.linspace(0, D_out-1, D_out, dtype='int')]

Action/annealing parameters





In [36]:



    


# RM, RF0
RM = 1.0
RF0 = 1.0e-8 * RM * float(np.sum(structure) - structure[0]) / float(structure[0] + structure[-1])
g0 = RF0 / RM

# alpha, and beta ladder
alpha = 1.1
beta_array = np.linspace(0, 435, 436)
gammas_all = g0 * alpha**beta_array

Load MNIST data set (images and labels)





In [10]:



    


M = 2  # number of training examples
data_in = np.load("MNIST/MNIST_data/imtrain_norm.npy")[:M]
data_out = np.load("MNIST/MNIST_data/labtrain.npy")[:M]

Initial neuron activation/weight guesses





In [11]:



    


np.random.seed(27509436)

# Neuron states
Xin = np.random.randn(D_in)
Xin = (Xin - np.average(Xin)) / np.std(Xin)
X0 = np.copy(Xin)

for n in xrange(N-2):
    X0 = np.append(X0, 0.2*np.random.rand(D_hidden) + 0.4)
X0 = np.append(X0, 0.2*np.random.rand(D_out) + 0.4)

for m in xrange(M - 1):
    Xin = np.random.randn(D_in)
    Xin = (Xin - np.average(Xin)) / np.std(Xin)
    X0 = np.append(X0, Xin)
    for n in xrange(N-2):
        X0 = np.append(X0, 0.2*np.random.rand(D_hidden) + 0.4)
    X0 = np.append(X0, 0.2*np.random.rand(D_out) + 0.4)

X0 = np.array(X0).flatten()

# Parameters
NP = np.sum(structure[1:]*structure[:-1] + structure[1:])
P0 = np.array([], dtype=np.float64)

W_i0 = 0
W_if = structure[0]*structure[1]
b_i0 = W_if
b_if = b_i0 + structure[1]

# Set up Pidx so that only weights, not biases, are estimated
for n in xrange(N - 1):
    if n == 0:
        Pidx = np.arange(W_i0, W_if, 1, dtype='int')
    else:
        Pidx = np.append(Pidx, np.arange(W_i0, W_if, 1, dtype='int'))
    if n == 0:
        P0 = np.append(P0, (2.0*np.random.rand(structure[n]*structure[n+1]) - 1.0) / D_in)
    else:
        P0 = np.append(P0, (2.0*np.random.rand(structure[n]*structure[n+1]) - 1.0) / D_hidden)
    P0 = np.append(P0, np.zeros(structure[n+1]))

    if n < N - 2:
        W_i0 = b_if
        W_if = W_i0 + structure[n+1]*structure[n+2]
        b_i0 = W_if
        b_if = b_i0 + structure[n+2]

P0 = np.array(P0).flatten()
Pidx = np.array(Pidx).flatten().tolist()

Use VA to train the network weights





In [13]:



    


# Initialize Annealer
anneal1 = va_nnet.Annealer()
# Set the network structure
anneal1.set_structure(structure)
# Set the activation function
anneal1.set_activation(sigmoid)
# Set the input and output data
anneal1.set_input_data(data_in)
anneal1.set_output_data(data_out)

Run VA using L-BFGS-B





In [15]:



    


BFGS_options = {'gtol':1.0e-12, 'ftol':1.0e-12, 'maxfun':1000000, 'maxiter':1000000}

tstart = time.time()
anneal1.anneal(X0, P0, alpha, beta_array, RM, RF0, Pidx, Lidx=Lidx,
               method='L-BFGS-B', opt_args=BFGS_options, adolcID=0)

print("\nADOL-C annealing completed in %f s."%(time.time() - tstart))



    


















    






------------------------------
Step 1 of 436
beta = 0, RF = 5.03778338e-10

Taping action evaluation...
Done!
Time = 0.207283973694 s

Beginning optimization...
Optimization complete!
Time = 0.0408389568329 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.28725617e-11]

------------------------------
Step 2 of 436
beta = 1, RF = 5.54156171e-10

Taping action evaluation...
Done!
Time = 0.16681098938 s

Beginning optimization...
Optimization complete!
Time = 0.0275549888611 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.61598127e-11]

------------------------------
Step 3 of 436
beta = 2, RF = 6.09571788e-10

Taping action evaluation...
Done!
Time = 0.17361497879 s

Beginning optimization...
Optimization complete!
Time = 0.0329079627991 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.97757882e-11]

------------------------------
Step 4 of 436
beta = 3, RF = 6.70528967e-10

Taping action evaluation...
Done!
Time = 0.176285982132 s

Beginning optimization...
Optimization complete!
Time = 0.0585899353027 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.37533601e-11]

------------------------------
Step 5 of 436
beta = 4, RF = 7.37581864e-10

Taping action evaluation...
Done!
Time = 0.147356033325 s

Beginning optimization...
Optimization complete!
Time = 0.0551490783691 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.81286883e-11]

------------------------------
Step 6 of 436
beta = 5, RF = 8.11340050e-10

Taping action evaluation...
Done!
Time = 0.174240112305 s

Beginning optimization...
Optimization complete!
Time = 0.0308618545532 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.29415474e-11]

------------------------------
Step 7 of 436
beta = 6, RF = 8.92474055e-10

Taping action evaluation...
Done!
Time = 0.183092832565 s

Beginning optimization...
Optimization complete!
Time = 0.0310189723969 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.82356912e-11]

------------------------------
Step 8 of 436
beta = 7, RF = 9.81721461e-10

Taping action evaluation...
Done!
Time = 0.174162864685 s

Beginning optimization...
Optimization complete!
Time = 0.0270640850067 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.40592464e-11]

------------------------------
Step 9 of 436
beta = 8, RF = 1.07989361e-09

Taping action evaluation...
Done!
Time = 0.189844846725 s

Beginning optimization...
Optimization complete!
Time = 0.0304028987885 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.04651553e-11]

------------------------------
Step 10 of 436
beta = 9, RF = 1.18788297e-09

Taping action evaluation...
Done!
Time = 0.20986700058 s

Beginning optimization...
Optimization complete!
Time = 0.0499210357666 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.75116504e-11]

------------------------------
Step 11 of 436
beta = 10, RF = 1.30667126e-09

Taping action evaluation...
Done!
Time = 0.199637889862 s

Beginning optimization...
Optimization complete!
Time = 0.0538098812103 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.52627921e-11]

------------------------------
Step 12 of 436
beta = 11, RF = 1.43733839e-09

Taping action evaluation...
Done!
Time = 0.19420003891 s

Beginning optimization...
Optimization complete!
Time = 0.0732550621033 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.37890404e-11]

------------------------------
Step 13 of 436
beta = 12, RF = 1.58107223e-09

Taping action evaluation...
Done!
Time = 0.151098966599 s

Beginning optimization...
Optimization complete!
Time = 0.055046081543 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.03167909e-10]

------------------------------
Step 14 of 436
beta = 13, RF = 1.73917945e-09

Taping action evaluation...
Done!
Time = 0.143642902374 s

Beginning optimization...
Optimization complete!
Time = 0.051393032074 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.13484652e-10]

------------------------------
Step 15 of 436
beta = 14, RF = 1.91309740e-09

Taping action evaluation...
Done!
Time = 0.175591945648 s

Beginning optimization...
Optimization complete!
Time = 0.0306680202484 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.24833062e-10]

------------------------------
Step 16 of 436
beta = 15, RF = 2.10440714e-09

Taping action evaluation...
Done!
Time = 0.178088903427 s

Beginning optimization...
Optimization complete!
Time = 0.0303981304169 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.37316290e-10]

------------------------------
Step 17 of 436
beta = 16, RF = 2.31484785e-09

Taping action evaluation...
Done!
Time = 0.169936180115 s

Beginning optimization...
Optimization complete!
Time = 0.028862953186 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.51047830e-10]

------------------------------
Step 18 of 436
beta = 17, RF = 2.54633264e-09

Taping action evaluation...
Done!
Time = 0.182311058044 s

Beginning optimization...
Optimization complete!
Time = 0.0307950973511 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.66152483e-10]

------------------------------
Step 19 of 436
beta = 18, RF = 2.80096590e-09

Taping action evaluation...
Done!
Time = 0.169265985489 s

Beginning optimization...
Optimization complete!
Time = 0.0274381637573 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.82767584e-10]

------------------------------
Step 20 of 436
beta = 19, RF = 3.08106249e-09

Taping action evaluation...
Done!
Time = 0.174711942673 s

Beginning optimization...
Optimization complete!
Time = 0.0321829319 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.01044116e-10]

------------------------------
Step 21 of 436
beta = 20, RF = 3.38916874e-09

Taping action evaluation...
Done!
Time = 0.224292993546 s

Beginning optimization...
Optimization complete!
Time = 0.0557539463043 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.21148273e-10]

------------------------------
Step 22 of 436
beta = 21, RF = 3.72808561e-09

Taping action evaluation...
Done!
Time = 0.149304866791 s

Beginning optimization...
Optimization complete!
Time = 0.0549790859222 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.43262689e-10]

------------------------------
Step 23 of 436
beta = 22, RF = 4.10089418e-09

Taping action evaluation...
Done!
Time = 0.197201013565 s

Beginning optimization...
Optimization complete!
Time = 0.0299220085144 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.67588496e-10]

------------------------------
Step 24 of 436
beta = 23, RF = 4.51098359e-09

Taping action evaluation...
Done!
Time = 0.179483175278 s

Beginning optimization...
Optimization complete!
Time = 0.0298089981079 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.94346555e-10]

------------------------------
Step 25 of 436
beta = 24, RF = 4.96208195e-09

Taping action evaluation...
Done!
Time = 0.175837039948 s

Beginning optimization...
Optimization complete!
Time = 0.0337221622467 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.23780327e-10]

------------------------------
Step 26 of 436
beta = 25, RF = 5.45829015e-09

Taping action evaluation...
Done!
Time = 0.176738977432 s

Beginning optimization...
Optimization complete!
Time = 0.044093132019 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.56156721e-10]

------------------------------
Step 27 of 436
beta = 26, RF = 6.00411916e-09

Taping action evaluation...
Done!
Time = 0.170716047287 s

Beginning optimization...
Optimization complete!
Time = 0.0319008827209 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.91770570e-10]

------------------------------
Step 28 of 436
beta = 27, RF = 6.60453108e-09

Taping action evaluation...
Done!
Time = 0.17210316658 s

Beginning optimization...
Optimization complete!
Time = 0.0299038887024 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.30943857e-10]

------------------------------
Step 29 of 436
beta = 28, RF = 7.26498419e-09

Taping action evaluation...
Done!
Time = 0.171993017197 s

Beginning optimization...
Optimization complete!
Time = 0.0567140579224 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.74034066e-10]

------------------------------
Step 30 of 436
beta = 29, RF = 7.99148261e-09

Taping action evaluation...
Done!
Time = 0.144304990768 s

Beginning optimization...
Optimization complete!
Time = 0.0535340309143 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.21427240e-10]

------------------------------
Step 31 of 436
beta = 30, RF = 8.79063087e-09

Taping action evaluation...
Done!
Time = 0.147794008255 s

Beginning optimization...
Optimization complete!
Time = 0.0591058731079 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.73558665e-10]

------------------------------
Step 32 of 436
beta = 31, RF = 9.66969395e-09

Taping action evaluation...
Done!
Time = 0.176513910294 s

Beginning optimization...
Optimization complete!
Time = 0.0294950008392 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.30876411e-10]

------------------------------
Step 33 of 436
beta = 32, RF = 1.06366633e-08

Taping action evaluation...
Done!
Time = 0.176320075989 s

Beginning optimization...
Optimization complete!
Time = 0.0309131145477 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.93922068e-10]

------------------------------
Step 34 of 436
beta = 33, RF = 1.17003297e-08

Taping action evaluation...
Done!
Time = 0.1739320755 s

Beginning optimization...
Optimization complete!
Time = 0.0297150611877 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.62955485e-10]

------------------------------
Step 35 of 436
beta = 34, RF = 1.28703627e-08

Taping action evaluation...
Done!
Time = 0.174423933029 s

Beginning optimization...
Optimization complete!
Time = 0.0340719223022 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.38857232e-10]

------------------------------
Step 36 of 436
beta = 35, RF = 1.41573989e-08

Taping action evaluation...
Done!
Time = 0.186172962189 s

Beginning optimization...
Optimization complete!
Time = 0.0501270294189 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  8.67584006e-10]

------------------------------
Step 37 of 436
beta = 36, RF = 1.55731388e-08

Taping action evaluation...
Done!
Time = 0.17027592659 s

Beginning optimization...
Optimization complete!
Time = 0.0284368991852 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.54334932e-10]

------------------------------
Step 38 of 436
beta = 37, RF = 1.71304527e-08

Taping action evaluation...
Done!
Time = 0.179630994797 s

Beginning optimization...
Optimization complete!
Time = 0.0631890296936 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.04976009e-09]

------------------------------
Step 39 of 436
beta = 38, RF = 1.88434980e-08

Taping action evaluation...
Done!
Time = 0.144722938538 s

Beginning optimization...
Optimization complete!
Time = 0.0568909645081 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.15472735e-09]

------------------------------
Step 40 of 436
beta = 39, RF = 2.07278478e-08

Taping action evaluation...
Done!
Time = 0.144567966461 s

Beginning optimization...
Optimization complete!
Time = 0.0567059516907 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.27019029e-09]

------------------------------
Step 41 of 436
beta = 40, RF = 2.28006325e-08

Taping action evaluation...
Done!
Time = 0.17391204834 s

Beginning optimization...
Optimization complete!
Time = 0.0309729576111 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.39719942e-09]

------------------------------
Step 42 of 436
beta = 41, RF = 2.50806958e-08

Taping action evaluation...
Done!
Time = 0.176017045975 s

Beginning optimization...
Optimization complete!
Time = 0.0310459136963 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.53690820e-09]

------------------------------
Step 43 of 436
beta = 42, RF = 2.75887654e-08

Taping action evaluation...
Done!
Time = 0.176738023758 s

Beginning optimization...
Optimization complete!
Time = 0.0310208797455 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.69058770e-09]

------------------------------
Step 44 of 436
beta = 43, RF = 3.03476419e-08

Taping action evaluation...
Done!
Time = 0.18831205368 s

Beginning optimization...
Optimization complete!
Time = 0.0327250957489 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.85963358e-09]

------------------------------
Step 45 of 436
beta = 44, RF = 3.33824061e-08

Taping action evaluation...
Done!
Time = 0.188362836838 s

Beginning optimization...
Optimization complete!
Time = 0.0332279205322 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.04558358e-09]

------------------------------
Step 46 of 436
beta = 45, RF = 3.67206467e-08

Taping action evaluation...
Done!
Time = 0.180087089539 s

Beginning optimization...
Optimization complete!
Time = 0.0317940711975 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.25012653e-09]

------------------------------
Step 47 of 436
beta = 46, RF = 4.03927114e-08

Taping action evaluation...
Done!
Time = 0.172219991684 s

Beginning optimization...
Optimization complete!
Time = 0.0578858852386 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.47512284e-09]

------------------------------
Step 48 of 436
beta = 47, RF = 4.44319825e-08

Taping action evaluation...
Done!
Time = 0.146614074707 s

Beginning optimization...
Optimization complete!
Time = 0.058580160141 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.72261594e-09]

------------------------------
Step 49 of 436
beta = 48, RF = 4.88751807e-08

Taping action evaluation...
Done!
Time = 0.174515008926 s

Beginning optimization...
Optimization complete!
Time = 0.0325379371643 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.99485675e-09]

------------------------------
Step 50 of 436
beta = 49, RF = 5.37626988e-08

Taping action evaluation...
Done!
Time = 0.180155992508 s

Beginning optimization...
Optimization complete!
Time = 0.0347480773926 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.29431753e-09]

------------------------------
Step 51 of 436
beta = 50, RF = 5.91389687e-08

Taping action evaluation...
Done!
Time = 0.172741889954 s

Beginning optimization...
Optimization complete!
Time = 0.0318231582642 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.62372183e-09]

------------------------------
Step 52 of 436
beta = 51, RF = 6.50528656e-08

Taping action evaluation...
Done!
Time = 0.171976089478 s

Beginning optimization...
Optimization complete!
Time = 0.0317220687866 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.98606035e-09]

------------------------------
Step 53 of 436
beta = 52, RF = 7.15581521e-08

Taping action evaluation...
Done!
Time = 0.173966169357 s

Beginning optimization...
Optimization complete!
Time = 0.0294270515442 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.38462877e-09]

------------------------------
Step 54 of 436
beta = 53, RF = 7.87139673e-08

Taping action evaluation...
Done!
Time = 0.18477678299 s

Beginning optimization...
Optimization complete!
Time = 0.0344438552856 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.82304423e-09]

------------------------------
Step 55 of 436
beta = 54, RF = 8.65853641e-08

Taping action evaluation...
Done!
Time = 0.171561002731 s

Beginning optimization...
Optimization complete!
Time = 0.0548498630524 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.30529519e-09]

------------------------------
Step 56 of 436
beta = 55, RF = 9.52439005e-08

Taping action evaluation...
Done!
Time = 0.145802974701 s

Beginning optimization...
Optimization complete!
Time = 0.0612740516663 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.83575516e-09]

------------------------------
Step 57 of 436
beta = 56, RF = 1.04768291e-07

Taping action evaluation...
Done!
Time = 0.146765947342 s

Beginning optimization...
Optimization complete!
Time = 0.0557060241699 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.41925186e-09]

------------------------------
Step 58 of 436
beta = 57, RF = 1.15245120e-07

Taping action evaluation...
Done!
Time = 0.176957130432 s

Beginning optimization...
Optimization complete!
Time = 0.0287551879883 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.06107078e-09]

------------------------------
Step 59 of 436
beta = 58, RF = 1.26769632e-07

Taping action evaluation...
Done!
Time = 0.176004171371 s

Beginning optimization...
Optimization complete!
Time = 0.0305318832397 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.76705721e-09]

------------------------------
Step 60 of 436
beta = 59, RF = 1.39446595e-07

Taping action evaluation...
Done!
Time = 0.174901008606 s

Beginning optimization...
Optimization complete!
Time = 0.0302040576935 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.54359366e-09]

------------------------------
Step 61 of 436
beta = 60, RF = 1.53391254e-07

Taping action evaluation...
Done!
Time = 0.176490068436 s

Beginning optimization...
Optimization complete!
Time = 0.0303168296814 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.39776099e-09]

------------------------------
Step 62 of 436
beta = 61, RF = 1.68730380e-07

Taping action evaluation...
Done!
Time = 0.173987150192 s

Beginning optimization...
Optimization complete!
Time = 0.0314931869507 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.03372557e-08]

------------------------------
Step 63 of 436
beta = 62, RF = 1.85603418e-07

Taping action evaluation...
Done!
Time = 0.190356969833 s

Beginning optimization...
Optimization complete!
Time = 0.0339679718018 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.13706629e-08]

------------------------------
Step 64 of 436
beta = 63, RF = 2.04163759e-07

Taping action evaluation...
Done!
Time = 0.172940015793 s

Beginning optimization...
Optimization complete!
Time = 0.0549101829529 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.25072410e-08]

------------------------------
Step 65 of 436
beta = 64, RF = 2.24580135e-07

Taping action evaluation...
Done!
Time = 0.146435022354 s

Beginning optimization...
Optimization complete!
Time = 0.0528130531311 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.37574146e-08]

------------------------------
Step 66 of 436
beta = 65, RF = 2.47038149e-07

Taping action evaluation...
Done!
Time = 0.171190023422 s

Beginning optimization...
Optimization complete!
Time = 0.0291421413422 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.51322725e-08]

------------------------------
Step 67 of 436
beta = 66, RF = 2.71741964e-07

Taping action evaluation...
Done!
Time = 0.176737070084 s

Beginning optimization...
Optimization complete!
Time = 0.0300529003143 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.66445082e-08]

------------------------------
Step 68 of 436
beta = 67, RF = 2.98916160e-07

Taping action evaluation...
Done!
Time = 0.175187110901 s

Beginning optimization...
Optimization complete!
Time = 0.293487071991 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 46
Obj. function value = [  4.17033365e-09]

------------------------------
Step 69 of 436
beta = 68, RF = 3.28807776e-07

Taping action evaluation...
Done!
Time = 0.167511940002 s

Beginning optimization...
Optimization complete!
Time = 0.0774178504944 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 8
Obj. function value = [  4.13780432e-09]

------------------------------
Step 70 of 436
beta = 69, RF = 3.61688554e-07

Taping action evaluation...
Done!
Time = 0.166878938675 s

Beginning optimization...
Optimization complete!
Time = 0.0344769954681 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.55147646e-09]

------------------------------
Step 71 of 436
beta = 70, RF = 3.97857409e-07

Taping action evaluation...
Done!
Time = 0.186941862106 s

Beginning optimization...
Optimization complete!
Time = 0.032457113266 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.00655671e-09]

------------------------------
Step 72 of 436
beta = 71, RF = 4.37643150e-07

Taping action evaluation...
Done!
Time = 0.174706935883 s

Beginning optimization...
Optimization complete!
Time = 0.0304310321808 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.50713456e-09]

------------------------------
Step 73 of 436
beta = 72, RF = 4.81407465e-07

Taping action evaluation...
Done!
Time = 0.173285961151 s

Beginning optimization...
Optimization complete!
Time = 0.0593838691711 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.05778687e-09]

------------------------------
Step 74 of 436
beta = 73, RF = 5.29548211e-07

Taping action evaluation...
Done!
Time = 0.145220041275 s

Beginning optimization...
Optimization complete!
Time = 0.0540409088135 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.66348962e-09]

------------------------------
Step 75 of 436
beta = 74, RF = 5.82503032e-07

Taping action evaluation...
Done!
Time = 0.177155017853 s

Beginning optimization...
Optimization complete!
Time = 0.0355348587036 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.32976665e-09]

------------------------------
Step 76 of 436
beta = 75, RF = 6.40753336e-07

Taping action evaluation...
Done!
Time = 0.174913883209 s

Beginning optimization...
Optimization complete!
Time = 0.0321657657623 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.06264104e-09]

------------------------------
Step 77 of 436
beta = 76, RF = 7.04828669e-07

Taping action evaluation...
Done!
Time = 0.171248912811 s

Beginning optimization...
Optimization complete!
Time = 0.0302188396454 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.86879789e-09]

------------------------------
Step 78 of 436
beta = 77, RF = 7.75311536e-07

Taping action evaluation...
Done!
Time = 0.17277598381 s

Beginning optimization...
Optimization complete!
Time = 0.0312378406525 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.75548535e-09]

------------------------------
Step 79 of 436
beta = 78, RF = 8.52842690e-07

Taping action evaluation...
Done!
Time = 0.175847053528 s

Beginning optimization...
Optimization complete!
Time = 0.0340819358826 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.07308231e-08]

------------------------------
Step 80 of 436
beta = 79, RF = 9.38126959e-07

Taping action evaluation...
Done!
Time = 0.189464092255 s

Beginning optimization...
Optimization complete!
Time = 0.0300681591034 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.18033738e-08]

------------------------------
Step 81 of 436
beta = 80, RF = 1.03193965e-06

Taping action evaluation...
Done!
Time = 0.172563076019 s

Beginning optimization...
Optimization complete!
Time = 0.0557780265808 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.29831220e-08]

------------------------------
Step 82 of 436
beta = 81, RF = 1.13513362e-06

Taping action evaluation...
Done!
Time = 0.143554925919 s

Beginning optimization...
Optimization complete!
Time = 0.100279092789 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 8
Obj. function value = [  1.42169770e-08]

------------------------------
Step 83 of 436
beta = 82, RF = 1.24864698e-06

Taping action evaluation...
Done!
Time = 0.165734052658 s

Beginning optimization...
Optimization complete!
Time = 0.027801990509 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.56386111e-08]

------------------------------
Step 84 of 436
beta = 83, RF = 1.37351168e-06

Taping action evaluation...
Done!
Time = 0.174499988556 s

Beginning optimization...
Optimization complete!
Time = 0.025759935379 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.72023764e-08]

------------------------------
Step 85 of 436
beta = 84, RF = 1.51086285e-06

Taping action evaluation...
Done!
Time = 0.176567077637 s

Beginning optimization...
Optimization complete!
Time = 0.0285789966583 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.89225174e-08]

------------------------------
Step 86 of 436
beta = 85, RF = 1.66194913e-06

Taping action evaluation...
Done!
Time = 0.180439949036 s

Beginning optimization...
Optimization complete!
Time = 0.0327839851379 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.08144938e-08]

------------------------------
Step 87 of 436
beta = 86, RF = 1.82814405e-06

Taping action evaluation...
Done!
Time = 0.173146009445 s

Beginning optimization...
Optimization complete!
Time = 0.032586812973 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.28956328e-08]

------------------------------
Step 88 of 436
beta = 87, RF = 2.01095845e-06

Taping action evaluation...
Done!
Time = 0.173631191254 s

Beginning optimization...
Optimization complete!
Time = 0.0325059890747 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.51850508e-08]

------------------------------
Step 89 of 436
beta = 88, RF = 2.21205430e-06

Taping action evaluation...
Done!
Time = 0.189909934998 s

Beginning optimization...
Optimization complete!
Time = 0.0353031158447 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.77029581e-08]

------------------------------
Step 90 of 436
beta = 89, RF = 2.43325973e-06

Taping action evaluation...
Done!
Time = 0.174236059189 s

Beginning optimization...
Optimization complete!
Time = 0.0555598735809 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.04726023e-08]

------------------------------
Step 91 of 436
beta = 90, RF = 2.67658570e-06

Taping action evaluation...
Done!
Time = 0.14503288269 s

Beginning optimization...
Optimization complete!
Time = 0.0513260364532 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.35195334e-08]

------------------------------
Step 92 of 436
beta = 91, RF = 2.94424427e-06

Taping action evaluation...
Done!
Time = 0.179789066315 s

Beginning optimization...
Optimization complete!
Time = 0.033038854599 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.68710694e-08]

------------------------------
Step 93 of 436
beta = 92, RF = 3.23866870e-06

Taping action evaluation...
Done!
Time = 0.177695035934 s

Beginning optimization...
Optimization complete!
Time = 0.0348567962646 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.05578012e-08]

------------------------------
Step 94 of 436
beta = 93, RF = 3.56253556e-06

Taping action evaluation...
Done!
Time = 0.178938865662 s

Beginning optimization...
Optimization complete!
Time = 0.866024971008 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 152
Obj. function value = [  6.96729532e-10]

------------------------------
Step 95 of 436
beta = 94, RF = 3.91878912e-06

Taping action evaluation...
Done!
Time = 0.165563106537 s

Beginning optimization...
Optimization complete!
Time = 0.0253188610077 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.65955416e-10]

------------------------------
Step 96 of 436
beta = 95, RF = 4.31066803e-06

Taping action evaluation...
Done!
Time = 0.176705121994 s

Beginning optimization...
Optimization complete!
Time = 0.0325989723206 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.42043881e-10]

------------------------------
Step 97 of 436
beta = 96, RF = 4.74173484e-06

Taping action evaluation...
Done!
Time = 0.1763651371 s

Beginning optimization...
Optimization complete!
Time = 0.0301780700684 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.25597810e-10]

------------------------------
Step 98 of 436
beta = 97, RF = 5.21590832e-06

Taping action evaluation...
Done!
Time = 0.175482988358 s

Beginning optimization...
Optimization complete!
Time = 0.055037021637 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.01742117e-09]

------------------------------
Step 99 of 436
beta = 98, RF = 5.73749915e-06

Taping action evaluation...
Done!
Time = 0.144520998001 s

Beginning optimization...
Optimization complete!
Time = 0.0536329746246 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.11823570e-09]

------------------------------
Step 100 of 436
beta = 99, RF = 6.31124907e-06

Taping action evaluation...
Done!
Time = 0.172603845596 s

Beginning optimization...
Optimization complete!
Time = 0.0346069335938 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.22901050e-09]

------------------------------
Step 101 of 436
beta = 100, RF = 6.94237397e-06

Taping action evaluation...
Done!
Time = 0.175919055939 s

Beginning optimization...
Optimization complete!
Time = 0.255574941635 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 41
Obj. function value = [  7.54837234e-10]

------------------------------
Step 102 of 436
beta = 101, RF = 7.63661137e-06

Taping action evaluation...
Done!
Time = 0.169775009155 s

Beginning optimization...
Optimization complete!
Time = 0.0375590324402 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.29548584e-10]

------------------------------
Step 103 of 436
beta = 102, RF = 8.40027251e-06

Taping action evaluation...
Done!
Time = 0.179793119431 s

Beginning optimization...
Optimization complete!
Time = 0.0352649688721 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.11647874e-10]

------------------------------
Step 104 of 436
beta = 103, RF = 9.24029976e-06

Taping action evaluation...
Done!
Time = 0.171972990036 s

Beginning optimization...
Optimization complete!
Time = 0.0323150157928 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.00229999e-09]

------------------------------
Step 105 of 436
beta = 104, RF = 1.01643297e-05

Taping action evaluation...
Done!
Time = 0.172846794128 s

Beginning optimization...
Optimization complete!
Time = 0.030641078949 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.10195998e-09]

------------------------------
Step 106 of 436
beta = 105, RF = 1.11807627e-05

Taping action evaluation...
Done!
Time = 0.174387216568 s

Beginning optimization...
Optimization complete!
Time = 0.0299432277679 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.21176130e-09]

------------------------------
Step 107 of 436
beta = 106, RF = 1.22988390e-05

Taping action evaluation...
Done!
Time = 0.175213813782 s

Beginning optimization...
Optimization complete!
Time = 0.0575737953186 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.33248043e-09]

------------------------------
Step 108 of 436
beta = 107, RF = 1.35287229e-05

Taping action evaluation...
Done!
Time = 0.144882917404 s

Beginning optimization...
Optimization complete!
Time = 0.0546369552612 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.46534100e-09]

------------------------------
Step 109 of 436
beta = 108, RF = 1.48815952e-05

Taping action evaluation...
Done!
Time = 0.183110952377 s

Beginning optimization...
Optimization complete!
Time = 0.0333490371704 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.61138496e-09]

------------------------------
Step 110 of 436
beta = 109, RF = 1.63697547e-05

Taping action evaluation...
Done!
Time = 0.175638914108 s

Beginning optimization...
Optimization complete!
Time = 0.0315380096436 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.77204967e-09]

------------------------------
Step 111 of 436
beta = 110, RF = 1.80067302e-05

Taping action evaluation...
Done!
Time = 0.174273014069 s

Beginning optimization...
Optimization complete!
Time = 0.0304758548737 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.94854495e-09]

------------------------------
Step 112 of 436
beta = 111, RF = 1.98074032e-05

Taping action evaluation...
Done!
Time = 0.189856052399 s

Beginning optimization...
Optimization complete!
Time = 0.0316429138184 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.14266631e-09]

------------------------------
Step 113 of 436
beta = 112, RF = 2.17881435e-05

Taping action evaluation...
Done!
Time = 0.173564910889 s

Beginning optimization...
Optimization complete!
Time = 0.062087059021 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 7
Obj. function value = [  2.32656562e-09]

------------------------------
Step 114 of 436
beta = 113, RF = 2.39669578e-05

Taping action evaluation...
Done!
Time = 0.166154146194 s

Beginning optimization...
Optimization complete!
Time = 0.0572419166565 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  2.54224347e-09]

------------------------------
Step 115 of 436
beta = 114, RF = 2.63636536e-05

Taping action evaluation...
Done!
Time = 0.172266960144 s

Beginning optimization...
Optimization complete!
Time = 0.0765972137451 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  2.78047557e-09]

------------------------------
Step 116 of 436
beta = 115, RF = 2.90000190e-05

Taping action evaluation...
Done!
Time = 0.144587039948 s

Beginning optimization...
Optimization complete!
Time = 0.0543458461761 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.05802634e-09]

------------------------------
Step 117 of 436
beta = 116, RF = 3.19000209e-05

Taping action evaluation...
Done!
Time = 0.171579122543 s

Beginning optimization...
Optimization complete!
Time = 0.0303659439087 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.36311539e-09]

------------------------------
Step 118 of 436
beta = 117, RF = 3.50900230e-05

Taping action evaluation...
Done!
Time = 0.174654960632 s

Beginning optimization...
Optimization complete!
Time = 0.0312898159027 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.69861904e-09]

------------------------------
Step 119 of 436
beta = 118, RF = 3.85990253e-05

Taping action evaluation...
Done!
Time = 0.175940036774 s

Beginning optimization...
Optimization complete!
Time = 0.0318419933319 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.06772855e-09]

------------------------------
Step 120 of 436
beta = 119, RF = 4.24589278e-05

Taping action evaluation...
Done!
Time = 0.177922964096 s

Beginning optimization...
Optimization complete!
Time = 0.0371978282928 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.47354953e-09]

------------------------------
Step 121 of 436
beta = 120, RF = 4.67048206e-05

Taping action evaluation...
Done!
Time = 0.187395095825 s

Beginning optimization...
Optimization complete!
Time = 0.036700963974 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.91996137e-09]

------------------------------
Step 122 of 436
beta = 121, RF = 5.13753026e-05

Taping action evaluation...
Done!
Time = 0.17603802681 s

Beginning optimization...
Optimization complete!
Time = 0.0311279296875 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.41042656e-09]

------------------------------
Step 123 of 436
beta = 122, RF = 5.65128329e-05

Taping action evaluation...
Done!
Time = 0.172566890717 s

Beginning optimization...
Optimization complete!
Time = 1.20393109322 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 219
Obj. function value = [  7.24672670e-10]

------------------------------
Step 124 of 436
beta = 123, RF = 6.21641162e-05

Taping action evaluation...
Done!
Time = 0.178992986679 s

Beginning optimization...
Optimization complete!
Time = 0.0764448642731 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  7.68490036e-10]

------------------------------
Step 125 of 436
beta = 124, RF = 6.83805278e-05

Taping action evaluation...
Done!
Time = 0.143676042557 s

Beginning optimization...
Optimization complete!
Time = 0.0538640022278 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  8.44702045e-10]

------------------------------
Step 126 of 436
beta = 125, RF = 7.52185806e-05

Taping action evaluation...
Done!
Time = 0.176918029785 s

Beginning optimization...
Optimization complete!
Time = 0.029913187027 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  9.28392044e-10]

------------------------------
Step 127 of 436
beta = 126, RF = 8.27404386e-05

Taping action evaluation...
Done!
Time = 0.177375078201 s

Beginning optimization...
Optimization complete!
Time = 0.0355041027069 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.02040214e-09]

------------------------------
Step 128 of 436
beta = 127, RF = 9.10144825e-05

Taping action evaluation...
Done!
Time = 0.194869041443 s

Beginning optimization...
Optimization complete!
Time = 0.0337860584259 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.12139437e-09]

------------------------------
Step 129 of 436
beta = 128, RF = 1.00115931e-04

Taping action evaluation...
Done!
Time = 0.172247886658 s

Beginning optimization...
Optimization complete!
Time = 0.0363211631775 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.23238525e-09]

------------------------------
Step 130 of 436
beta = 129, RF = 1.10127524e-04

Taping action evaluation...
Done!
Time = 0.173373937607 s

Beginning optimization...
Optimization complete!
Time = 0.0939199924469 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 11
Obj. function value = [  1.27464517e-09]

------------------------------
Step 131 of 436
beta = 130, RF = 1.21140276e-04

Taping action evaluation...
Done!
Time = 0.164734125137 s

Beginning optimization...
Optimization complete!
Time = 0.0283100605011 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.40164725e-09]

------------------------------
Step 132 of 436
beta = 131, RF = 1.33254304e-04

Taping action evaluation...
Done!
Time = 0.179673910141 s

Beginning optimization...
Optimization complete!
Time = 0.0576469898224 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.54111082e-09]

------------------------------
Step 133 of 436
beta = 132, RF = 1.46579734e-04

Taping action evaluation...
Done!
Time = 0.162176132202 s

Beginning optimization...
Optimization complete!
Time = 0.0620770454407 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.69452659e-09]

------------------------------
Step 134 of 436
beta = 133, RF = 1.61237708e-04

Taping action evaluation...
Done!
Time = 0.172410011292 s

Beginning optimization...
Optimization complete!
Time = 0.0331490039825 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.86324394e-09]

------------------------------
Step 135 of 436
beta = 134, RF = 1.77361478e-04

Taping action evaluation...
Done!
Time = 0.175538063049 s

Beginning optimization...
Optimization complete!
Time = 0.0333549976349 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.04850729e-09]

------------------------------
Step 136 of 436
beta = 135, RF = 1.95097626e-04

Taping action evaluation...
Done!
Time = 0.176646947861 s

Beginning optimization...
Optimization complete!
Time = 0.0299167633057 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.25226206e-09]

------------------------------
Step 137 of 436
beta = 136, RF = 2.14607389e-04

Taping action evaluation...
Done!
Time = 0.174993991852 s

Beginning optimization...
Optimization complete!
Time = 0.0330791473389 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.47631502e-09]

------------------------------
Step 138 of 436
beta = 137, RF = 2.36068128e-04

Taping action evaluation...
Done!
Time = 0.174378871918 s

Beginning optimization...
Optimization complete!
Time = 0.0305149555206 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.72233331e-09]

------------------------------
Step 139 of 436
beta = 138, RF = 2.59674940e-04

Taping action evaluation...
Done!
Time = 0.17355799675 s

Beginning optimization...
Optimization complete!
Time = 0.0341470241547 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.99275913e-09]

------------------------------
Step 140 of 436
beta = 139, RF = 2.85642435e-04

Taping action evaluation...
Done!
Time = 0.174216032028 s

Beginning optimization...
Optimization complete!
Time = 0.0340850353241 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.29006690e-09]

------------------------------
Step 141 of 436
beta = 140, RF = 3.14206678e-04

Taping action evaluation...
Done!
Time = 0.176661014557 s

Beginning optimization...
Optimization complete!
Time = 0.0617120265961 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.61667732e-09]

------------------------------
Step 142 of 436
beta = 141, RF = 3.45627346e-04

Taping action evaluation...
Done!
Time = 0.161000013351 s

Beginning optimization...
Optimization complete!
Time = 0.063845872879 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.97530985e-09]

------------------------------
Step 143 of 436
beta = 142, RF = 3.80190080e-04

Taping action evaluation...
Done!
Time = 0.176434993744 s

Beginning optimization...
Optimization complete!
Time = 0.0333170890808 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.36926213e-09]

------------------------------
Step 144 of 436
beta = 143, RF = 4.18209088e-04

Taping action evaluation...
Done!
Time = 0.175825119019 s

Beginning optimization...
Optimization complete!
Time = 0.0322501659393 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.80210781e-09]

------------------------------
Step 145 of 436
beta = 144, RF = 4.60029997e-04

Taping action evaluation...
Done!
Time = 0.177071094513 s

Beginning optimization...
Optimization complete!
Time = 0.0327060222626 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.27749542e-09]

------------------------------
Step 146 of 436
beta = 145, RF = 5.06032997e-04

Taping action evaluation...
Done!
Time = 0.174792051315 s

Beginning optimization...
Optimization complete!
Time = 0.0324840545654 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  5.79938089e-09]

------------------------------
Step 147 of 436
beta = 146, RF = 5.56636297e-04

Taping action evaluation...
Done!
Time = 0.17297410965 s

Beginning optimization...
Optimization complete!
Time = 0.032210111618 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  6.37219297e-09]

------------------------------
Step 148 of 436
beta = 147, RF = 6.12299926e-04

Taping action evaluation...
Done!
Time = 0.171856880188 s

Beginning optimization...
Optimization complete!
Time = 0.0322160720825 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.00081478e-09]

------------------------------
Step 149 of 436
beta = 148, RF = 6.73529919e-04

Taping action evaluation...
Done!
Time = 0.171741962433 s

Beginning optimization...
Optimization complete!
Time = 0.0589179992676 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  7.69061727e-09]

------------------------------
Step 150 of 436
beta = 149, RF = 7.40882911e-04

Taping action evaluation...
Done!
Time = 0.179018974304 s

Beginning optimization...
Optimization complete!
Time = 0.522661924362 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 73
Obj. function value = [  1.39958754e-09]

------------------------------
Step 151 of 436
beta = 150, RF = 8.14971202e-04

Taping action evaluation...
Done!
Time = 0.179502010345 s

Beginning optimization...
Optimization complete!
Time = 0.0326061248779 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.53699554e-09]

------------------------------
Step 152 of 436
beta = 151, RF = 8.96468322e-04

Taping action evaluation...
Done!
Time = 0.17557311058 s

Beginning optimization...
Optimization complete!
Time = 0.0334689617157 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.68776472e-09]

------------------------------
Step 153 of 436
beta = 152, RF = 9.86115154e-04

Taping action evaluation...
Done!
Time = 0.173578977585 s

Beginning optimization...
Optimization complete!
Time = 0.0325319766998 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  1.85307146e-09]

------------------------------
Step 154 of 436
beta = 153, RF = 1.08472667e-03

Taping action evaluation...
Done!
Time = 0.175890922546 s

Beginning optimization...
Optimization complete!
Time = 0.0285041332245 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.03428378e-09]

------------------------------
Step 155 of 436
beta = 154, RF = 1.19319934e-03

Taping action evaluation...
Done!
Time = 0.173233032227 s

Beginning optimization...
Optimization complete!
Time = 0.0302441120148 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.23283856e-09]

------------------------------
Step 156 of 436
beta = 155, RF = 1.31251927e-03

Taping action evaluation...
Done!
Time = 0.173314094543 s

Beginning optimization...
Optimization complete!
Time = 0.0363409519196 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.45028587e-09]

------------------------------
Step 157 of 436
beta = 156, RF = 1.44377120e-03

Taping action evaluation...
Done!
Time = 0.173489093781 s

Beginning optimization...
Optimization complete!
Time = 0.0284659862518 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.68831016e-09]

------------------------------
Step 158 of 436
beta = 157, RF = 1.58814832e-03

Taping action evaluation...
Done!
Time = 0.218745946884 s

Beginning optimization...
Optimization complete!
Time = 0.057315826416 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  2.94873084e-09]

------------------------------
Step 159 of 436
beta = 158, RF = 1.74696315e-03

Taping action evaluation...
Done!
Time = 0.14350104332 s

Beginning optimization...
Optimization complete!
Time = 0.0569460391998 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.23350549e-09]

------------------------------
Step 160 of 436
beta = 159, RF = 1.92165946e-03

Taping action evaluation...
Done!
Time = 0.1777780056 s

Beginning optimization...
Optimization complete!
Time = 0.0506689548492 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.54016083e-09]

------------------------------
Step 161 of 436
beta = 160, RF = 2.11382541e-03

Taping action evaluation...
Done!
Time = 0.171993970871 s

Beginning optimization...
Optimization complete!
Time = 0.0360250473022 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  3.87720241e-09]

------------------------------
Step 162 of 436
beta = 161, RF = 2.32520795e-03

Taping action evaluation...
Done!
Time = 0.175847053528 s

Beginning optimization...
Optimization complete!
Time = 0.0372080802917 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.24683792e-09]

------------------------------
Step 163 of 436
beta = 162, RF = 2.55772875e-03

Taping action evaluation...
Done!
Time = 0.179730892181 s

Beginning optimization...
Optimization complete!
Time = 0.03706407547 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.65021230e-09]

------------------------------
Step 164 of 436
beta = 163, RF = 2.81350162e-03

Taping action evaluation...
Done!
Time = 0.171042203903 s

Beginning optimization...
Optimization complete!
Time = 0.0374209880829 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  5.08983146e-09]

------------------------------
Step 165 of 436
beta = 164, RF = 3.09485178e-03

Taping action evaluation...
Done!
Time = 0.175188779831 s

Beginning optimization...
Optimization complete!
Time = 0.038990020752 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  5.56874446e-09]

------------------------------
Step 166 of 436
beta = 165, RF = 3.40433696e-03

Taping action evaluation...
Done!
Time = 0.173593997955 s

Beginning optimization...
Optimization complete!
Time = 0.0681049823761 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  6.08886960e-09]

------------------------------
Step 167 of 436
beta = 166, RF = 3.74477066e-03

Taping action evaluation...
Done!
Time = 0.165513038635 s

Beginning optimization...
Optimization complete!
Time = 0.06804895401 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  6.65498299e-09]

------------------------------
Step 168 of 436
beta = 167, RF = 4.11924772e-03

Taping action evaluation...
Done!
Time = 0.179193019867 s

Beginning optimization...
Optimization complete!
Time = 0.0372080802917 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  7.26943674e-09]

------------------------------
Step 169 of 436
beta = 168, RF = 4.53117250e-03

Taping action evaluation...
Done!
Time = 0.176856994629 s

Beginning optimization...
Optimization complete!
Time = 0.0359320640564 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  7.93558629e-09]

------------------------------
Step 170 of 436
beta = 169, RF = 4.98428975e-03

Taping action evaluation...
Done!
Time = 0.175096988678 s

Beginning optimization...
Optimization complete!
Time = 0.0404059886932 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  8.65680180e-09]

------------------------------
Step 171 of 436
beta = 170, RF = 5.48271872e-03

Taping action evaluation...
Done!
Time = 0.170799970627 s

Beginning optimization...
Optimization complete!
Time = 0.0354180335999 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  9.43653886e-09]

------------------------------
Step 172 of 436
beta = 171, RF = 6.03099059e-03

Taping action evaluation...
Done!
Time = 0.174899101257 s

Beginning optimization...
Optimization complete!
Time = 0.039046049118 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  1.02782003e-08]

------------------------------
Step 173 of 436
beta = 172, RF = 6.63408965e-03

Taping action evaluation...
Done!
Time = 0.170387029648 s

Beginning optimization...
Optimization complete!
Time = 0.0439729690552 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.11843596e-08]

------------------------------
Step 174 of 436
beta = 173, RF = 7.29749862e-03

Taping action evaluation...
Done!
Time = 0.173098087311 s

Beginning optimization...
Optimization complete!
Time = 0.286951065063 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 46
Obj. function value = [  5.38716726e-09]

------------------------------
Step 175 of 436
beta = 174, RF = 8.02724848e-03

Taping action evaluation...
Done!
Time = 0.165517091751 s

Beginning optimization...
Optimization complete!
Time = 0.0710270404816 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  5.85134673e-09]

------------------------------
Step 176 of 436
beta = 175, RF = 8.82997332e-03

Taping action evaluation...
Done!
Time = 0.144489049911 s

Beginning optimization...
Optimization complete!
Time = 0.0566639900208 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  6.34884387e-09]

------------------------------
Step 177 of 436
beta = 176, RF = 9.71297066e-03

Taping action evaluation...
Done!
Time = 0.174770116806 s

Beginning optimization...
Optimization complete!
Time = 0.0400059223175 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  6.88075337e-09]

------------------------------
Step 178 of 436
beta = 177, RF = 1.06842677e-02

Taping action evaluation...
Done!
Time = 0.179423809052 s

Beginning optimization...
Optimization complete!
Time = 0.0467910766602 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  7.44794842e-09]

------------------------------
Step 179 of 436
beta = 178, RF = 1.17526945e-02

Taping action evaluation...
Done!
Time = 0.176573991776 s

Beginning optimization...
Optimization complete!
Time = 0.0393879413605 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  8.05140629e-09]

------------------------------
Step 180 of 436
beta = 179, RF = 1.29279639e-02

Taping action evaluation...
Done!
Time = 0.170871019363 s

Beginning optimization...
Optimization complete!
Time = 0.0423400402069 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  8.69158034e-09]

------------------------------
Step 181 of 436
beta = 180, RF = 1.42207603e-02

Taping action evaluation...
Done!
Time = 0.173382997513 s

Beginning optimization...
Optimization complete!
Time = 0.0404779911041 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  9.36889272e-09]

------------------------------
Step 182 of 436
beta = 181, RF = 1.56428364e-02

Taping action evaluation...
Done!
Time = 0.179038047791 s

Beginning optimization...
Optimization complete!
Time = 0.0468220710754 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.00831837e-08]

------------------------------
Step 183 of 436
beta = 182, RF = 1.72071200e-02

Taping action evaluation...
Done!
Time = 0.188009023666 s

Beginning optimization...
Optimization complete!
Time = 0.0720891952515 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  1.08341847e-08]

------------------------------
Step 184 of 436
beta = 183, RF = 1.89278320e-02

Taping action evaluation...
Done!
Time = 0.146280050278 s

Beginning optimization...
Optimization complete!
Time = 0.0700891017914 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.16210094e-08]

------------------------------
Step 185 of 436
beta = 184, RF = 2.08206152e-02

Taping action evaluation...
Done!
Time = 0.169781923294 s

Beginning optimization...
Optimization complete!
Time = 0.0401940345764 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.24424285e-08]

------------------------------
Step 186 of 436
beta = 185, RF = 2.29026767e-02

Taping action evaluation...
Done!
Time = 0.176550149918 s

Beginning optimization...
Optimization complete!
Time = 0.0416979789734 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.32970878e-08]

------------------------------
Step 187 of 436
beta = 186, RF = 2.51929444e-02

Taping action evaluation...
Done!
Time = 0.174535036087 s

Beginning optimization...
Optimization complete!
Time = 0.0512809753418 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  1.41820474e-08]

------------------------------
Step 188 of 436
beta = 187, RF = 2.77122388e-02

Taping action evaluation...
Done!
Time = 0.177479982376 s

Beginning optimization...
Optimization complete!
Time = 0.0458521842957 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.50959787e-08]

------------------------------
Step 189 of 436
beta = 188, RF = 3.04834627e-02

Taping action evaluation...
Done!
Time = 0.172448158264 s

Beginning optimization...
Optimization complete!
Time = 0.0429530143738 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.60358623e-08]

------------------------------
Step 190 of 436
beta = 189, RF = 3.35318090e-02

Taping action evaluation...
Done!
Time = 0.171169996262 s

Beginning optimization...
Optimization complete!
Time = 0.0510470867157 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  1.69963355e-08]

------------------------------
Step 191 of 436
beta = 190, RF = 3.68849899e-02

Taping action evaluation...
Done!
Time = 0.170799016953 s

Beginning optimization...
Optimization complete!
Time = 0.0412731170654 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.79761337e-08]

------------------------------
Step 192 of 436
beta = 191, RF = 4.05734889e-02

Taping action evaluation...
Done!
Time = 0.187837123871 s

Beginning optimization...
Optimization complete!
Time = 0.0688149929047 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  1.89707408e-08]

------------------------------
Step 193 of 436
beta = 192, RF = 4.46308378e-02

Taping action evaluation...
Done!
Time = 0.148323059082 s

Beginning optimization...
Optimization complete!
Time = 0.0797710418701 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  1.99743428e-08]

------------------------------
Step 194 of 436
beta = 193, RF = 4.90939216e-02

Taping action evaluation...
Done!
Time = 0.1755027771 s

Beginning optimization...
Optimization complete!
Time = 0.0452358722687 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  2.09840666e-08]

------------------------------
Step 195 of 436
beta = 194, RF = 5.40033137e-02

Taping action evaluation...
Done!
Time = 0.173802137375 s

Beginning optimization...
Optimization complete!
Time = 0.0465199947357 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  2.19949996e-08]

------------------------------
Step 196 of 436
beta = 195, RF = 5.94036451e-02

Taping action evaluation...
Done!
Time = 0.176327943802 s

Beginning optimization...
Optimization complete!
Time = 0.0491180419922 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  2.30022587e-08]

------------------------------
Step 197 of 436
beta = 196, RF = 6.53440096e-02

Taping action evaluation...
Done!
Time = 0.173969984055 s

Beginning optimization...
Optimization complete!
Time = 0.0421710014343 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  2.40019964e-08]

------------------------------
Step 198 of 436
beta = 197, RF = 7.18784106e-02

Taping action evaluation...
Done!
Time = 0.168869018555 s

Beginning optimization...
Optimization complete!
Time = 0.0477359294891 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  2.49887181e-08]

------------------------------
Step 199 of 436
beta = 198, RF = 7.90662516e-02

Taping action evaluation...
Done!
Time = 0.172358036041 s

Beginning optimization...
Optimization complete!
Time = 0.0544860363007 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  2.59590154e-08]

------------------------------
Step 200 of 436
beta = 199, RF = 8.69728768e-02

Taping action evaluation...
Done!
Time = 0.169250011444 s

Beginning optimization...
Optimization complete!
Time = 0.0914480686188 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  2.69091708e-08]

------------------------------
Step 201 of 436
beta = 200, RF = 9.56701645e-02

Taping action evaluation...
Done!
Time = 0.149297952652 s

Beginning optimization...
Optimization complete!
Time = 0.0876622200012 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  2.78341063e-08]

------------------------------
Step 202 of 436
beta = 201, RF = 1.05237181e-01

Taping action evaluation...
Done!
Time = 0.165827989578 s

Beginning optimization...
Optimization complete!
Time = 0.0374867916107 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  2.87337084e-08]

------------------------------
Step 203 of 436
beta = 202, RF = 1.15760899e-01

Taping action evaluation...
Done!
Time = 0.180302143097 s

Beginning optimization...
Optimization complete!
Time = 0.0543510913849 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  2.96021610e-08]

------------------------------
Step 204 of 436
beta = 203, RF = 1.27336989e-01

Taping action evaluation...
Done!
Time = 0.175276994705 s

Beginning optimization...
Optimization complete!
Time = 0.0484418869019 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.04393346e-08]

------------------------------
Step 205 of 436
beta = 204, RF = 1.40070688e-01

Taping action evaluation...
Done!
Time = 0.173779964447 s

Beginning optimization...
Optimization complete!
Time = 0.053239107132 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.12423038e-08]

------------------------------
Step 206 of 436
beta = 205, RF = 1.54077757e-01

Taping action evaluation...
Done!
Time = 0.169746160507 s

Beginning optimization...
Optimization complete!
Time = 0.0398030281067 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.20103530e-08]

------------------------------
Step 207 of 436
beta = 206, RF = 1.69485532e-01

Taping action evaluation...
Done!
Time = 0.175266981125 s

Beginning optimization...
Optimization complete!
Time = 0.0694699287415 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  3.27411728e-08]

------------------------------
Step 208 of 436
beta = 207, RF = 1.86434085e-01

Taping action evaluation...
Done!
Time = 0.165234088898 s

Beginning optimization...
Optimization complete!
Time = 0.0367569923401 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  3.34359821e-08]

------------------------------
Step 209 of 436
beta = 208, RF = 2.05077494e-01

Taping action evaluation...
Done!
Time = 0.191057920456 s

Beginning optimization...
Optimization complete!
Time = 0.07204413414 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.40935163e-08]

------------------------------
Step 210 of 436
beta = 209, RF = 2.25585243e-01

Taping action evaluation...
Done!
Time = 0.144638061523 s

Beginning optimization...
Optimization complete!
Time = 0.0683479309082 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  3.47144566e-08]

------------------------------
Step 211 of 436
beta = 210, RF = 2.48143768e-01

Taping action evaluation...
Done!
Time = 0.175721168518 s

Beginning optimization...
Optimization complete!
Time = 0.0516738891602 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.52985685e-08]

------------------------------
Step 212 of 436
beta = 211, RF = 2.72958145e-01

Taping action evaluation...
Done!
Time = 0.174973011017 s

Beginning optimization...
Optimization complete!
Time = 0.0609600543976 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  3.58468510e-08]

------------------------------
Step 213 of 436
beta = 212, RF = 3.00253959e-01

Taping action evaluation...
Done!
Time = 0.171010017395 s

Beginning optimization...
Optimization complete!
Time = 0.0474989414215 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.63609427e-08]

------------------------------
Step 214 of 436
beta = 213, RF = 3.30279355e-01

Taping action evaluation...
Done!
Time = 0.173342943192 s

Beginning optimization...
Optimization complete!
Time = 0.055370092392 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.68400258e-08]

------------------------------
Step 215 of 436
beta = 214, RF = 3.63307290e-01

Taping action evaluation...
Done!
Time = 0.170783042908 s

Beginning optimization...
Optimization complete!
Time = 0.0388491153717 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  3.72875004e-08]

------------------------------
Step 216 of 436
beta = 215, RF = 3.99638019e-01

Taping action evaluation...
Done!
Time = 0.176036119461 s

Beginning optimization...
Optimization complete!
Time = 0.0489630699158 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.77035052e-08]

------------------------------
Step 217 of 436
beta = 216, RF = 4.39601821e-01

Taping action evaluation...
Done!
Time = 0.171523809433 s

Beginning optimization...
Optimization complete!
Time = 0.0744409561157 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  3.80901262e-08]

------------------------------
Step 218 of 436
beta = 217, RF = 4.83562003e-01

Taping action evaluation...
Done!
Time = 0.150501012802 s

Beginning optimization...
Optimization complete!
Time = 0.0778939723969 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.84486080e-08]

------------------------------
Step 219 of 436
beta = 218, RF = 5.31918204e-01

Taping action evaluation...
Done!
Time = 0.173154115677 s

Beginning optimization...
Optimization complete!
Time = 0.0279560089111 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  3.88084276e-08]

------------------------------
Step 220 of 436
beta = 219, RF = 5.85110024e-01

Taping action evaluation...
Done!
Time = 0.17839884758 s

Beginning optimization...
Optimization complete!
Time = 0.0683381557465 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  3.90871386e-08]

------------------------------
Step 221 of 436
beta = 220, RF = 6.43621027e-01

Taping action evaluation...
Done!
Time = 0.169132947922 s

Beginning optimization...
Optimization complete!
Time = 0.0459659099579 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.93699754e-08]

------------------------------
Step 222 of 436
beta = 221, RF = 7.07983129e-01

Taping action evaluation...
Done!
Time = 0.176940917969 s

Beginning optimization...
Optimization complete!
Time = 0.0440208911896 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  3.96309288e-08]

------------------------------
Step 223 of 436
beta = 222, RF = 7.78781442e-01

Taping action evaluation...
Done!
Time = 0.170920848846 s

Beginning optimization...
Optimization complete!
Time = 0.0506529808044 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  3.98711452e-08]

------------------------------
Step 224 of 436
beta = 223, RF = 8.56659586e-01

Taping action evaluation...
Done!
Time = 0.171061038971 s

Beginning optimization...
Optimization complete!
Time = 0.0559401512146 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.00919917e-08]

------------------------------
Step 225 of 436
beta = 224, RF = 9.42325545e-01

Taping action evaluation...
Done!
Time = 0.16903591156 s

Beginning optimization...
Optimization complete!
Time = 0.0591311454773 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.02949253e-08]

------------------------------
Step 226 of 436
beta = 225, RF = 1.03655810e+00

Taping action evaluation...
Done!
Time = 0.189294099808 s

Beginning optimization...
Optimization complete!
Time = 0.0866138935089 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.04815426e-08]

------------------------------
Step 227 of 436
beta = 226, RF = 1.14021391e+00

Taping action evaluation...
Done!
Time = 0.140871047974 s

Beginning optimization...
Optimization complete!
Time = 0.0861580371857 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.06520108e-08]

------------------------------
Step 228 of 436
beta = 227, RF = 1.25423530e+00

Taping action evaluation...
Done!
Time = 0.179170846939 s

Beginning optimization...
Optimization complete!
Time = 0.0675251483917 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 7
Obj. function value = [  4.08080088e-08]

------------------------------
Step 229 of 436
beta = 228, RF = 1.37965883e+00

Taping action evaluation...
Done!
Time = 0.170092821121 s

Beginning optimization...
Optimization complete!
Time = 0.0532109737396 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.09512158e-08]

------------------------------
Step 230 of 436
beta = 229, RF = 1.51762471e+00

Taping action evaluation...
Done!
Time = 0.175487041473 s

Beginning optimization...
Optimization complete!
Time = 0.0530071258545 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.10827039e-08]

------------------------------
Step 231 of 436
beta = 230, RF = 1.66938718e+00

Taping action evaluation...
Done!
Time = 0.174986124039 s

Beginning optimization...
Optimization complete!
Time = 0.0525062084198 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.12023462e-08]

------------------------------
Step 232 of 436
beta = 231, RF = 1.83632590e+00

Taping action evaluation...
Done!
Time = 0.169903039932 s

Beginning optimization...
Optimization complete!
Time = 0.0321431159973 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.13207264e-08]

------------------------------
Step 233 of 436
beta = 232, RF = 2.01995849e+00

Taping action evaluation...
Done!
Time = 0.179491996765 s

Beginning optimization...
Optimization complete!
Time = 0.0595769882202 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.14120594e-08]

------------------------------
Step 234 of 436
beta = 233, RF = 2.22195434e+00

Taping action evaluation...
Done!
Time = 0.164927959442 s

Beginning optimization...
Optimization complete!
Time = 0.0821979045868 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  4.15039696e-08]

------------------------------
Step 235 of 436
beta = 234, RF = 2.44414978e+00

Taping action evaluation...
Done!
Time = 0.150131940842 s

Beginning optimization...
Optimization complete!
Time = 0.0789129734039 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.15871599e-08]

------------------------------
Step 236 of 436
beta = 235, RF = 2.68856476e+00

Taping action evaluation...
Done!
Time = 0.171015977859 s

Beginning optimization...
Optimization complete!
Time = 0.0347352027893 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.16699236e-08]

------------------------------
Step 237 of 436
beta = 236, RF = 2.95742123e+00

Taping action evaluation...
Done!
Time = 0.176712036133 s

Beginning optimization...
Optimization complete!
Time = 0.0674750804901 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.17331009e-08]

------------------------------
Step 238 of 436
beta = 237, RF = 3.25316335e+00

Taping action evaluation...
Done!
Time = 0.171748876572 s

Beginning optimization...
Optimization complete!
Time = 0.0389270782471 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  4.17968617e-08]

------------------------------
Step 239 of 436
beta = 238, RF = 3.57847969e+00

Taping action evaluation...
Done!
Time = 0.182496070862 s

Beginning optimization...
Optimization complete!
Time = 0.0370168685913 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.18597707e-08]

------------------------------
Step 240 of 436
beta = 239, RF = 3.93632766e+00

Taping action evaluation...
Done!
Time = 0.180310964584 s

Beginning optimization...
Optimization complete!
Time = 0.0593008995056 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.19073125e-08]

------------------------------
Step 241 of 436
beta = 240, RF = 4.32996042e+00

Taping action evaluation...
Done!
Time = 0.162384986877 s

Beginning optimization...
Optimization complete!
Time = 0.0311579704285 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.19575836e-08]

------------------------------
Step 242 of 436
beta = 241, RF = 4.76295647e+00

Taping action evaluation...
Done!
Time = 0.184473991394 s

Beginning optimization...
Optimization complete!
Time = 0.0398190021515 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.20040659e-08]

------------------------------
Step 243 of 436
beta = 242, RF = 5.23925211e+00

Taping action evaluation...
Done!
Time = 0.186570167542 s

Beginning optimization...
Optimization complete!
Time = 0.0784358978271 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.20397735e-08]

------------------------------
Step 244 of 436
beta = 243, RF = 5.76317732e+00

Taping action evaluation...
Done!
Time = 0.143219947815 s

Beginning optimization...
Optimization complete!
Time = 0.0674941539764 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.20780904e-08]

------------------------------
Step 245 of 436
beta = 244, RF = 6.33949506e+00

Taping action evaluation...
Done!
Time = 0.173034906387 s

Beginning optimization...
Optimization complete!
Time = 0.0361068248749 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.21193184e-08]

------------------------------
Step 246 of 436
beta = 245, RF = 6.97344456e+00

Taping action evaluation...
Done!
Time = 0.173701047897 s

Beginning optimization...
Optimization complete!
Time = 0.0642631053925 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 6
Obj. function value = [  4.21389245e-08]

------------------------------
Step 247 of 436
beta = 246, RF = 7.67078902e+00

Taping action evaluation...
Done!
Time = 0.167927026749 s

Beginning optimization...
Optimization complete!
Time = 0.0288181304932 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.21687536e-08]

------------------------------
Step 248 of 436
beta = 247, RF = 8.43786792e+00

Taping action evaluation...
Done!
Time = 0.176931142807 s

Beginning optimization...
Optimization complete!
Time = 0.0611011981964 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.21912869e-08]

------------------------------
Step 249 of 436
beta = 248, RF = 9.28165471e+00

Taping action evaluation...
Done!
Time = 0.172070026398 s

Beginning optimization...
Optimization complete!
Time = 0.0313980579376 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.22163952e-08]

------------------------------
Step 250 of 436
beta = 249, RF = 1.02098202e+01

Taping action evaluation...
Done!
Time = 0.169981002808 s

Beginning optimization...
Optimization complete!
Time = 0.0271401405334 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.22435486e-08]

------------------------------
Step 251 of 436
beta = 250, RF = 1.12308022e+01

Taping action evaluation...
Done!
Time = 0.178377866745 s

Beginning optimization...
Optimization complete!
Time = 0.0815989971161 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  4.22545536e-08]

------------------------------
Step 252 of 436
beta = 251, RF = 1.23538824e+01

Taping action evaluation...
Done!
Time = 0.154502153397 s

Beginning optimization...
Optimization complete!
Time = 0.0570249557495 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.22749616e-08]

------------------------------
Step 253 of 436
beta = 252, RF = 1.35892707e+01

Taping action evaluation...
Done!
Time = 0.17067694664 s

Beginning optimization...
Optimization complete!
Time = 0.0488719940186 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.22864927e-08]

------------------------------
Step 254 of 436
beta = 253, RF = 1.49481977e+01

Taping action evaluation...
Done!
Time = 0.172820091248 s

Beginning optimization...
Optimization complete!
Time = 0.0324919223785 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23017553e-08]

------------------------------
Step 255 of 436
beta = 254, RF = 1.64430175e+01

Taping action evaluation...
Done!
Time = 0.177076101303 s

Beginning optimization...
Optimization complete!
Time = 0.0425989627838 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 3
Obj. function value = [  4.23133802e-08]

------------------------------
Step 256 of 436
beta = 255, RF = 1.80873193e+01

Taping action evaluation...
Done!
Time = 0.172441959381 s

Beginning optimization...
Optimization complete!
Time = 0.0360088348389 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23262566e-08]

------------------------------
Step 257 of 436
beta = 256, RF = 1.98960512e+01

Taping action evaluation...
Done!
Time = 0.175764799118 s

Beginning optimization...
Optimization complete!
Time = 0.0341370105743 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23401431e-08]

------------------------------
Step 258 of 436
beta = 257, RF = 2.18856563e+01

Taping action evaluation...
Done!
Time = 0.171495914459 s

Beginning optimization...
Optimization complete!
Time = 0.035003900528 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23548309e-08]

------------------------------
Step 259 of 436
beta = 258, RF = 2.40742219e+01

Taping action evaluation...
Done!
Time = 0.172111988068 s

Beginning optimization...
Optimization complete!
Time = 0.0633988380432 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.23542162e-08]

------------------------------
Step 260 of 436
beta = 259, RF = 2.64816441e+01

Taping action evaluation...
Done!
Time = 0.166322946548 s

Beginning optimization...
Optimization complete!
Time = 0.056499004364 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23628122e-08]

------------------------------
Step 261 of 436
beta = 260, RF = 2.91298085e+01

Taping action evaluation...
Done!
Time = 0.164129972458 s

Beginning optimization...
Optimization complete!
Time = 0.0590879917145 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23719904e-08]

------------------------------
Step 262 of 436
beta = 261, RF = 3.20427894e+01

Taping action evaluation...
Done!
Time = 0.177264928818 s

Beginning optimization...
Optimization complete!
Time = 0.0339040756226 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23816859e-08]

------------------------------
Step 263 of 436
beta = 262, RF = 3.52470683e+01

Taping action evaluation...
Done!
Time = 0.174304008484 s

Beginning optimization...
Optimization complete!
Time = 0.0332140922546 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23918851e-08]

------------------------------
Step 264 of 436
beta = 263, RF = 3.87717752e+01

Taping action evaluation...
Done!
Time = 0.175563097 s

Beginning optimization...
Optimization complete!
Time = 0.058305978775 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.23877300e-08]

------------------------------
Step 265 of 436
beta = 264, RF = 4.26489527e+01

Taping action evaluation...
Done!
Time = 0.174031019211 s

Beginning optimization...
Optimization complete!
Time = 0.030170917511 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23931123e-08]

------------------------------
Step 266 of 436
beta = 265, RF = 4.69138479e+01

Taping action evaluation...
Done!
Time = 0.175658941269 s

Beginning optimization...
Optimization complete!
Time = 0.0330579280853 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.23987722e-08]

------------------------------
Step 267 of 436
beta = 266, RF = 5.16052327e+01

Taping action evaluation...
Done!
Time = 0.170253992081 s

Beginning optimization...
Optimization complete!
Time = 0.0288820266724 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24048404e-08]

------------------------------
Step 268 of 436
beta = 267, RF = 5.67657560e+01

Taping action evaluation...
Done!
Time = 0.172384977341 s

Beginning optimization...
Optimization complete!
Time = 0.0674340724945 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.24094016e-08]

------------------------------
Step 269 of 436
beta = 268, RF = 6.24423316e+01

Taping action evaluation...
Done!
Time = 0.146590948105 s

Beginning optimization...
Optimization complete!
Time = 0.0595450401306 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24154346e-08]

------------------------------
Step 270 of 436
beta = 269, RF = 6.86865648e+01

Taping action evaluation...
Done!
Time = 0.191191911697 s

Beginning optimization...
Optimization complete!
Time = 0.0393240451813 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24216533e-08]

------------------------------
Step 271 of 436
beta = 270, RF = 7.55552212e+01

Taping action evaluation...
Done!
Time = 0.173228979111 s

Beginning optimization...
Optimization complete!
Time = 0.0619170665741 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.24145013e-08]

------------------------------
Step 272 of 436
beta = 271, RF = 8.31107434e+01

Taping action evaluation...
Done!
Time = 0.167458057404 s

Beginning optimization...
Optimization complete!
Time = 0.0278341770172 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24173063e-08]

------------------------------
Step 273 of 436
beta = 272, RF = 9.14218177e+01

Taping action evaluation...
Done!
Time = 0.198083162308 s

Beginning optimization...
Optimization complete!
Time = 0.0330390930176 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24202780e-08]

------------------------------
Step 274 of 436
beta = 273, RF = 1.00563999e+02

Taping action evaluation...
Done!
Time = 0.169989824295 s

Beginning optimization...
Optimization complete!
Time = 0.0314130783081 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24234052e-08]

------------------------------
Step 275 of 436
beta = 274, RF = 1.10620399e+02

Taping action evaluation...
Done!
Time = 0.169486045837 s

Beginning optimization...
Optimization complete!
Time = 0.0315709114075 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24264136e-08]

------------------------------
Step 276 of 436
beta = 275, RF = 1.21682439e+02

Taping action evaluation...
Done!
Time = 0.170117139816 s

Beginning optimization...
Optimization complete!
Time = 0.0354981422424 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24296938e-08]

------------------------------
Step 277 of 436
beta = 276, RF = 1.33850683e+02

Taping action evaluation...
Done!
Time = 0.17461681366 s

Beginning optimization...
Optimization complete!
Time = 0.0735769271851 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 2
Obj. function value = [  4.24310631e-08]

------------------------------
Step 278 of 436
beta = 277, RF = 1.47235752e+02

Taping action evaluation...
Done!
Time = 0.146018981934 s

Beginning optimization...
Optimization complete!
Time = 0.0647628307343 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24342293e-08]

------------------------------
Step 279 of 436
beta = 278, RF = 1.61959327e+02

Taping action evaluation...
Done!
Time = 0.182364940643 s

Beginning optimization...
Optimization complete!
Time = 0.0323970317841 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24374907e-08]

------------------------------
Step 280 of 436
beta = 279, RF = 1.78155259e+02

Taping action evaluation...
Done!
Time = 0.171230077744 s

Beginning optimization...
Optimization complete!
Time = 0.0371658802032 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24408507e-08]

------------------------------
Step 281 of 436
beta = 280, RF = 1.95970785e+02

Taping action evaluation...
Done!
Time = 0.173428058624 s

Beginning optimization...
Optimization complete!
Time = 0.0359580516815 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24443053e-08]

------------------------------
Step 282 of 436
beta = 281, RF = 2.15567864e+02

Taping action evaluation...
Done!
Time = 0.176566839218 s

Beginning optimization...
Optimization complete!
Time = 0.0601499080658 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 5
Obj. function value = [  4.24328642e-08]

------------------------------
Step 283 of 436
beta = 282, RF = 2.37124650e+02

Taping action evaluation...
Done!
Time = 0.16856098175 s

Beginning optimization...
Optimization complete!
Time = 0.0378069877625 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24338370e-08]

------------------------------
Step 284 of 436
beta = 283, RF = 2.60837115e+02

Taping action evaluation...
Done!
Time = 0.173930883408 s

Beginning optimization...
Optimization complete!
Time = 0.0305330753326 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24348120e-08]

------------------------------
Step 285 of 436
beta = 284, RF = 2.86920827e+02

Taping action evaluation...
Done!
Time = 0.174545049667 s

Beginning optimization...
Optimization complete!
Time = 0.0615639686584 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24357476e-08]

------------------------------
Step 286 of 436
beta = 285, RF = 3.15612910e+02

Taping action evaluation...
Done!
Time = 0.145885944366 s

Beginning optimization...
Optimization complete!
Time = 0.0583679676056 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24366193e-08]

------------------------------
Step 287 of 436
beta = 286, RF = 3.47174201e+02

Taping action evaluation...
Done!
Time = 0.173892021179 s

Beginning optimization...
Optimization complete!
Time = 0.0390069484711 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24373944e-08]

------------------------------
Step 288 of 436
beta = 287, RF = 3.81891621e+02

Taping action evaluation...
Done!
Time = 0.183422088623 s

Beginning optimization...
Optimization complete!
Time = 0.0325019359589 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24382940e-08]

------------------------------
Step 289 of 436
beta = 288, RF = 4.20080783e+02

Taping action evaluation...
Done!
Time = 0.17658996582 s

Beginning optimization...
Optimization complete!
Time = 0.0343198776245 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24385141e-08]

------------------------------
Step 290 of 436
beta = 289, RF = 4.62088861e+02

Taping action evaluation...
Done!
Time = 0.17583990097 s

Beginning optimization...
Optimization complete!
Time = 0.0304620265961 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24391385e-08]

------------------------------
Step 291 of 436
beta = 290, RF = 5.08297747e+02

Taping action evaluation...
Done!
Time = 0.172023057938 s

Beginning optimization...
Optimization complete!
Time = 0.0362901687622 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24394779e-08]

------------------------------
Step 292 of 436
beta = 291, RF = 5.59127522e+02

Taping action evaluation...
Done!
Time = 0.177403926849 s

Beginning optimization...
Optimization complete!
Time = 0.0381791591644 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24401690e-08]

------------------------------
Step 293 of 436
beta = 292, RF = 6.15040274e+02

Taping action evaluation...
Done!
Time = 0.172032117844 s

Beginning optimization...
Optimization complete!
Time = 0.036055803299 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24408674e-08]

------------------------------
Step 294 of 436
beta = 293, RF = 6.76544301e+02

Taping action evaluation...
Done!
Time = 0.17143201828 s

Beginning optimization...
Optimization complete!
Time = 0.0580439567566 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24416111e-08]

------------------------------
Step 295 of 436
beta = 294, RF = 7.44198731e+02

Taping action evaluation...
Done!
Time = 0.149620056152 s

Beginning optimization...
Optimization complete!
Time = 0.0604729652405 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24423663e-08]

------------------------------
Step 296 of 436
beta = 295, RF = 8.18618605e+02

Taping action evaluation...
Done!
Time = 0.177865982056 s

Beginning optimization...
Optimization complete!
Time = 0.0414071083069 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24431045e-08]

------------------------------
Step 297 of 436
beta = 296, RF = 9.00480465e+02

Taping action evaluation...
Done!
Time = 0.177213907242 s

Beginning optimization...
Optimization complete!
Time = 0.0390820503235 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24438489e-08]

------------------------------
Step 298 of 436
beta = 297, RF = 9.90528512e+02

Taping action evaluation...
Done!
Time = 0.176008939743 s

Beginning optimization...
Optimization complete!
Time = 0.0363578796387 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24445693e-08]

------------------------------
Step 299 of 436
beta = 298, RF = 1.08958136e+03

Taping action evaluation...
Done!
Time = 0.171786785126 s

Beginning optimization...
Optimization complete!
Time = 0.0367438793182 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24452919e-08]

------------------------------
Step 300 of 436
beta = 299, RF = 1.19853950e+03

Taping action evaluation...
Done!
Time = 0.170688867569 s

Beginning optimization...
Optimization complete!
Time = 0.0387558937073 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24459860e-08]

------------------------------
Step 301 of 436
beta = 300, RF = 1.31839345e+03

Taping action evaluation...
Done!
Time = 0.180489063263 s

Beginning optimization...
Optimization complete!
Time = 0.0393521785736 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24466793e-08]

------------------------------
Step 302 of 436
beta = 301, RF = 1.45023279e+03

Taping action evaluation...
Done!
Time = 0.170027017593 s

Beginning optimization...
Optimization complete!
Time = 0.0582718849182 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24473412e-08]

------------------------------
Step 303 of 436
beta = 302, RF = 1.59525607e+03

Taping action evaluation...
Done!
Time = 0.145045042038 s

Beginning optimization...
Optimization complete!
Time = 0.0571839809418 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24480001e-08]

------------------------------
Step 304 of 436
beta = 303, RF = 1.75478168e+03

Taping action evaluation...
Done!
Time = 0.175675153732 s

Beginning optimization...
Optimization complete!
Time = 0.0362629890442 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24486260e-08]

------------------------------
Step 305 of 436
beta = 304, RF = 1.93025985e+03

Taping action evaluation...
Done!
Time = 0.171812057495 s

Beginning optimization...
Optimization complete!
Time = 0.0458240509033 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24492472e-08]

------------------------------
Step 306 of 436
beta = 305, RF = 2.12328583e+03

Taping action evaluation...
Done!
Time = 0.174721956253 s

Beginning optimization...
Optimization complete!
Time = 0.0358769893646 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24498347e-08]

------------------------------
Step 307 of 436
beta = 306, RF = 2.33561442e+03

Taping action evaluation...
Done!
Time = 0.166985034943 s

Beginning optimization...
Optimization complete!
Time = 0.0339031219482 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24504165e-08]

------------------------------
Step 308 of 436
beta = 307, RF = 2.56917586e+03

Taping action evaluation...
Done!
Time = 0.171890974045 s

Beginning optimization...
Optimization complete!
Time = 0.0320551395416 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24509647e-08]

------------------------------
Step 309 of 436
beta = 308, RF = 2.82609344e+03

Taping action evaluation...
Done!
Time = 0.172178030014 s

Beginning optimization...
Optimization complete!
Time = 0.0316159725189 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24515063e-08]

------------------------------
Step 310 of 436
beta = 309, RF = 3.10870279e+03

Taping action evaluation...
Done!
Time = 0.175034999847 s

Beginning optimization...
Optimization complete!
Time = 0.055282831192 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 4
Obj. function value = [  4.24420765e-08]

------------------------------
Step 311 of 436
beta = 310, RF = 3.41957307e+03

Taping action evaluation...
Done!
Time = 0.170355081558 s

Beginning optimization...
Optimization complete!
Time = 0.0634868144989 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24421438e-08]

------------------------------
Step 312 of 436
beta = 311, RF = 3.76153037e+03

Taping action evaluation...
Done!
Time = 0.154449939728 s

Beginning optimization...
Optimization complete!
Time = 0.0650069713593 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24422110e-08]

------------------------------
Step 313 of 436
beta = 312, RF = 4.13768341e+03

Taping action evaluation...
Done!
Time = 0.1801841259 s

Beginning optimization...
Optimization complete!
Time = 0.038519859314 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24422755e-08]

------------------------------
Step 314 of 436
beta = 313, RF = 4.55145175e+03

Taping action evaluation...
Done!
Time = 0.186957836151 s

Beginning optimization...
Optimization complete!
Time = 0.0355319976807 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24423305e-08]

------------------------------
Step 315 of 436
beta = 314, RF = 5.00659693e+03

Taping action evaluation...
Done!
Time = 0.166734933853 s

Beginning optimization...
Optimization complete!
Time = 0.028382062912 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24423779e-08]

------------------------------
Step 316 of 436
beta = 315, RF = 5.50725662e+03

Taping action evaluation...
Done!
Time = 0.175408124924 s

Beginning optimization...
Optimization complete!
Time = 0.036102771759 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24424162e-08]

------------------------------
Step 317 of 436
beta = 316, RF = 6.05798228e+03

Taping action evaluation...
Done!
Time = 0.172491073608 s

Beginning optimization...
Optimization complete!
Time = 0.034353017807 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24424482e-08]

------------------------------
Step 318 of 436
beta = 317, RF = 6.66378051e+03

Taping action evaluation...
Done!
Time = 0.171085119247 s

Beginning optimization...
Optimization complete!
Time = 0.0295159816742 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24424774e-08]

------------------------------
Step 319 of 436
beta = 318, RF = 7.33015856e+03

Taping action evaluation...
Done!
Time = 0.180512905121 s

Beginning optimization...
Optimization complete!
Time = 0.0614850521088 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24425131e-08]

------------------------------
Step 320 of 436
beta = 319, RF = 8.06317442e+03

Taping action evaluation...
Done!
Time = 0.145582199097 s

Beginning optimization...
Optimization complete!
Time = 0.0575180053711 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24425566e-08]

------------------------------
Step 321 of 436
beta = 320, RF = 8.86949186e+03

Taping action evaluation...
Done!
Time = 0.180191040039 s

Beginning optimization...
Optimization complete!
Time = 0.0331881046295 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24426048e-08]

------------------------------
Step 322 of 436
beta = 321, RF = 9.75644105e+03

Taping action evaluation...
Done!
Time = 0.173106908798 s

Beginning optimization...
Optimization complete!
Time = 0.0293741226196 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24426557e-08]

------------------------------
Step 323 of 436
beta = 322, RF = 1.07320852e+04

Taping action evaluation...
Done!
Time = 0.175758123398 s

Beginning optimization...
Optimization complete!
Time = 0.0419571399689 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24427074e-08]

------------------------------
Step 324 of 436
beta = 323, RF = 1.18052937e+04

Taping action evaluation...
Done!
Time = 0.175494909286 s

Beginning optimization...
Optimization complete!
Time = 0.0341060161591 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24427604e-08]

------------------------------
Step 325 of 436
beta = 324, RF = 1.29858230e+04

Taping action evaluation...
Done!
Time = 0.175019979477 s

Beginning optimization...
Optimization complete!
Time = 0.0355100631714 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24428136e-08]

------------------------------
Step 326 of 436
beta = 325, RF = 1.42844053e+04

Taping action evaluation...
Done!
Time = 0.169751882553 s

Beginning optimization...
Optimization complete!
Time = 0.0375621318817 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24428679e-08]

------------------------------
Step 327 of 436
beta = 326, RF = 1.57128459e+04

Taping action evaluation...
Done!
Time = 0.17321395874 s

Beginning optimization...
Optimization complete!
Time = 0.0336120128632 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24429221e-08]

------------------------------
Step 328 of 436
beta = 327, RF = 1.72841305e+04

Taping action evaluation...
Done!
Time = 0.173970937729 s

Beginning optimization...
Optimization complete!
Time = 0.0615880489349 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24429772e-08]

------------------------------
Step 329 of 436
beta = 328, RF = 1.90125435e+04

Taping action evaluation...
Done!
Time = 0.147444009781 s

Beginning optimization...
Optimization complete!
Time = 0.0618450641632 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24430321e-08]

------------------------------
Step 330 of 436
beta = 329, RF = 2.09137979e+04

Taping action evaluation...
Done!
Time = 0.175602912903 s

Beginning optimization...
Optimization complete!
Time = 0.0359399318695 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24430876e-08]

------------------------------
Step 331 of 436
beta = 330, RF = 2.30051776e+04

Taping action evaluation...
Done!
Time = 0.175990104675 s

Beginning optimization...
Optimization complete!
Time = 0.0358481407166 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24431430e-08]

------------------------------
Step 332 of 436
beta = 331, RF = 2.53056954e+04

Taping action evaluation...
Done!
Time = 0.17330121994 s

Beginning optimization...
Optimization complete!
Time = 0.0452327728271 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24431989e-08]

------------------------------
Step 333 of 436
beta = 332, RF = 2.78362649e+04

Taping action evaluation...
Done!
Time = 0.175295114517 s

Beginning optimization...
Optimization complete!
Time = 0.033707857132 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24432546e-08]

------------------------------
Step 334 of 436
beta = 333, RF = 3.06198914e+04

Taping action evaluation...
Done!
Time = 0.173312902451 s

Beginning optimization...
Optimization complete!
Time = 0.0323719978333 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24433108e-08]

------------------------------
Step 335 of 436
beta = 334, RF = 3.36818806e+04

Taping action evaluation...
Done!
Time = 0.177321910858 s

Beginning optimization...
Optimization complete!
Time = 0.0383260250092 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24433668e-08]

------------------------------
Step 336 of 436
beta = 335, RF = 3.70500686e+04

Taping action evaluation...
Done!
Time = 0.175010919571 s

Beginning optimization...
Optimization complete!
Time = 0.0641078948975 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24434234e-08]

------------------------------
Step 337 of 436
beta = 336, RF = 4.07550755e+04

Taping action evaluation...
Done!
Time = 0.159028053284 s

Beginning optimization...
Optimization complete!
Time = 0.0655701160431 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24434797e-08]

------------------------------
Step 338 of 436
beta = 337, RF = 4.48305831e+04

Taping action evaluation...
Done!
Time = 0.175698041916 s

Beginning optimization...
Optimization complete!
Time = 0.0377988815308 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24435365e-08]

------------------------------
Step 339 of 436
beta = 338, RF = 4.93136414e+04

Taping action evaluation...
Done!
Time = 0.183872938156 s

Beginning optimization...
Optimization complete!
Time = 0.040274143219 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24435931e-08]

------------------------------
Step 340 of 436
beta = 339, RF = 5.42450055e+04

Taping action evaluation...
Done!
Time = 0.183704853058 s

Beginning optimization...
Optimization complete!
Time = 0.0386781692505 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24436501e-08]

------------------------------
Step 341 of 436
beta = 340, RF = 5.96695060e+04

Taping action evaluation...
Done!
Time = 0.19336104393 s

Beginning optimization...
Optimization complete!
Time = 0.0363929271698 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24437070e-08]

------------------------------
Step 342 of 436
beta = 341, RF = 6.56364567e+04

Taping action evaluation...
Done!
Time = 0.175806045532 s

Beginning optimization...
Optimization complete!
Time = 0.0337688922882 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24437643e-08]

------------------------------
Step 343 of 436
beta = 342, RF = 7.22001023e+04

Taping action evaluation...
Done!
Time = 0.176795005798 s

Beginning optimization...
Optimization complete!
Time = 0.0365240573883 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24438215e-08]

------------------------------
Step 344 of 436
beta = 343, RF = 7.94201125e+04

Taping action evaluation...
Done!
Time = 0.175088882446 s

Beginning optimization...
Optimization complete!
Time = 0.037024974823 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24438791e-08]

------------------------------
Step 345 of 436
beta = 344, RF = 8.73621238e+04

Taping action evaluation...
Done!
Time = 0.171168088913 s

Beginning optimization...
Optimization complete!
Time = 0.0597240924835 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24439365e-08]

------------------------------
Step 346 of 436
beta = 345, RF = 9.60983362e+04

Taping action evaluation...
Done!
Time = 0.143610954285 s

Beginning optimization...
Optimization complete!
Time = 0.0539870262146 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24439944e-08]

------------------------------
Step 347 of 436
beta = 346, RF = 1.05708170e+05

Taping action evaluation...
Done!
Time = 0.175122976303 s

Beginning optimization...
Optimization complete!
Time = 0.0345649719238 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24440521e-08]

------------------------------
Step 348 of 436
beta = 347, RF = 1.16278987e+05

Taping action evaluation...
Done!
Time = 0.174106121063 s

Beginning optimization...
Optimization complete!
Time = 0.0357990264893 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24441103e-08]

------------------------------
Step 349 of 436
beta = 348, RF = 1.27906885e+05

Taping action evaluation...
Done!
Time = 0.172524929047 s

Beginning optimization...
Optimization complete!
Time = 0.0384089946747 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24441685e-08]

------------------------------
Step 350 of 436
beta = 349, RF = 1.40697574e+05

Taping action evaluation...
Done!
Time = 0.186402082443 s

Beginning optimization...
Optimization complete!
Time = 0.0365250110626 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24442271e-08]

------------------------------
Step 351 of 436
beta = 350, RF = 1.54767331e+05

Taping action evaluation...
Done!
Time = 0.173298835754 s

Beginning optimization...
Optimization complete!
Time = 0.0329599380493 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24442857e-08]

------------------------------
Step 352 of 436
beta = 351, RF = 1.70244065e+05

Taping action evaluation...
Done!
Time = 0.173584938049 s

Beginning optimization...
Optimization complete!
Time = 0.0370750427246 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24443448e-08]

------------------------------
Step 353 of 436
beta = 352, RF = 1.87268471e+05

Taping action evaluation...
Done!
Time = 0.174077987671 s

Beginning optimization...
Optimization complete!
Time = 0.0592670440674 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24444040e-08]

------------------------------
Step 354 of 436
beta = 353, RF = 2.05995318e+05

Taping action evaluation...
Done!
Time = 0.146816968918 s

Beginning optimization...
Optimization complete!
Time = 0.0599730014801 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24444637e-08]

------------------------------
Step 355 of 436
beta = 354, RF = 2.26594850e+05

Taping action evaluation...
Done!
Time = 0.173694133759 s

Beginning optimization...
Optimization complete!
Time = 0.0362961292267 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24445235e-08]

------------------------------
Step 356 of 436
beta = 355, RF = 2.49254335e+05

Taping action evaluation...
Done!
Time = 0.175623893738 s

Beginning optimization...
Optimization complete!
Time = 0.0399870872498 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24445839e-08]

------------------------------
Step 357 of 436
beta = 356, RF = 2.74179768e+05

Taping action evaluation...
Done!
Time = 0.176003932953 s

Beginning optimization...
Optimization complete!
Time = 0.0367109775543 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24446445e-08]

------------------------------
Step 358 of 436
beta = 357, RF = 3.01597745e+05

Taping action evaluation...
Done!
Time = 0.171330213547 s

Beginning optimization...
Optimization complete!
Time = 0.0318069458008 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24447057e-08]

------------------------------
Step 359 of 436
beta = 358, RF = 3.31757520e+05

Taping action evaluation...
Done!
Time = 0.193098068237 s

Beginning optimization...
Optimization complete!
Time = 0.0339381694794 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24447672e-08]

------------------------------
Step 360 of 436
beta = 359, RF = 3.64933272e+05

Taping action evaluation...
Done!
Time = 0.173034906387 s

Beginning optimization...
Optimization complete!
Time = 0.0407311916351 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24448294e-08]

------------------------------
Step 361 of 436
beta = 360, RF = 4.01426599e+05

Taping action evaluation...
Done!
Time = 0.174992084503 s

Beginning optimization...
Optimization complete!
Time = 0.037082195282 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24448918e-08]

------------------------------
Step 362 of 436
beta = 361, RF = 4.41569259e+05

Taping action evaluation...
Done!
Time = 0.176093101501 s

Beginning optimization...
Optimization complete!
Time = 0.0621519088745 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24449551e-08]

------------------------------
Step 363 of 436
beta = 362, RF = 4.85726185e+05

Taping action evaluation...
Done!
Time = 0.144076108932 s

Beginning optimization...
Optimization complete!
Time = 0.058000087738 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24450187e-08]

------------------------------
Step 364 of 436
beta = 363, RF = 5.34298803e+05

Taping action evaluation...
Done!
Time = 0.178688049316 s

Beginning optimization...
Optimization complete!
Time = 0.036808013916 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24450831e-08]

------------------------------
Step 365 of 436
beta = 364, RF = 5.87728683e+05

Taping action evaluation...
Done!
Time = 0.173834085464 s

Beginning optimization...
Optimization complete!
Time = 0.0409550666809 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24451480e-08]

------------------------------
Step 366 of 436
beta = 365, RF = 6.46501552e+05

Taping action evaluation...
Done!
Time = 0.172319173813 s

Beginning optimization...
Optimization complete!
Time = 0.0345520973206 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24452137e-08]

------------------------------
Step 367 of 436
beta = 366, RF = 7.11151707e+05

Taping action evaluation...
Done!
Time = 0.17532992363 s

Beginning optimization...
Optimization complete!
Time = 0.036581993103 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24452800e-08]

------------------------------
Step 368 of 436
beta = 367, RF = 7.82266878e+05

Taping action evaluation...
Done!
Time = 0.187206029892 s

Beginning optimization...
Optimization complete!
Time = 0.0360388755798 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24453472e-08]

------------------------------
Step 369 of 436
beta = 368, RF = 8.60493565e+05

Taping action evaluation...
Done!
Time = 0.166911125183 s

Beginning optimization...
Optimization complete!
Time = 0.0313391685486 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24454150e-08]

------------------------------
Step 370 of 436
beta = 369, RF = 9.46542922e+05

Taping action evaluation...
Done!
Time = 0.171240091324 s

Beginning optimization...
Optimization complete!
Time = 0.0596199035645 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24454838e-08]

------------------------------
Step 371 of 436
beta = 370, RF = 1.04119721e+06

Taping action evaluation...
Done!
Time = 0.144206047058 s

Beginning optimization...
Optimization complete!
Time = 0.0564601421356 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24455533e-08]

------------------------------
Step 372 of 436
beta = 371, RF = 1.14531694e+06

Taping action evaluation...
Done!
Time = 0.175631046295 s

Beginning optimization...
Optimization complete!
Time = 0.0338380336761 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24456239e-08]

------------------------------
Step 373 of 436
beta = 372, RF = 1.25984863e+06

Taping action evaluation...
Done!
Time = 0.178183078766 s

Beginning optimization...
Optimization complete!
Time = 0.0338008403778 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24456952e-08]

------------------------------
Step 374 of 436
beta = 373, RF = 1.38583349e+06

Taping action evaluation...
Done!
Time = 0.174355983734 s

Beginning optimization...
Optimization complete!
Time = 0.0368189811707 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24457676e-08]

------------------------------
Step 375 of 436
beta = 374, RF = 1.52441684e+06

Taping action evaluation...
Done!
Time = 0.168594121933 s

Beginning optimization...
Optimization complete!
Time = 0.0292119979858 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24458409e-08]

------------------------------
Step 376 of 436
beta = 375, RF = 1.67685853e+06

Taping action evaluation...
Done!
Time = 0.175961971283 s

Beginning optimization...
Optimization complete!
Time = 0.0328378677368 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24459153e-08]

------------------------------
Step 377 of 436
beta = 376, RF = 1.84454438e+06

Taping action evaluation...
Done!
Time = 0.189218997955 s

Beginning optimization...
Optimization complete!
Time = 0.0361199378967 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24459907e-08]

------------------------------
Step 378 of 436
beta = 377, RF = 2.02899882e+06

Taping action evaluation...
Done!
Time = 0.174916028976 s

Beginning optimization...
Optimization complete!
Time = 0.0379321575165 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24460673e-08]

------------------------------
Step 379 of 436
beta = 378, RF = 2.23189870e+06

Taping action evaluation...
Done!
Time = 0.17528796196 s

Beginning optimization...
Optimization complete!
Time = 0.0592088699341 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24461449e-08]

------------------------------
Step 380 of 436
beta = 379, RF = 2.45508857e+06

Taping action evaluation...
Done!
Time = 0.147974014282 s

Beginning optimization...
Optimization complete!
Time = 0.0622038841248 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24462238e-08]

------------------------------
Step 381 of 436
beta = 380, RF = 2.70059742e+06

Taping action evaluation...
Done!
Time = 0.179855108261 s

Beginning optimization...
Optimization complete!
Time = 0.0370268821716 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24463038e-08]

------------------------------
Step 382 of 436
beta = 381, RF = 2.97065717e+06

Taping action evaluation...
Done!
Time = 0.179435968399 s

Beginning optimization...
Optimization complete!
Time = 0.0351932048798 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24463852e-08]

------------------------------
Step 383 of 436
beta = 382, RF = 3.26772288e+06

Taping action evaluation...
Done!
Time = 0.176007032394 s

Beginning optimization...
Optimization complete!
Time = 0.0348539352417 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24464677e-08]

------------------------------
Step 384 of 436
beta = 383, RF = 3.59449517e+06

Taping action evaluation...
Done!
Time = 0.178751945496 s

Beginning optimization...
Optimization complete!
Time = 0.0369470119476 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24465517e-08]

------------------------------
Step 385 of 436
beta = 384, RF = 3.95394469e+06

Taping action evaluation...
Done!
Time = 0.173876047134 s

Beginning optimization...
Optimization complete!
Time = 0.0401170253754 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24466370e-08]

------------------------------
Step 386 of 436
beta = 385, RF = 4.34933916e+06

Taping action evaluation...
Done!
Time = 0.253059864044 s

Beginning optimization...
Optimization complete!
Time = 0.0470659732819 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24467237e-08]

------------------------------
Step 387 of 436
beta = 386, RF = 4.78427307e+06

Taping action evaluation...
Done!
Time = 0.17450594902 s

Beginning optimization...
Optimization complete!
Time = 0.0589649677277 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24468118e-08]

------------------------------
Step 388 of 436
beta = 387, RF = 5.26270038e+06

Taping action evaluation...
Done!
Time = 0.1473300457 s

Beginning optimization...
Optimization complete!
Time = 0.0633130073547 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24469015e-08]

------------------------------
Step 389 of 436
beta = 388, RF = 5.78897042e+06

Taping action evaluation...
Done!
Time = 0.178823232651 s

Beginning optimization...
Optimization complete!
Time = 0.0373330116272 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24469927e-08]

------------------------------
Step 390 of 436
beta = 389, RF = 6.36786746e+06

Taping action evaluation...
Done!
Time = 0.177469968796 s

Beginning optimization...
Optimization complete!
Time = 0.0394039154053 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24470855e-08]

------------------------------
Step 391 of 436
beta = 390, RF = 7.00465421e+06

Taping action evaluation...
Done!
Time = 0.175978899002 s

Beginning optimization...
Optimization complete!
Time = 0.0379061698914 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24471798e-08]

------------------------------
Step 392 of 436
beta = 391, RF = 7.70511963e+06

Taping action evaluation...
Done!
Time = 0.174830913544 s

Beginning optimization...
Optimization complete!
Time = 0.0350489616394 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24472758e-08]

------------------------------
Step 393 of 436
beta = 392, RF = 8.47563159e+06

Taping action evaluation...
Done!
Time = 0.173216104507 s

Beginning optimization...
Optimization complete!
Time = 0.0360820293427 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24473735e-08]

------------------------------
Step 394 of 436
beta = 393, RF = 9.32319475e+06

Taping action evaluation...
Done!
Time = 0.17041683197 s

Beginning optimization...
Optimization complete!
Time = 0.0379569530487 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24474730e-08]

------------------------------
Step 395 of 436
beta = 394, RF = 1.02555142e+07

Taping action evaluation...
Done!
Time = 0.192952871323 s

Beginning optimization...
Optimization complete!
Time = 0.0414209365845 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24475742e-08]

------------------------------
Step 396 of 436
beta = 395, RF = 1.12810656e+07

Taping action evaluation...
Done!
Time = 0.177608013153 s

Beginning optimization...
Optimization complete!
Time = 0.0634241104126 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24476773e-08]

------------------------------
Step 397 of 436
beta = 396, RF = 1.24091722e+07

Taping action evaluation...
Done!
Time = 0.150443792343 s

Beginning optimization...
Optimization complete!
Time = 0.0645599365234 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24477822e-08]

------------------------------
Step 398 of 436
beta = 397, RF = 1.36500894e+07

Taping action evaluation...
Done!
Time = 0.174717187881 s

Beginning optimization...
Optimization complete!
Time = 0.0342099666595 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24478891e-08]

------------------------------
Step 399 of 436
beta = 398, RF = 1.50150984e+07

Taping action evaluation...
Done!
Time = 0.17995595932 s

Beginning optimization...
Optimization complete!
Time = 0.033016204834 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24479978e-08]

------------------------------
Step 400 of 436
beta = 399, RF = 1.65166082e+07

Taping action evaluation...
Done!
Time = 0.175852060318 s

Beginning optimization...
Optimization complete!
Time = 0.0392560958862 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24481087e-08]

------------------------------
Step 401 of 436
beta = 400, RF = 1.81682690e+07

Taping action evaluation...
Done!
Time = 0.174076080322 s

Beginning optimization...
Optimization complete!
Time = 0.0403861999512 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24482216e-08]

------------------------------
Step 402 of 436
beta = 401, RF = 1.99850959e+07

Taping action evaluation...
Done!
Time = 0.175353050232 s

Beginning optimization...
Optimization complete!
Time = 0.0340690612793 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24483366e-08]

------------------------------
Step 403 of 436
beta = 402, RF = 2.19836055e+07

Taping action evaluation...
Done!
Time = 0.183475971222 s

Beginning optimization...
Optimization complete!
Time = 0.0429759025574 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24484537e-08]

------------------------------
Step 404 of 436
beta = 403, RF = 2.41819661e+07

Taping action evaluation...
Done!
Time = 0.172477006912 s

Beginning optimization...
Optimization complete!
Time = 0.0581879615784 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24485731e-08]

------------------------------
Step 405 of 436
beta = 404, RF = 2.66001627e+07

Taping action evaluation...
Done!
Time = 0.146662950516 s

Beginning optimization...
Optimization complete!
Time = 0.057226896286 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24486947e-08]

------------------------------
Step 406 of 436
beta = 405, RF = 2.92601790e+07

Taping action evaluation...
Done!
Time = 0.174092054367 s

Beginning optimization...
Optimization complete!
Time = 0.0331990718842 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24488186e-08]

------------------------------
Step 407 of 436
beta = 406, RF = 3.21861969e+07

Taping action evaluation...
Done!
Time = 0.176719903946 s

Beginning optimization...
Optimization complete!
Time = 0.0347900390625 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24489449e-08]

------------------------------
Step 408 of 436
beta = 407, RF = 3.54048165e+07

Taping action evaluation...
Done!
Time = 0.178089857101 s

Beginning optimization...
Optimization complete!
Time = 0.0334270000458 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24490736e-08]

------------------------------
Step 409 of 436
beta = 408, RF = 3.89452982e+07

Taping action evaluation...
Done!
Time = 0.174914836884 s

Beginning optimization...
Optimization complete!
Time = 0.0335710048676 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24492048e-08]

------------------------------
Step 410 of 436
beta = 409, RF = 4.28398280e+07

Taping action evaluation...
Done!
Time = 0.174762964249 s

Beginning optimization...
Optimization complete!
Time = 0.0355038642883 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24493385e-08]

------------------------------
Step 411 of 436
beta = 410, RF = 4.71238108e+07

Taping action evaluation...
Done!
Time = 0.165106058121 s

Beginning optimization...
Optimization complete!
Time = 0.0321469306946 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24494748e-08]

------------------------------
Step 412 of 436
beta = 411, RF = 5.18361919e+07

Taping action evaluation...
Done!
Time = 0.178618907928 s

Beginning optimization...
Optimization complete!
Time = 0.0496799945831 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24496137e-08]

------------------------------
Step 413 of 436
beta = 412, RF = 5.70198111e+07

Taping action evaluation...
Done!
Time = 0.171016931534 s

Beginning optimization...
Optimization complete!
Time = 0.0590319633484 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24497553e-08]

------------------------------
Step 414 of 436
beta = 413, RF = 6.27217922e+07

Taping action evaluation...
Done!
Time = 0.159059047699 s

Beginning optimization...
Optimization complete!
Time = 0.0646500587463 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24498997e-08]

------------------------------
Step 415 of 436
beta = 414, RF = 6.89939714e+07

Taping action evaluation...
Done!
Time = 0.175849914551 s

Beginning optimization...
Optimization complete!
Time = 0.0330951213837 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24500469e-08]

------------------------------
Step 416 of 436
beta = 415, RF = 7.58933686e+07

Taping action evaluation...
Done!
Time = 0.173472166061 s

Beginning optimization...
Optimization complete!
Time = 0.0351800918579 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24501970e-08]

------------------------------
Step 417 of 436
beta = 416, RF = 8.34827054e+07

Taping action evaluation...
Done!
Time = 0.174523115158 s

Beginning optimization...
Optimization complete!
Time = 0.0368680953979 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24503500e-08]

------------------------------
Step 418 of 436
beta = 417, RF = 9.18309759e+07

Taping action evaluation...
Done!
Time = 0.172600030899 s

Beginning optimization...
Optimization complete!
Time = 0.0335140228271 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24505060e-08]

------------------------------
Step 419 of 436
beta = 418, RF = 1.01014074e+08

Taping action evaluation...
Done!
Time = 0.172991037369 s

Beginning optimization...
Optimization complete!
Time = 0.0391998291016 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24506650e-08]

------------------------------
Step 420 of 436
beta = 419, RF = 1.11115481e+08

Taping action evaluation...
Done!
Time = 0.17202091217 s

Beginning optimization...
Optimization complete!
Time = 0.0305199623108 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24508272e-08]

------------------------------
Step 421 of 436
beta = 420, RF = 1.22227029e+08

Taping action evaluation...
Done!
Time = 0.174024820328 s

Beginning optimization...
Optimization complete!
Time = 0.0726239681244 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24509926e-08]

------------------------------
Step 422 of 436
beta = 421, RF = 1.34449732e+08

Taping action evaluation...
Done!
Time = 0.143834114075 s

Beginning optimization...
Optimization complete!
Time = 0.0624070167542 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24511613e-08]

------------------------------
Step 423 of 436
beta = 422, RF = 1.47894705e+08

Taping action evaluation...
Done!
Time = 0.174519062042 s

Beginning optimization...
Optimization complete!
Time = 0.0330889225006 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24513333e-08]

------------------------------
Step 424 of 436
beta = 423, RF = 1.62684176e+08

Taping action evaluation...
Done!
Time = 0.186542987823 s

Beginning optimization...
Optimization complete!
Time = 0.0427429676056 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24515087e-08]

------------------------------
Step 425 of 436
beta = 424, RF = 1.78952593e+08

Taping action evaluation...
Done!
Time = 0.171829938889 s

Beginning optimization...
Optimization complete!
Time = 0.0645499229431 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24516876e-08]

------------------------------
Step 426 of 436
beta = 425, RF = 1.96847852e+08

Taping action evaluation...
Done!
Time = 0.19739484787 s

Beginning optimization...
Optimization complete!
Time = 0.0440468788147 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24518700e-08]

------------------------------
Step 427 of 436
beta = 426, RF = 2.16532638e+08

Taping action evaluation...
Done!
Time = 0.171040058136 s

Beginning optimization...
Optimization complete!
Time = 0.0338311195374 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24520561e-08]

------------------------------
Step 428 of 436
beta = 427, RF = 2.38185901e+08

Taping action evaluation...
Done!
Time = 0.175945997238 s

Beginning optimization...
Optimization complete!
Time = 0.0366868972778 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24522459e-08]

------------------------------
Step 429 of 436
beta = 428, RF = 2.62004492e+08

Taping action evaluation...
Done!
Time = 0.176718950272 s

Beginning optimization...
Optimization complete!
Time = 0.0374138355255 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24524394e-08]

------------------------------
Step 430 of 436
beta = 429, RF = 2.88204941e+08

Taping action evaluation...
Done!
Time = 0.185431003571 s

Beginning optimization...
Optimization complete!
Time = 0.059562921524 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24526369e-08]

------------------------------
Step 431 of 436
beta = 430, RF = 3.17025435e+08

Taping action evaluation...
Done!
Time = 0.150067806244 s

Beginning optimization...
Optimization complete!
Time = 0.0625629425049 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24528382e-08]

------------------------------
Step 432 of 436
beta = 431, RF = 3.48727978e+08

Taping action evaluation...
Done!
Time = 0.179214000702 s

Beginning optimization...
Optimization complete!
Time = 0.0388188362122 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24530436e-08]

------------------------------
Step 433 of 436
beta = 432, RF = 3.83600776e+08

Taping action evaluation...
Done!
Time = 0.177404880524 s

Beginning optimization...
Optimization complete!
Time = 0.0388901233673 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24532531e-08]

------------------------------
Step 434 of 436
beta = 433, RF = 4.21960854e+08

Taping action evaluation...
Done!
Time = 0.179281949997 s

Beginning optimization...
Optimization complete!
Time = 0.0392589569092 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24534667e-08]

------------------------------
Step 435 of 436
beta = 434, RF = 4.64156939e+08

Taping action evaluation...
Done!
Time = 0.172077178955 s

Beginning optimization...
Optimization complete!
Time = 0.0331020355225 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24536847e-08]

------------------------------
Step 436 of 436
beta = 435, RF = 5.10572633e+08

Taping action evaluation...
Done!
Time = 0.174222946167 s

Beginning optimization...
Optimization complete!
Time = 0.0370991230011 s
Exit flag = 0
Exit message: CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Iterations = 1
Obj. function value = [  4.24539070e-08]


ADOL-C annealing completed in 101.516544 s.

Save the output





In [18]:



    


anneal1.save_io("MNIST/results/io.npy")
anneal1.save_Wb("MNIST/results/W.npy", "MNIST/results/b.npy")
anneal1.save_action_errors("MNIST/results/action_errors.npy")

Plot the results





In [37]:



    


# Load action curves
aerr = np.load("MNIST/results/action_errors.npy")



    











In [41]:



    


fig,ax = plt.subplots(1, 3, figsize=(6.75, 2.1), sharey=True)
fig.set_tight_layout(True)

ymin = 1.0e20
ymax = 0.0

plotlw = 1.0
plotalpha = .7
#plotcolors = ["C0", "C0", "C0"]
plotcolors = ["black", "black", "black"]

action_vals = aerr[:, 1]
ax[0].plot(gammas_all[:], action_vals, lw=plotlw, color=plotcolors[0], alpha=plotalpha)
ax[0].set_xlabel(r"$R_f/R_m$")
ax[0].set_ylabel("Action")
ax[0].axhline(y=1, lw=1, ls="--", color="C3", alpha=.7)

measerr_vals = aerr[:, 2]
ax[1].plot(gammas_all[:], measerr_vals, lw=plotlw, color=plotcolors[1], alpha=plotalpha)
ax[1].set_xlabel(r"$R_f/R_m$")
ax[1].set_ylabel("Meas. error")

moderr_vals = aerr[:, 3]
ax[2].plot(gammas_all[:], moderr_vals, lw=plotlw, color=plotcolors[2], alpha=plotalpha)
ax[2].set_xlabel(r"$R_f/R_m$")
ax[2].set_ylabel("Model error")

fig.suptitle("L96 action levels ($L=8$, single data set)", y=1.0)

for i in range(3):
    ax[i].set_yscale('log')
    ax[i].set_xscale('log')
    ax[i].set_xlim(gammas_all[0], gammas_all[-1])
    #ax[i].set_ylim(.001, 1.0e2)

plt.show()

Parameter estimates





In [43]:



    


#structure = np.array([25, 3, 4])
Nbias = np.sum(structure[1:])
Nweight = np.sum(structure[1:] * structure[:-1])

sortedp = []

West = np.load("MNIST/results/W.npy")
best = np.load("MNIST/results/b.npy")

for i in xrange(len(gammas_all)):
    #West = np.load("W.npy")
    #best = np.load("b.npy")
    for j, West_n in enumerate(West[i]):
        if j == 0:
            pest = West_n.flatten()#, best_n.flatten())
        else:
            pest = np.append(pest, West_n.flatten())
    for j, best_n in enumerate(best[i]):
        pest = np.append(pest, best_n.flatten())
    sortedp.append(pest.flatten())

sortedp = np.array(sortedp)



    











In [44]:



    


fig,ax = plt.subplots(1, 1, figsize=(5,4))
fig.set_tight_layout(True)

nrange = range(Nweight + Nbias)
surf = ax.pcolormesh(gammas_all, nrange, np.abs(sortedp.T), rasterized=True, cmap='jet', norm=mplcolors.LogNorm())#, vmin=1.0E-7, vmax=1.0E3)
#surf = ax.pcolormesh(g, nrange, sortedp.T, rasterized=True, cmap='jet')#, norm=mplcolors.LogNorm())
ax.set_xscale("log")
ax.set_xlabel(r"$R_f/R_m$")
ax.set_ylabel(r"Parameter metaindex")
cbar = fig.colorbar(surf, ax=ax)
cbar.set_label(r"$W$")
ax.set_xlim((gammas_all[0], gammas_all[-1]))
ax.set_ylim((nrange[0], nrange[-1]))
#ax.set_title("Est. err. vs. data, sec%d mem%d"%(secID, ensID))

plt.show()
#fig.savefig("/home/prozdeba/Desktop/plots_20170801/nnet/params_M%d.pdf"%(Ntraining,))
#plt.close()



    


















    




















    























In [ ]:

Content source: paulrozdeba/varanneal

Similar notebooks:

notebook.community | gallery | about