In this notebook, we use the memexp module to show some examples of memory explorations using energetic as well as kinetic exploratory mechanisms. To begin, we load the module (as well as other modules that will be useful).
In [1]:
import memories.memexp as mx
%matplotlib notebook
import matplotlib.pyplot as plt
import os as os
import numpy as np
We will start studying the simple case of relaxation to a stored memory using stochastic dynamics. That is, we use a stochastic Hopfield model. This will introduce us to the Simulation class, as well as the Network class
In [2]:
MySim = mx.Simulation(model = 'hopfield', N = 300, p=10, mu=3, seed=100, beta = 100., swipes=300, ds=1.)
Above we have created a simulation object for a system with $N=500$ neurons and $p=10$ patterns. We have chosen patter $\mu=3$ as the target pattern, which means that the original state contains an overlap of 50 neurons with the target state pattern. Since we have fixed the inverse temperature to a large value $\beta=100$ the system will behave deterministically, as in Hopfield's original paper. We will run the simulation for $10^2$ swipes, which means $500\times 10^2$ time steps, and we will save the state fo the system every $ds=1$ swipes.
In [3]:
MySim.run()
Out[3]:
We can see that the system converged by checking the last state of the network class inside the simulation:
In [4]:
MySim.net.overlaps()
Out[4]:
Graphically the evolution can be seen by loading the stored data and plotting the overlaps over time. First we load the network class, which contains the network parameters and useful functions.
In [5]:
import pickle
net = pickle.load( open( MySim.path + '/net.pkl','rb') )
We now create the variables where we will store the overlaps.
In [6]:
neuron_t = np.zeros_like(net.N)
file_list =[ x for x in os.listdir(MySim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [7]:
for i in range(len(file_list)):
sigma = np.load(MySim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [8]:
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
Out[8]:
We now generate a simulation class for the kinetic dynamics (model = 'kinetic'), which runs over a longer time $T$ (swipes = 300). We choose values for the delay time $\tau$ (tau = 50.*300) and the non-equilibrium driving $\lambda$ (lambda=1.5). The length of the encoded sequence is k=3.
In [278]:
MyKinSim = mx.Simulation(model = 'kinetic', N = 300, p = 8, k = 3, seed=250, beta = 5., swipes=100*5, ds=1., lamda = 1.5, tau=150.*300)
We run the simulation
In [279]:
MyKinSim.run()
Out[279]:
Before evaluating the simulaion results, we create the variables where we will store the overlaps
In [280]:
net = pickle.load( open( MyKinSim.path + '/net.pkl','rb') )
neuron_t = np.zeros_like(MyKinSim.N)
file_list = [ x for x in os.listdir(MyKinSim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [281]:
for i in range(len(file_list)):
sigma = np.load(MyKinSim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [282]:
from matplotlib.ticker import MultipleLocator, FormatStrFormatter
ax = plt.subplot(111)
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
plt.axis([0, MyKinSim.swipes, -.4, 1.2])
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
ax.xaxis.set_major_locator(MultipleLocator(50))
ax.xaxis.grid(True,'major')
Test of analysis
In [23]:
import tools.evaluation as ev
ev.my_score((net.k, net.p), [overlaps_t], cutoff = 0.5, time_retrieved = 0.10 * net.tau / net.N)
Out[23]:
We generate a simulation class for the energetic dynamics case (model = 'energetic'), keeping all values equal to the kinetic case.
In [58]:
reload(mx)
MyEnSim = mx.Simulation(model = 'energetic', N = 300, p=10, k = 3, seed=250, beta = 3., swipes=15*100, ds=1., tau=150.*300)
We run the simulation, which is now slower. The reason is that for each time step the hebbian matrix has to be re-calculated.
In [59]:
MyEnSim.run()
Out[59]:
As before, we create the variables where the results will be stored
In [60]:
net = pickle.load( open( MyEnSim.path + '/net.pkl','rb') )
neuron_t = np.zeros_like(net.N)
file_list = [ x for x in os.listdir(MyEnSim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [61]:
for i in range(len(file_list)):
sigma = np.load(MyEnSim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [62]:
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
Out[62]:
In [64]:
import tools.evaluation as ev
ev.my_score((net.k, net.p), [overlaps_t], cutoff = 0.5, time_retrieved = 0.1 * net.tau / net.N)
Out[64]:
We generate a simulation class for the energetic dynamics case (model = 'energetic'), keeping all values equal to the kinetic case.
In [152]:
reload(mx)
MyEnSim = mx.Simulation(model = 'energetic-modified', N = 300, p=20, k = 3, seed=250, beta = 3., swipes=5*100, ds=1., tau=150.*300)
We run the simulation, which is now slower. The reason is that for each time step the hebbian matrix has to be re-calculated.
In [153]:
MyEnSim.run()
Out[153]:
As before, we create the variables where the results will be stored
In [154]:
net = pickle.load( open( MyEnSim.path + '/net.pkl','rb') )
neuron_t = np.zeros_like(net.N)
file_list = [ x for x in os.listdir(MyEnSim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [155]:
for i in range(len(file_list)):
sigma = np.load(MyEnSim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [156]:
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
Out[156]:
We now generate a simulation class for the kinetic dynamics (model = 'kinetic'), which runs over a longer time $T$ (swipes = 300). We choose values for the delay time $\tau$ (tau = 50.*300) and the non-equilibrium driving $\lambda$ (lambda=1.5).
In [232]:
reload(mx)
MyKinSim = mx.Simulation(model = 'kinetic', N = 300, p = 7, k = 10, correlation = 10, seed=250, beta = 5., swipes = 100*15, ds = 1., lamda = 1.5, tau=150.*300)
We run the simulation
In [233]:
MyKinSim.run()
Out[233]:
Before evaluating the simulaion results, we create the variables where we will store the overlaps
In [234]:
net = pickle.load( open( MyKinSim.path + '/net.pkl','rb') )
neuron_t = np.zeros_like(net.N)
file_list = [ x for x in os.listdir(MyKinSim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [235]:
for i in range(len(file_list)):
sigma = np.load(MyKinSim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [236]:
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
Out[236]:
We generate a simulation class for the energetic dynamics case (model = 'energetic'), keeping all values equal to the kinetic case.
In [291]:
reload(mx)
MyEnSim = mx.Simulation(model = 'energetic', N = 300, p = 10, k = 7, correlation = 30, seed=298, beta = 5., swipes = 100*15, ds=1., tau=150.*300)
We run the simulation, which is now slower. The reason is that for each time step the hebbian matrix has to be re-calculated.
In [292]:
MyEnSim.run()
Out[292]:
As before, we create the variables where the results will be stored
In [293]:
net = pickle.load( open( MyKinSim.path + '/net.pkl','rb') )
neuron_t = np.zeros_like(net.N)
file_list = [ x for x in os.listdir(MyEnSim.path) if x[0]=='S' ]
overlaps_t = np.zeros((len(file_list), net.p))
We can now read the files and calculate the overlaps over time
In [294]:
for i in range(len(file_list)):
sigma = np.load(MyEnSim.path + '/' + file_list[i])
net.neurons = sigma
overlaps_t[i,:] = net.overlaps()
Finally, we plot the overlaps
In [295]:
plt.ylabel('overlaps, $m^{\mu}$',fontdict={'fontsize':20})
plt.xlabel('time, ',fontdict={'fontsize':20})
for i in range(len(overlaps_t[0,:])):
plt.plot(overlaps_t[:,i],label='%d' % (i))
plt.legend(title='overlaps')
Out[295]:
In [300]:
import tools.evaluation as ev
ev.my_score((net.k, net.p), [overlaps_t], cutoff = 0.5, time_retrieved = 0.1 * net.tau / net.N)
Out[300]: