Sequences

In this notebook we show that the mechanism of sequence bulding in the sense of Amit both for synchronous and asynchronous processes.



In [1]:

    
from __future__ import print_function
import sys
sys.path.append('../')

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib.gridspec as gridspec
import seaborn as sns

from hopfield import HopfieldSequence

%matplotlib inline
sns.set(font_scale=2.0)

Synchronous

Parameters



In [2]:

    
prng = np.random.RandomState(seed=101)

n_dim = 200
n_store = 10
q = 10
T = 0.0
tau = 5
N = 150

Then we create the patterns that we store and the network



In [3]:

    
list_of_patterns = [np.sign(np.random.normal(size=n_dim)) for i in range(n_store)]
list_of_patterns_sequence = list_of_patterns[:q]

nn = HopfieldSequence(n_dim=n_dim, tau=tau, g_delay=2.0, T=T, prng=prng)
nn.train(list_of_patterns, normalize=True)
nn.train_delays(list_of_patterns_sequence, normalize=True)

Now we run the simulation



In [4]:

    
nn.s = np.copy(list_of_patterns[0])
history = np.zeros((N, n_store))
for i in range(N):
    nn.update_sync()
    # nn.update_async_random_sequence()
    # nn.update_async()
    history[i, :] = nn.calculate_overlap()

Plotting



In [5]:

    
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)

for index, overlap in enumerate(history.T):
    ax.plot(overlap, '-*', label=str(index))

ax.set_ylim([-0.2, 1.2])
ax.set_title('Overlap vs Time')
ax.legend()
plt.show()

Asynchronous update



In [6]:

    
nn = HopfieldSequence(n_dim=n_dim, tau=tau, g_delay=2.0, T=T, prng=prng)
nn.train(list_of_patterns, normalize=True)
nn.train_delays(list_of_patterns_sequence, normalize=True)



In [7]:

    
nn.s = np.copy(list_of_patterns[0])
history = np.zeros((N, n_store))
for i in range(N):
    nn.update_async_random_sequence()
    history[i, :] = nn.calculate_overlap()

Plotting



In [8]:

    
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)

for index, overlap in enumerate(history.T):
    ax.plot(overlap, '-*', label=str(index))

ax.set_ylim([-0.2, 1.2])
ax.set_title('Overlap vs Time')
ax.legend()
plt.show()

Other examples

Let's run the simulations above with different values to get a feeling of how they actually cope with each other



In [9]:

    
prng = np.random.RandomState(seed=101)

n_dim = 300
n_store = 10
q = 10
T = 0.0
tau = 10
N = q * n_store + q

list_of_patterns = [np.sign(np.random.normal(size=n_dim)) for i in range(n_store)]
list_of_patterns_sequence = list_of_patterns[:q]

nn = HopfieldSequence(n_dim=n_dim, tau=tau, g_delay=1.5, T=T, prng=prng)
nn.train(list_of_patterns, normalize=True)
nn.train_delays(list_of_patterns_sequence, normalize=True)



In [10]:

    
nn.s = np.copy(list_of_patterns[0])
history = np.zeros((N, n_store))
for i in range(N):
    nn.update_sync()
    # nn.update_async_random_sequence()
    # nn.update_async()
    history[i, :] = nn.calculate_overlap()



In [11]:

    
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)

cmap = cm.terrain
cmap = cm.brg
norm = cm.colors.Normalize(vmin=0, vmax=n_store)

for index, overlap in enumerate(history.T):
    ax.plot(overlap, '-*', color=cmap(norm(index)), label=str(index))

ax.set_ylim([-0.2, 1.2])
ax.set_title('Overlap vs Time')
ax.legend()
plt.show()



In [ ]: