Baayen, R. H. and Milin, P. and Filipovic Durdevic, D. and Hendrix, P. and Marelli, M. 2011. "An amorphous model for morphological processing in visual comprehension based on naive discriminative learning." Psychological Review 118:438-482.
In [1]:
import pandas as pd
import pandas.rpy.common as com
import numpy as np
from sklearn.feature_extraction import DictVectorizer
%load_ext autoreload
%autoreload 2
%load_ext rmagic
%precision 2
pd.set_option('display.precision', 3)
In [2]:
%%R
library(ndl)
In [3]:
data = com.load_data('plurals')
data['Cues'] = [list(w) for w in data['WordForm']]
data['Outcomes'] = [w.split('_') for w in data['Outcomes']]
data
Out[3]:
In [4]:
cues = DictVectorizer(dtype=float,sparse=False)
D = cues.fit_transform({}.fromkeys(c,True) for c in data.Cues) * data.Frequency[:,np.newaxis]
D
Out[4]:
In [5]:
cues.get_feature_names()
Out[5]:
Now sum up to get $C$:
In [6]:
n = len(cues.get_feature_names())
C = np.zeros((n,n))
for row in D:
for nz in np.nonzero(row):
C[nz] += row
C
Out[6]:
Then we normalize to get $C'$, the conditional probabilty matrix (eqs. 38 and 39), where: $$C'_{ij}=p(j|i)=\frac{C_{ij}}{\sum_kC_{ik}}$$
In [7]:
Z = C.sum(axis=1)
C1 = C / Z[:,np.newaxis]
C1
Out[7]:
Next, Outcome matrix $O$, where $O_{ij}$ is number of types cue $i$ occurred with outcome $j$:
In [8]:
out = DictVectorizer(dtype=float,sparse=False)
X = out.fit_transform([{}.fromkeys(c,True) for c in data.Outcomes]) * data.Frequency[:,np.newaxis]
X
Out[8]:
In [9]:
out.get_feature_names()
Out[9]:
In [10]:
O = np.zeros((len(cues.get_feature_names()),len(out.get_feature_names())))
for i in xrange(len(X)):
for nz in np.nonzero(D[i]):
O[nz] += X[i]
O
Out[10]:
As above, we renormalize $O$ to get the conditional outcome matrix $O'$, where: $$O'_{ij}=p(o_j|c_i)=\frac{p(c_i,o_j)}{p(c_i)}=\frac{O_{ij}}{\sum_kC_{ik}}$$
In [11]:
O1 = O / Z[:,np.newaxis]
O1
Out[11]:
Finally, we find the weight matrix W by solving equation (47): $C'W=O'$
In [12]:
np.linalg.solve(C1,O1)
Out[12]:
Alternatively, find weight matrix $W$ using the pseudoinverse $C^+$ as in equation (48): $W=C^+O'$ This has the advantage of working even when $C$ is singular.
In [13]:
W = np.linalg.pinv(C1).dot(O1)
W
Out[13]:
In [17]:
pd.DataFrame(W,columns=out.get_feature_names(),index=cues.get_feature_names())
Out[17]:
Compute activations. Let $u$ be a vector of cues that are active for a given input. For example, for the input hands, we have:
In [20]:
u=cues.transform([{}.fromkeys(list('hands'),True)]).T
u
Out[20]:
Given $u$, the activation $a_j$ of a meaning $j$ is: $$a_j=\sum_iW_{ij}=W^Tu$$
In [21]:
W.T.dot(u)
Out[21]:
In [22]:
pd.DataFrame(W.T.dot(u),index=out.get_feature_names())
Out[22]:
In [27]:
targets = ['hands','hand']
pd.DataFrame(W.T.dot(cues.transform([{}.fromkeys(list(t),True) for t in targets]).T),index=out.get_feature_names(),columns=targets)
Out[27]:
The same thing, but packaged up in a function:
In [28]:
from ndl import *
In [29]:
ndl(data)
Out[29]:
In [ ]: