26 May 2016

I trained a simple fp_linear network (FingerprintLayer -> Linear Regression) to learn how to count the sum of all of the nodes. This was a sanity check before progressing further with training on actual HIV data.

I was expecting that most of the weights and biases should be some really small number close to zero, while the final linear regression weights should be something close to the array [0, 1, 2, 3, 4, ..., N] for N features.



In [26]:

    
import pickle as pkl
from pprint import pprint



In [44]:

    
def open_wb(path):
    with open(path, 'rb') as f:
        wb = pkl.load(f)
    
    return wb

wb = open_wb('../experiments/wbs/fp_linear-cf.score_sum-5000_iters-10_wb.pkl')



In [45]:

    
pprint(wb)









    



{'layer0_FingerprintLayer': {'weights': array([[ -1.82938430e-03,   1.28537180e-04,   1.25939735e-03,
         -1.37757490e-03,   1.38815480e-03,   7.95808946e-04,
          8.98581642e-04,  -1.75878735e-03,  -8.46583857e-04,
          5.67086313e-04],
       [  4.85434614e-04,  -5.58140318e-04,   5.93085394e-05,
         -1.53195501e-03,  -8.08616308e-04,   3.79263083e-04,
          1.62015871e-04,   5.25874353e-04,   9.00587594e-04,
         -2.62768840e-04],
       [  2.02132809e-03,  -7.55417203e-04,   3.93024240e-04,
          8.67096605e-04,   1.24515514e-04,   1.37086098e-03,
          3.03657589e-04,   1.15658204e-03,   9.38176307e-05,
         -1.08259882e-03],
       [ -6.49572415e-04,   1.43850684e-03,   1.00840703e-03,
          8.77183571e-04,   1.92033055e-03,  -2.49702660e-04,
         -2.17082453e-03,   2.67745361e-03,  -9.03158490e-05,
          1.11301540e-03],
       [ -6.66779649e-04,   1.51784505e-03,  -2.91507422e-04,
          1.57676961e-03,  -2.01433884e-04,   8.89911408e-04,
         -6.68276657e-04,   4.81024346e-04,  -1.00187225e-03,
          8.55244054e-04],
       [  9.23380222e-04,   7.29130574e-04,   2.37343933e-04,
          6.60071964e-04,   1.23282462e-04,   2.77194266e-04,
          1.32177302e-04,  -3.24088668e-04,   2.76534547e-04,
          9.64625347e-05],
       [  3.09904446e-04,   5.05393607e-04,   1.11774383e-03,
          9.20718821e-04,   5.46402681e-04,   8.02197189e-04,
          4.64945085e-04,   1.11330945e-03,   5.95947914e-04,
         -2.16061439e-03],
       [  3.42218303e-04,   7.09206912e-04,   1.62782264e-03,
         -4.12195632e-04,   6.67863401e-05,  -1.05862359e-04,
          9.94624215e-04,  -9.72875374e-04,   1.36233315e-04,
         -1.59348783e-03],
       [  8.80063346e-04,  -7.89120060e-04,  -7.45481277e-04,
         -8.55201322e-04,  -8.17038032e-04,   1.58628840e-03,
          7.20717793e-04,  -8.86885005e-04,  -2.16854701e-04,
         -1.72287481e-03],
       [ -8.63692087e-05,  -1.21558231e-03,   1.42079564e-04,
         -8.04164811e-05,   7.03723900e-04,   3.12766200e-03,
         -9.27187291e-04,   6.36031800e-04,  -9.19588672e-05,
          8.33672859e-04]])},
 'layer1_LinearRegressionLayer': {'bias': array([[ 0.33256752]]),
                                  'linweights': array([[-0.06453606],
       [ 0.95183146],
       [ 1.95748996],
       [ 2.95066376],
       [ 3.95140532],
       [ 4.93350427],
       [ 5.94740695],
       [ 6.94750695],
       [ 7.95568526],
       [ 8.93649513]])}}

Okay, focus in on the LinReg weights:



In [46]:

    
wb['layer1_LinearRegressionLayer']['linweights'].shape









    Out[46]:





(10, 1)



In [47]:

    
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

sns.set_style('white')
sns.set_context('poster')

%matplotlib inline



In [48]:

    
plt.bar(np.arange(1, 11) - 0.35, wb['layer1_LinearRegressionLayer']['linweights'])









    Out[48]:





<Container object of 10 artists>

Exactly what I was expecting! Yay!

I'm also curious to see what the weights and biases look like for score_sine.



In [ ]:

    
wb = open_wb('../experiments/wbs/')