Parallel Forest Tutorial

This notebooke show the traing process of Parallel Randaom Forest. For cluster training please check https://github.com/wasit7/parallel_forest

import modules

Import all necessary modules



In [24]:

    
import numpy as np
from matplotlib import pyplot as plt
import pickle
import os
%pylab inline









    



Populating the interactive namespace from numpy and matplotlib

Generating datasets



In [25]:

    
clmax=5
spc=5e2
theta_range=2
samples=np.zeros(spc*clmax,dtype=np.uint32)
I=np.zeros((spc*clmax,theta_range),dtype=np.float32)
marker=['bo','co','go','ro','mo','yo','ko',
        'bs','cs','gs','rs','ms','ys','ks']

# number of datasets being generated 
# 8 for training
# another one for evaluation
N=9 
path="train/"
if not os.path.exists(path):
    os.makedirs(path)
for n in xrange(N):
    for cl in xrange(clmax):
        xo=cl*spc
        #define label
        samples[xo:xo+spc]=cl
        phi = np.linspace(0, 2*np.pi, spc) + \
        np.random.randn(spc)*0.4*np.pi/clmax + \
        2*np.pi*cl/clmax
        r = np.linspace(0.1, 1, spc)
        I[xo:xo+spc,:]=np.transpose(np.array([r*np.cos(phi), r*np.sin(phi)]))
    with open(path+'dataset%02d.pic'%(n), 'wb') as pickleFile:
    #write label and feature vector
        theta_dim=1
        pickle.dump((clmax,theta_dim,theta_range,len(samples),samples,I,None), pickleFile, pickle.HIGHEST_PROTOCOL)

Visualization of the dataset



In [26]:

    
z=np.random.randint( 0,spc*clmax,1000)
for i in z:
    #ax.plot(dset.I[i,0],dset.I[i,1],marker[dset2.samples[i]])
    plt.plot(I[i,0],I[i,1],marker[samples[i]])
    plt.hold(True)

Training



In [27]:

    
from pforest.master import master
m=master()
m.reset()
m.train()









    



master>>init() dsetname: dataset
master>> create dview
master>> init engine
Found pforest
debug:master:__init__: ['train/dataset00.pic', 'train/dataset01.pic', 'train/dataset02.pic', 'train/dataset03.pic', 'train/dataset04.pic', 'train/dataset05.pic', 'train/dataset06.pic', 'train/dataset07.pic']
master>> init local variables
master>>reset()
master>>reset() H: 2.3219
master>>reset() Q: 20000
master::train() node20*-
master::train() node19R-
master::train() node18R-
master::train() node17RQ
master::train() node17LQ
master::train() node18L-
master::train() node17R-
master::train() node16RQ
master::train() node16L-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13RG
master::train() node13LQ
master::train() node15LQ
master::train() node17L-
master::train() node16R-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13R-
master::train() node12R-
master::train() node11RQ
master::train() node11LG
master::train() node12LQ
master::train() node13LQ
master::train() node15LQ
master::train() node16LQ
master::train() node19L-
master::train() node18R-
master::train() node17R-
master::train() node16R-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13R-
master::train() node12R-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9R-
master::train() node8R-
master::train() node7RG
master::train() node7LQ
master::train() node8LQ
master::train() node9LQ
master::train() node11LQ
master::train() node12LQ
master::train() node13LQ
master::train() node15L-
master::train() node14R-
master::train() node13R-
master::train() node12R-
master::train() node11R-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8LG
master::train() node10LQ
master::train() node11LQ
master::train() node12L-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9RG
master::train() node9LQ
master::train() node11LQ
master::train() node13L-
master::train() node12R-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9R-
master::train() node8R-
master::train() node7RQ
master::train() node7L-
master::train() node6R-
master::train() node5RQ
master::train() node5LG
master::train() node6LQ
master::train() node8LQ
master::train() node9LQ
master::train() node11L-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8R-
master::train() node7RQ
master::train() node7L-
master::train() node6R-
master::train() node5RQ
master::train() node5L-
master::train() node4RQ
master::train() node4LG
master::train() node6LQ
master::train() node8LQ
master::train() node10L-
master::train() node9RQ
master::train() node9LG
master::train() node12L-
master::train() node11R-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8LG
master::train() node10L-
master::train() node9RQ
master::train() node9L-
master::train() node8R-
master::train() node7RQ
master::train() node7L-
master::train() node6R-
master::train() node5RQ
master::train() node5LG
master::train() node6LQ
master::train() node8LQ
master::train() node11L-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8LQ
master::train() node10L-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8LG
master::train() node14L-
master::train() node13R-
master::train() node12R-
master::train() node11RQ
master::train() node11L-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8R-
master::train() node7RQ
master::train() node7LG
master::train() node8LQ
master::train() node10LQ
master::train() node12LQ
master::train() node13L-
master::train() node12RQ
master::train() node12L-
master::train() node11R-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8L-
master::train() node7RQ
master::train() node7L-
master::train() node6RQ
master::train() node6L-
master::train() node5RQ
master::train() node5LQ
master::train() node10LQ
master::train() node11LQ
master::train() node16L-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13R-
master::train() node12R-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9RQ
master::train() node9LG
master::train() node12LQ
master::train() node13LQ
master::train() node15L-
master::train() node14R-
master::train() node13R-
master::train() node12RQ
master::train() node12L-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10LG
master::train() node13LQ
master::train() node14LQ
master::train() node17L-
master::train() node16R-
master::train() node15R-
master::train() node14RQ
master::train() node14LQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13L-
master::train() node12RQ
master::train() node12L-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9RG
master::train() node9LQ
master::train() node11LQ
master::train() node14L-
master::train() node13RQ
master::train() node13LQ
master::train() node16L-
master::train() node15RQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13L-
master::train() node12RQ
master::train() node12L-
master::train() node11RQ
master::train() node11L-
master::train() node10RG
master::train() node10LQ
master::train() node14LQ
master::train() node18L-
master::train() node17R-
master::train() node16RQ
master::train() node16L-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13RQ
master::train() node13L-
master::train() node12R-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9RQ
master::train() node9LG
master::train() node12LQ
master::train() node15LQ
master::train() node17L-
master::train() node16RQ
master::train() node16L-
master::train() node15RQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13LG
master::train() node14LQ

Write and read the tree

You may need to save/load the tree to/from a pickle file



In [28]:

    
with open('out_tree.pic', 'wb') as pickleFile:
    pickle.dump(m.root, pickleFile, pickle.HIGHEST_PROTOCOL)
    
with open('out_tree.pic', 'rb') as pickleFile:
    root = pickle.load(pickleFile)

Check the file size



In [29]:

    
ls









    



 Volume in drive C has no label.
 Volume Serial Number is A8BF-C894

 Directory of C:\Users\Wasit\Documents\GitHub\tutorials\notebooks

25/11/2015  02:51    <DIR>          .
25/11/2015  02:51    <DIR>          ..
25/11/2015  01:39    <DIR>          .ipynb_checkpoints
08/11/2015  03:02             2,280 Basic Python.ipynb
24/11/2015  23:51    <DIR>          dataset
25/11/2015  01:51             3,074 ipcluster.ipynb
25/11/2015  02:51            50,697 out_tree.pic
25/11/2015  02:51            67,568 parallel forest.ipynb
25/11/2015  00:50    <DIR>          train
               4 File(s)        123,619 bytes
               5 Dir(s)  106,478,432,256 bytes free

The result decision tree

Termination code (Q:min bag size, G:no information gain, D:reaching maximum depth)



In [30]:

    
from pforest.dataset import dataset
from pforest.tree import tree

#init the test tree
t=tree()
t.settree(root)
t.show()









    



*- 20 H:2.322e+00,Q:020000 tau:0.545468151569 theta:[ 1.]
L- 19 H:2.312e+00,Q:017665 tau:-0.604479908943 theta:[ 0.]
L- 18 H:1.300e+00,Q:001696 tau:-0.779527366161 theta:[ 0.]
L- 17 H:6.386e-01,Q:000675 tau:0.274054408073 theta:[ 1.]
L- 16 H:3.672e-01,Q:000571 tau:-0.83270907402 theta:[ 0.]
L- 15 H:9.085e-02,Q:000462 tau:-0.960045099258 theta:[ 0.]
LQ 14 H:6.455e-01,Q:000019 (cl,P):(003,0.74) (002,0.26) (004,0.00)
R- 14 H:1.639e-02,Q:000443 tau:-0.840126633644 theta:[ 0.]
LG 13 H:1.160e-37,Q:000427 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 13 H:1.722e-01,Q:000016 (cl,P):(003,0.94) (004,0.06) (002,0.00)
RQ 15 H:8.064e-01,Q:000109 (cl,P):(003,0.72) (004,0.28) (002,0.00)
RQ 16 H:8.344e-01,Q:000104 (cl,P):(003,0.56) (002,0.44) (004,0.00)
R- 17 H:1.015e+00,Q:001021 tau:0.373595952988 theta:[ 1.]
L- 16 H:7.615e-01,Q:000879 tau:-0.580779314041 theta:[ 1.]
LQ 15 H:8.222e-01,Q:000070 (cl,P):(003,0.73) (004,0.27) (002,0.00)
R- 15 H:5.940e-01,Q:000809 tau:0.228774279356 theta:[ 1.]
L- 14 H:3.869e-01,Q:000698 tau:-0.629576921463 theta:[ 0.]
L- 13 H:1.607e-01,Q:000615 tau:-0.494080752134 theta:[ 1.]
LQ 12 H:3.963e-01,Q:000040 (cl,P):(004,0.88) (003,0.12) (002,0.00)
R- 12 H:1.071e-01,Q:000575 tau:-0.658746540546 theta:[ 0.]
L- 11 H:2.914e-02,Q:000506 tau:0.0939490869641 theta:[ 1.]
L- 10 H:1.629e-02,Q:000434 tau:-0.67863202095 theta:[ 0.]
LG 09 H:1.263e-37,Q:000392 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 09 H:8.594e-02,Q:000042 (cl,P):(004,0.98) (000,0.02) (003,0.00)
RQ 10 H:6.897e-02,Q:000072 (cl,P):(004,0.99) (003,0.01) (002,0.00)
RQ 11 H:4.131e-01,Q:000069 (cl,P):(004,0.90) (000,0.10) (003,0.00)
RQ 13 H:8.548e-01,Q:000083 (cl,P):(004,0.57) (000,0.43) (003,0.00)
RQ 14 H:8.903e-01,Q:000111 (cl,P):(004,0.64) (003,0.36) (002,0.00)
RQ 16 H:7.844e-02,Q:000142 (cl,P):(003,0.99) (004,0.01) (002,0.01)
R- 18 H:2.300e+00,Q:015969 tau:-0.249093413353 theta:[ 0.]
L- 17 H:1.775e+00,Q:003227 tau:-0.44518700242 theta:[ 0.]
L- 16 H:1.229e+00,Q:001194 tau:0.223648548126 theta:[ 1.]
L- 15 H:9.195e-01,Q:000932 tau:-0.495375275612 theta:[ 1.]
LQ 14 H:7.998e-01,Q:000200 (cl,P):(004,0.81) (000,0.14) (003,0.06)
R- 14 H:2.793e-01,Q:000732 tau:-0.46365454793 theta:[ 0.]
L- 13 H:1.453e-01,Q:000673 tau:0.120921842754 theta:[ 1.]
L- 12 H:8.629e-02,Q:000600 tau:-0.485971719027 theta:[ 0.]
L- 11 H:2.773e-02,Q:000546 tau:-0.491587013006 theta:[ 1.]
LQ 10 H:6.667e-01,Q:000003 (cl,P):(004,0.67) (000,0.33) (003,0.00)
RG 10 H:9.150e-38,Q:000543 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 11 H:3.480e-01,Q:000054 (cl,P):(000,0.91) (001,0.09) (004,0.00)
RQ 12 H:3.740e-01,Q:000073 (cl,P):(000,0.92) (004,0.08) (003,0.00)
RQ 13 H:7.243e-01,Q:000059 (cl,P):(000,0.69) (001,0.31) (004,0.00)
RQ 15 H:9.356e-01,Q:000262 (cl,P):(004,0.78) (003,0.14) (000,0.08)
R- 16 H:1.676e+00,Q:002033 tau:0.236698538065 theta:[ 1.]
L- 15 H:1.477e+00,Q:001574 tau:-0.519201993942 theta:[ 1.]
L- 14 H:1.096e+00,Q:000381 tau:-0.722985446453 theta:[ 1.]
LQ 13 H:1.970e-01,Q:000173 (cl,P):(004,0.97) (003,0.02) (000,0.02)
RQ 13 H:2.257e-01,Q:000208 (cl,P):(000,0.96) (004,0.02) (001,0.02)
R- 14 H:8.221e-01,Q:001193 tau:-0.294435083866 theta:[ 0.]
L- 13 H:5.546e-01,Q:000920 tau:0.120753273368 theta:[ 1.]
L- 12 H:3.183e-01,Q:000817 tau:-0.382931858301 theta:[ 1.]
LQ 11 H:9.090e-01,Q:000090 (cl,P):(001,0.57) (000,0.43) (004,0.00)
R- 11 H:7.374e-02,Q:000727 tau:-0.305050551891 theta:[ 0.]
L- 10 H:4.318e-02,Q:000683 tau:-0.419010609388 theta:[ 0.]
LQ 09 H:2.432e-01,Q:000063 (cl,P):(001,0.94) (000,0.06) (004,0.00)
RG 09 H:8.025e-38,Q:000620 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 10 H:2.250e-01,Q:000044 (cl,P):(001,0.93) (002,0.07) (004,0.00)
RQ 12 H:8.719e-01,Q:000103 (cl,P):(000,0.68) (001,0.32) (004,0.00)
RQ 13 H:9.095e-01,Q:000273 (cl,P):(001,0.67) (002,0.33) (000,0.00)
R- 15 H:9.935e-01,Q:000459 tau:0.389305233955 theta:[ 1.]
LQ 14 H:9.889e-02,Q:000255 (cl,P):(000,0.98) (004,0.01) (001,0.00)
RQ 14 H:1.667e-01,Q:000204 (cl,P):(004,0.97) (000,0.03) (003,0.00)
R- 17 H:2.318e+00,Q:012742 tau:-0.506381392479 theta:[ 1.]
L- 16 H:1.529e+00,Q:001941 tau:-0.687939465046 theta:[ 1.]
L- 15 H:9.790e-01,Q:000960 tau:-0.847006201744 theta:[ 1.]
LQ 14 H:9.224e-02,Q:000351 (cl,P):(004,0.99) (000,0.01) (003,0.00)
R- 14 H:2.751e-01,Q:000609 tau:-0.796041846275 theta:[ 1.]
LQ 13 H:5.408e-01,Q:000129 (cl,P):(000,0.84) (004,0.16) (003,0.00)
R- 13 H:1.144e-01,Q:000480 tau:0.571691632271 theta:[ 0.]
L- 12 H:4.715e-02,Q:000468 tau:-0.700112581253 theta:[ 1.]
L- 11 H:1.626e-02,Q:000433 tau:0.485592007637 theta:[ 0.]
LG 10 H:1.229e-37,Q:000403 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 10 H:1.082e-01,Q:000030 (cl,P):(000,0.97) (004,0.03) (003,0.00)
RQ 11 H:2.274e-01,Q:000035 (cl,P):(000,0.94) (001,0.06) (004,0.00)
RQ 12 H:1.667e-01,Q:000012 (cl,P):(000,0.58) (004,0.42) (003,0.00)
R- 15 H:8.826e-01,Q:000981 tau:0.570904374123 theta:[ 0.]
L- 14 H:5.292e-01,Q:000798 tau:-0.620866119862 theta:[ 1.]
LQ 13 H:9.408e-01,Q:000205 (cl,P):(001,0.62) (000,0.38) (004,0.00)
R- 13 H:1.378e-01,Q:000593 tau:-0.17462913692 theta:[ 0.]
LQ 12 H:2.913e-01,Q:000039 (cl,P):(001,0.92) (000,0.08) (004,0.00)
R- 12 H:1.049e-01,Q:000554 tau:0.497803211212 theta:[ 0.]
L- 11 H:6.604e-02,Q:000537 tau:-0.529267311096 theta:[ 1.]
L- 10 H:2.648e-02,Q:000472 tau:-0.554842293262 theta:[ 1.]
LG 09 H:1.352e-37,Q:000366 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 09 H:8.017e-02,Q:000106 (cl,P):(001,0.98) (002,0.02) (004,0.00)
RQ 10 H:1.843e-01,Q:000065 (cl,P):(001,0.95) (002,0.05) (004,0.00)
RQ 11 H:4.476e-01,Q:000017 (cl,P):(001,0.82) (000,0.18) (004,0.00)
RQ 14 H:1.229e-01,Q:000183 (cl,P):(000,0.98) (001,0.02) (004,0.01)
R- 16 H:2.301e+00,Q:010801 tau:0.686551511288 theta:[ 0.]
L- 15 H:2.257e+00,Q:009722 tau:-0.17086930573 theta:[ 1.]
L- 14 H:1.433e+00,Q:002630 tau:-0.333637356758 theta:[ 1.]
L- 13 H:9.229e-01,Q:001117 tau:0.470237374306 theta:[ 0.]
L- 12 H:6.371e-01,Q:000915 tau:-0.146611586213 theta:[ 0.]
LQ 11 H:6.661e-01,Q:000109 (cl,P):(001,0.81) (002,0.19) (004,0.00)
R- 11 H:3.116e-01,Q:000806 tau:-0.460010528564 theta:[ 1.]
LQ 10 H:7.765e-01,Q:000101 (cl,P):(002,0.73) (001,0.27) (004,0.00)
R- 10 H:1.252e-01,Q:000705 tau:-0.33806887269 theta:[ 1.]
L- 09 H:7.443e-02,Q:000684 tau:-0.343499451876 theta:[ 1.]
L- 08 H:3.552e-02,Q:000655 tau:-0.357903450727 theta:[ 1.]
L- 07 H:2.577e-02,Q:000593 tau:0.424847126007 theta:[ 0.]
L- 06 H:1.318e-02,Q:000580 tau:0.145782753825 theta:[ 0.]
LQ 05 H:1.544e-37,Q:000320 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 05 H:2.503e-02,Q:000260 (cl,P):(002,1.00) (003,0.00) (004,0.00)
RQ 06 H:3.662e-36,Q:000013 (cl,P):(002,0.92) (001,0.08) (004,0.00)
RQ 07 H:7.564e-02,Q:000062 (cl,P):(002,0.98) (003,0.02) (004,0.00)
RQ 08 H:3.666e-01,Q:000029 (cl,P):(002,0.86) (003,0.14) (004,0.00)
RQ 09 H:3.857e-01,Q:000021 (cl,P):(002,0.71) (003,0.29) (004,0.00)
RQ 12 H:2.673e-01,Q:000202 (cl,P):(001,0.95) (002,0.05) (004,0.00)
R- 13 H:1.141e+00,Q:001513 tau:-0.0972370132804 theta:[ 0.]
LQ 12 H:2.536e-01,Q:000332 (cl,P):(002,0.95) (003,0.05) (004,0.00)
R- 12 H:9.883e-01,Q:001181 tau:0.362753868103 theta:[ 0.]
L- 11 H:2.428e-01,Q:000889 tau:-0.284081041813 theta:[ 1.]
LQ 10 H:5.299e-01,Q:000126 (cl,P):(003,0.86) (002,0.14) (004,0.00)
R- 10 H:1.343e-01,Q:000763 tau:-0.195414304733 theta:[ 1.]
L- 09 H:4.673e-02,Q:000627 tau:-0.062436517328 theta:[ 0.]
LQ 08 H:2.606e-01,Q:000035 (cl,P):(003,0.91) (002,0.09) (004,0.00)
R- 08 H:1.297e-02,Q:000592 tau:-0.199384406209 theta:[ 1.]
LG 07 H:8.751e-38,Q:000568 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 07 H:1.352e-01,Q:000024 (cl,P):(003,0.96) (004,0.04) (002,0.00)
RQ 09 H:3.553e-01,Q:000136 (cl,P):(003,0.93) (004,0.07) (002,0.00)
RQ 11 H:7.951e-01,Q:000292 (cl,P):(002,0.81) (001,0.11) (003,0.08)
R- 14 H:2.273e+00,Q:007092 tau:0.33323392272 theta:[ 0.]
L- 13 H:2.164e+00,Q:005445 tau:0.0156290736049 theta:[ 0.]
L- 12 H:2.226e+00,Q:002547 tau:0.0897820070386 theta:[ 1.]
L- 11 H:1.057e+00,Q:001124 tau:-0.114449612796 theta:[ 0.]
L- 10 H:3.113e-01,Q:000635 tau:-0.163609325886 theta:[ 0.]
L- 09 H:1.693e-02,Q:000429 tau:0.0598266460001 theta:[ 1.]
LG 08 H:1.211e-37,Q:000409 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 08 H:1.805e-01,Q:000020 (cl,P):(002,0.95) (001,0.05) (004,0.00)
RQ 09 H:6.324e-01,Q:000206 (cl,P):(002,0.83) (003,0.17) (004,0.00)
R- 10 H:1.037e+00,Q:000489 tau:-0.0109110260382 theta:[ 1.]
L- 09 H:3.184e-01,Q:000369 tau:-0.0332340821624 theta:[ 0.]
LQ 08 H:1.502e-37,Q:000329 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 08 H:7.832e-01,Q:000040 (cl,P):(004,0.57) (003,0.42) (002,0.00)
RQ 09 H:8.989e-02,Q:000120 (cl,P):(002,0.98) (003,0.02) (004,0.00)
R- 11 H:1.583e+00,Q:001423 tau:0.279210358858 theta:[ 1.]
L- 10 H:4.381e-01,Q:000752 tau:0.247425511479 theta:[ 1.]
L- 09 H:2.759e-01,Q:000720 tau:0.11601690203 theta:[ 1.]
LQ 08 H:9.293e-01,Q:000071 (cl,P):(001,0.59) (002,0.41) (004,0.00)
R- 08 H:4.788e-02,Q:000649 tau:0.222964897752 theta:[ 1.]
L- 07 H:2.553e-02,Q:000601 tau:0.133864820004 theta:[ 1.]
LQ 06 H:5.500e-02,Q:000059 (cl,P):(001,0.98) (002,0.02) (004,0.00)
R- 06 H:1.395e-02,Q:000542 tau:0.00715682189912 theta:[ 0.]
LG 05 H:9.478e-38,Q:000524 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 05 H:1.111e-01,Q:000018 (cl,P):(001,0.94) (000,0.06) (004,0.00)
RQ 07 H:1.565e-01,Q:000048 (cl,P):(001,0.96) (000,0.04) (004,0.00)
RQ 09 H:6.766e-01,Q:000032 (cl,P):(000,0.72) (001,0.28) (004,0.00)
R- 10 H:9.526e-01,Q:000671 tau:0.40406832099 theta:[ 1.]
L- 09 H:1.756e-02,Q:000401 tau:0.391904950142 theta:[ 1.]
LG 08 H:1.309e-37,Q:000378 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 08 H:1.198e-01,Q:000023 (cl,P):(000,0.96) (004,0.04) (003,0.00)
RQ 09 H:3.493e-01,Q:000270 (cl,P):(004,0.93) (000,0.07) (003,0.00)
R- 12 H:1.403e+00,Q:002898 tau:0.193815156817 theta:[ 0.]
L- 11 H:1.349e+00,Q:001873 tau:-0.0336667634547 theta:[ 1.]
L- 10 H:1.668e-01,Q:000506 tau:-0.0882625207305 theta:[ 1.]
LG 09 H:1.211e-37,Q:000409 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 09 H:5.289e-01,Q:000097 (cl,P):(004,0.86) (000,0.14) (003,0.00)
R- 10 H:1.231e+00,Q:001367 tau:0.33358669281 theta:[ 1.]
L- 09 H:6.765e-01,Q:001067 tau:0.0716655105352 theta:[ 0.]
LQ 08 H:9.834e-01,Q:000278 (cl,P):(001,0.56) (000,0.44) (004,0.00)
R- 08 H:1.275e-01,Q:000789 tau:0.173603877425 theta:[ 0.]
L- 07 H:7.711e-02,Q:000718 tau:0.0971178635955 theta:[ 0.]
LQ 06 H:2.470e-01,Q:000078 (cl,P):(000,0.95) (001,0.05) (004,0.00)
R- 06 H:3.631e-02,Q:000640 tau:0.219309598207 theta:[ 1.]
L- 05 H:1.350e-02,Q:000547 tau:0.167662203312 theta:[ 0.]
LG 04 H:9.532e-38,Q:000521 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 04 H:1.060e-01,Q:000026 (cl,P):(000,0.96) (004,0.04) (003,0.00)
RQ 05 H:1.054e-01,Q:000093 (cl,P):(000,0.98) (004,0.02) (003,0.00)
RQ 07 H:3.735e-01,Q:000071 (cl,P):(000,0.92) (004,0.08) (003,0.00)
RQ 09 H:5.514e-01,Q:000300 (cl,P):(004,0.88) (000,0.10) (003,0.02)
R- 11 H:6.469e-01,Q:001025 tau:0.457582592964 theta:[ 1.]
L- 10 H:4.304e-01,Q:000955 tau:-0.132763758302 theta:[ 1.]
LQ 09 H:8.995e-01,Q:000035 (cl,P):(003,0.63) (004,0.37) (002,0.00)
R- 09 H:3.264e-01,Q:000920 tau:0.234841004014 theta:[ 0.]
LQ 08 H:5.029e-01,Q:000222 (cl,P):(004,0.88) (000,0.12) (003,0.00)
R- 08 H:1.798e-01,Q:000698 tau:0.354279518127 theta:[ 1.]
L- 07 H:6.814e-02,Q:000654 tau:-0.0986341834068 theta:[ 1.]
LQ 06 H:7.194e-01,Q:000016 (cl,P):(004,0.69) (003,0.31) (002,0.00)
R- 06 H:1.207e-02,Q:000638 tau:0.328137129545 theta:[ 0.]
LG 05 H:8.129e-38,Q:000612 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 05 H:1.388e-01,Q:000026 (cl,P):(004,0.96) (003,0.04) (002,0.00)
RQ 07 H:7.580e-01,Q:000044 (cl,P):(004,0.68) (003,0.32) (002,0.00)
RQ 10 H:1.510e-01,Q:000070 (cl,P):(003,0.96) (004,0.04) (002,0.00)
R- 13 H:1.255e+00,Q:001647 tau:0.510810375214 theta:[ 0.]
L- 12 H:5.879e-01,Q:000954 tau:0.383464396 theta:[ 0.]
LQ 11 H:9.435e-01,Q:000239 (cl,P):(003,0.63) (004,0.37) (002,0.00)
R- 11 H:2.074e-01,Q:000715 tau:0.427142649889 theta:[ 1.]
L- 10 H:3.525e-02,Q:000662 tau:0.404521465302 theta:[ 0.]
LQ 09 H:1.729e-01,Q:000084 (cl,P):(003,0.96) (004,0.04) (002,0.00)
RG 09 H:8.602e-38,Q:000578 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 10 H:8.851e-01,Q:000053 (cl,P):(003,0.64) (002,0.36) (004,0.00)
R- 12 H:4.070e-01,Q:000693 tau:0.540873348713 theta:[ 0.]
LQ 11 H:8.932e-01,Q:000075 (cl,P):(003,0.57) (002,0.43) (004,0.00)
R- 11 H:1.118e-01,Q:000618 tau:0.553581237793 theta:[ 0.]
LQ 10 H:5.217e-01,Q:000023 (cl,P):(002,0.78) (003,0.22) (004,0.00)
R- 10 H:5.118e-02,Q:000595 tau:0.501124083996 theta:[ 1.]
L- 09 H:2.619e-02,Q:000570 tau:0.672462165356 theta:[ 0.]
LG 08 H:9.424e-38,Q:000527 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 08 H:1.480e-01,Q:000043 (cl,P):(002,0.95) (001,0.05) (004,0.00)
RQ 09 H:1.298e-01,Q:000025 (cl,P):(002,0.92) (001,0.08) (004,0.00)
R- 15 H:1.142e+00,Q:001079 tau:0.882355332375 theta:[ 0.]
L- 14 H:9.118e-01,Q:000817 tau:-0.304201304913 theta:[ 1.]
LQ 13 H:6.089e-01,Q:000144 (cl,P):(000,0.83) (001,0.17) (004,0.00)
R- 13 H:4.504e-01,Q:000673 tau:0.720105588436 theta:[ 0.]
LQ 12 H:9.634e-01,Q:000085 (cl,P):(001,0.53) (002,0.47) (004,0.00)
R- 12 H:1.646e-01,Q:000588 tau:-0.274117946625 theta:[ 1.]
LQ 11 H:6.478e-01,Q:000016 (cl,P):(001,0.62) (000,0.38) (004,0.00)
R- 11 H:1.060e-01,Q:000572 tau:0.856958866119 theta:[ 0.]
L- 10 H:5.606e-02,Q:000521 tau:0.745365321636 theta:[ 0.]
LQ 09 H:1.444e-01,Q:000052 (cl,P):(001,0.96) (002,0.04) (004,0.00)
R- 09 H:3.034e-02,Q:000469 tau:-0.194502726197 theta:[ 1.]
LQ 08 H:1.060e-01,Q:000026 (cl,P):(001,0.96) (000,0.04) (004,0.00)
R- 08 H:1.590e-02,Q:000443 tau:0.77071428299 theta:[ 0.]
LQ 07 H:5.752e-02,Q:000072 (cl,P):(001,0.99) (002,0.01) (004,0.00)
RG 07 H:1.334e-37,Q:000371 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 10 H:3.302e-01,Q:000051 (cl,P):(001,0.92) (000,0.08) (004,0.00)
RQ 14 H:3.709e-01,Q:000262 (cl,P):(000,0.92) (001,0.08) (004,0.00)
R- 19 H:1.651e+00,Q:002335 tau:0.313386648893 theta:[ 0.]
L- 18 H:1.404e+00,Q:001760 tau:0.740284323692 theta:[ 1.]
L- 17 H:9.775e-01,Q:001001 tau:-0.556333303452 theta:[ 0.]
LQ 16 H:7.882e-01,Q:000166 (cl,P):(002,0.75) (003,0.25) (004,0.00)
R- 16 H:6.333e-01,Q:000835 tau:0.601712346077 theta:[ 1.]
LQ 15 H:9.034e-01,Q:000224 (cl,P):(003,0.67) (004,0.33) (002,0.00)
R- 15 H:2.891e-01,Q:000611 tau:0.161822989583 theta:[ 0.]
L- 14 H:9.059e-02,Q:000529 tau:-0.511734843254 theta:[ 0.]
LQ 13 H:2.655e-36,Q:000009 (cl,P):(003,0.67) (002,0.33) (004,0.00)
R- 13 H:5.286e-02,Q:000520 tau:0.624143183231 theta:[ 1.]
LQ 12 H:1.549e-01,Q:000051 (cl,P):(003,0.94) (004,0.06) (002,0.00)
R- 12 H:1.482e-02,Q:000469 tau:0.718567669392 theta:[ 1.]
LG 11 H:1.182e-37,Q:000419 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 11 H:5.510e-02,Q:000050 (cl,P):(003,0.98) (002,0.02) (004,0.00)
RQ 14 H:8.111e-01,Q:000082 (cl,P):(003,0.72) (002,0.28) (004,0.00)
R- 17 H:7.056e-01,Q:000759 tau:0.907142579556 theta:[ 1.]
L- 16 H:3.719e-01,Q:000651 tau:0.782421529293 theta:[ 1.]
LQ 15 H:8.070e-01,Q:000132 (cl,P):(002,0.72) (003,0.28) (004,0.00)
R- 15 H:9.676e-02,Q:000519 tau:0.274942785501 theta:[ 0.]
L- 14 H:2.922e-02,Q:000507 tau:0.794368624687 theta:[ 1.]
LQ 13 H:1.474e-01,Q:000054 (cl,P):(002,0.96) (003,0.04) (004,0.00)
RG 13 H:1.095e-37,Q:000453 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 14 H:5.000e-01,Q:000012 (cl,P):(002,0.58) (001,0.42) (004,0.00)
RQ 16 H:9.271e-01,Q:000108 (cl,P):(001,0.58) (002,0.42) (004,0.00)
R- 18 H:1.098e+00,Q:000575 tau:0.541333675385 theta:[ 0.]
LQ 17 H:1.126e+00,Q:000359 (cl,P):(002,0.59) (001,0.38) (003,0.03)
RQ 17 H:3.791e-01,Q:000216 (cl,P):(001,0.92) (002,0.07) (000,0.00)

Recall rate

Loading a new dataset, the last on, for computing a recall rate



In [31]:

    
#load the last dataset that never use for training
dset=dataset(8)
correct=0;
for x in xrange(dset.size):
    L=t.getL(np.array([x]),dset)
    if dset.getL(x) == L:
        correct=correct+1
    dset.setL(x,L)
print("recall rate: {}%".format(correct/float(dset.size)*100))









    



recall rate: 90.56%

Labelling

The computer use the decision tree to classify the unknown feature vector u



In [32]:

    
#setup the new test-set
#load dataset     
dset=dataset(8)
d=0.05
y, x = np.mgrid[slice(-1, 1+d, d), slice(-1, 1+d, d)]

#start labeling
L=np.zeros(x.shape,dtype=int)
for r in xrange(x.shape[0]):
    for c in xrange(x.shape[1]):
        u=( x[r,c],y[r,c] )
        Prob=t.classify(u)
        L[r,c]=np.argmax(Prob)

2D space partitioning by the decision tree

Displaying the labelled result



In [33]:

    
%matplotlib inline
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([-1,1,-1,1])
ax.pcolor(x,y,L)
ax.hold(True)

Overlay the dataset



In [34]:

    
z=np.random.randint(0,dset.size,1000)
for i in z:
    ax.plot(dset.I[i,0],dset.I[i,1],marker[dset.samples[i]])
fig









    Out[34]: