Parallel Forest Tutorial

This notebooke show the traing process of Parallel Randaom Forest. For cluster training please check https://github.com/wasit7/parallel_forest

import modules

Import all necessary modules



In [1]:

    
import numpy as np
from matplotlib import pyplot as plt
import pickle
import os
%pylab inline









    



Populating the interactive namespace from numpy and matplotlib

Generating datasets



In [2]:

    
clmax=5
spc=5e2
theta_range=2
samples=np.zeros(spc*clmax,dtype=np.uint32)
I=np.zeros((spc*clmax,theta_range),dtype=np.float32)
marker=['bo','co','go','ro','mo','yo','ko',
        'bs','cs','gs','rs','ms','ys','ks']

# number of datasets being generated 
# 8 for training
# another one for evaluation
N=9 
path="train/"
if not os.path.exists(path):
    os.makedirs(path)
for n in xrange(N):
    for cl in xrange(clmax):
        xo=cl*spc
        #define label
        samples[xo:xo+spc]=cl
        phi = np.linspace(0, 2*np.pi, spc) + \
        np.random.randn(spc)*0.4*np.pi/clmax + \
        2*np.pi*cl/clmax
        r = np.linspace(0.1, 1, spc)
        I[xo:xo+spc,:]=np.transpose(np.array([r*np.cos(phi), r*np.sin(phi)]))
    with open(path+'dataset%02d.pic'%(n), 'wb') as pickleFile:
    #write label and feature vector
        theta_dim=1
        pickle.dump((clmax,theta_dim,theta_range,len(samples),samples,I,None), pickleFile, pickle.HIGHEST_PROTOCOL)

Visualization of the dataset



In [3]:

    
z=np.random.randint( 0,spc*clmax,1000)
for i in z:
    #ax.plot(dset.I[i,0],dset.I[i,1],marker[dset2.samples[i]])
    plt.plot(I[i,0],I[i,1],marker[samples[i]])
    plt.hold(True)

Training



In [4]:

    
from pforest.master import master
m=master()
m.reset()
m.train()









    



master>>init() dsetname: dataset
master>> create dview
master>> init engine
Found pforest
debug:master:__init__: ['train/dataset00.pic', 'train/dataset01.pic', 'train/dataset02.pic', 'train/dataset03.pic']
master>> init local variables
master>>reset()
master>>reset() H: 2.3219
master>>reset() Q: 10000
master::train() node20*-
master::train() node19R-
master::train() node18R-
master::train() node17R-
master::train() node16RQ
master::train() node16L-
master::train() node15R-
master::train() node14RQ
master::train() node14L-
master::train() node13R-
master::train() node12RQ
master::train() node12L-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9RG
master::train() node9LQ
master::train() node13LQ
master::train() node15LQ
master::train() node17L-
master::train() node16R-
master::train() node15RQ
master::train() node15L-
master::train() node14R-
master::train() node13R-
master::train() node12RQ
master::train() node12L-
master::train() node11RQ
master::train() node11L-
master::train() node10RG
master::train() node10LQ
master::train() node13LQ
master::train() node14LQ
master::train() node16LQ
master::train() node18L-
master::train() node17R-
master::train() node16R-
master::train() node15RQ
master::train() node15L-
master::train() node14RQ
master::train() node14L-
master::train() node13R-
master::train() node12RQ
master::train() node12LG
master::train() node13LQ
master::train() node16L-
master::train() node15RQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13L-
master::train() node12RG
master::train() node12LQ
master::train() node14LQ
master::train() node17L-
master::train() node16R-
master::train() node15R-
master::train() node14R-
master::train() node13RQ
master::train() node13LQ
master::train() node14L-
master::train() node13R-
master::train() node12RQ
master::train() node12L-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9RQ
master::train() node9LG
master::train() node11LQ
master::train() node13L-
master::train() node12RQ
master::train() node12LQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13LQ
master::train() node14L-
master::train() node13R-
master::train() node12R-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9RG
master::train() node9LQ
master::train() node12LQ
master::train() node13L-
master::train() node12RQ
master::train() node12LQ
master::train() node16L-
master::train() node15R-
master::train() node14R-
master::train() node13R-
master::train() node12R-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9R-
master::train() node8R-
master::train() node7RQ
master::train() node7LG
master::train() node8LQ
master::train() node9LQ
master::train() node11L-
master::train() node10RG
master::train() node10L-
master::train() node9R-
master::train() node8RQ
master::train() node8L-
master::train() node7R-
master::train() node6RQ
master::train() node6LG
master::train() node7LQ
master::train() node9LQ
master::train() node12L-
master::train() node11R-
master::train() node10R-
master::train() node9RQ
master::train() node9L-
master::train() node8R-
master::train() node7RQ
master::train() node7L-
master::train() node6RG
master::train() node6LQ
master::train() node8LQ
master::train() node10L-
master::train() node9RQ
master::train() node9LG
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9R-
master::train() node8R-
master::train() node7RG
master::train() node7LQ
master::train() node8LQ
master::train() node9LQ
master::train() node13L-
master::train() node12R-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10L-
master::train() node9R-
master::train() node8RQ
master::train() node8LG
master::train() node9LQ
master::train() node12L-
master::train() node11RQ
master::train() node11L-
master::train() node10RQ
master::train() node10LG
master::train() node14L-
master::train() node13R-
master::train() node12RQ
master::train() node12L-
master::train() node11R-
master::train() node10R-
master::train() node9R-
master::train() node8RQ
master::train() node8L-
master::train() node7RQ
master::train() node7LG
master::train() node9LQ
master::train() node10L-
master::train() node9RQ
master::train() node9LQ
master::train() node11LQ
master::train() node13L-
master::train() node12R-
master::train() node11R-
master::train() node10R-
master::train() node9RQ
master::train() node9LQ
master::train() node10L-
master::train() node9RQ
master::train() node9L-
master::train() node8R-
master::train() node7RG
master::train() node7LQ
master::train() node8LQ
master::train() node11LQ
master::train() node12L-
master::train() node11R-
master::train() node10RQ
master::train() node10L-
master::train() node9RQ
master::train() node9L-
master::train() node8RQ
master::train() node8LQ
master::train() node11LQ
master::train() node15L-
master::train() node14R-
master::train() node13R-
master::train() node12RQ
master::train() node12LQ
master::train() node13L-
master::train() node12RQ
master::train() node12L-
master::train() node11R-
master::train() node10R-
master::train() node9R-
master::train() node8RG
master::train() node8LQ
master::train() node9LQ
master::train() node10LQ
master::train() node11LQ
master::train() node14L-
master::train() node13RQ
master::train() node13L-
master::train() node12R-
master::train() node11RQ
master::train() node11LG
master::train() node12LQ
master::train() node19L-
master::train() node18R-
master::train() node17R-
master::train() node16R-
master::train() node15RQ
master::train() node15L-
master::train() node14R-
master::train() node13RQ
master::train() node13L-
master::train() node12RG
master::train() node12LQ
master::train() node14LQ
master::train() node16LQ
master::train() node17LQ
master::train() node18L-
master::train() node17RQ
master::train() node17LQ

Write and read the tree

You may need to save/load the tree to/from a pickle file



In [5]:

    
with open('out_tree.pic', 'wb') as pickleFile:
    pickle.dump(m.root, pickleFile, pickle.HIGHEST_PROTOCOL)
    
with open('out_tree.pic', 'rb') as pickleFile:
    root = pickle.load(pickleFile)

Check the file size



In [6]:

    
ls









    



 Volume in drive D is DATA
 Volume Serial Number is E617-8A3A

 Directory of D:\project\GitHub\book_pae\pae\nb

12/02/2015  01:44 PM    <DIR>          .
12/02/2015  01:44 PM    <DIR>          ..
12/02/2015  01:33 PM    <DIR>          .ipynb_checkpoints
12/02/2015  01:38 PM    <DIR>          dataset
12/02/2015  01:44 PM            48,207 out_tree.pic
12/02/2015  01:41 PM           149,858 parallel forest.ipynb
12/02/2015  01:33 PM    <DIR>          train
               2 File(s)        198,065 bytes
               5 Dir(s)  249,558,003,712 bytes free

The result decision tree

Termination code (Q:min bag size, G:no information gain, D:reaching maximum depth)



In [7]:

    
from pforest.dataset import dataset
from pforest.tree import tree

#init the test tree
t=tree()
t.settree(root)
t.show()









    



Found pforest
*- 20 H:2.322e+00,Q:010000 tau:-0.662201881409 theta:[ 1.]
L- 19 H:1.205e+00,Q:000691 tau:-0.84463506937 theta:[ 1.]
L- 18 H:1.818e-01,Q:000214 tau:-0.884833276272 theta:[ 1.]
LQ 17 H:1.488e-37,Q:000166 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 17 H:5.241e-01,Q:000048 (cl,P):(004,0.88) (000,0.10) (003,0.02)
R- 18 H:1.069e+00,Q:000477 tau:-0.353509843349 theta:[ 0.]
LQ 17 H:6.242e-01,Q:000099 (cl,P):(004,0.83) (003,0.16) (000,0.01)
R- 17 H:5.079e-01,Q:000378 tau:-0.818423569202 theta:[ 1.]
LQ 16 H:9.357e-01,Q:000033 (cl,P):(004,0.55) (000,0.45) (003,0.00)
R- 16 H:3.234e-01,Q:000345 tau:-0.711905360222 theta:[ 1.]
L- 15 H:2.210e-01,Q:000249 tau:-0.327274113894 theta:[ 0.]
LQ 14 H:2.986e-36,Q:000002 (cl,P):(004,1.00) (003,0.00) (002,0.00)
R- 14 H:1.827e-01,Q:000247 tau:0.547853291035 theta:[ 0.]
L- 13 H:6.288e-02,Q:000236 tau:-0.141213282943 theta:[ 0.]
LQ 12 H:3.876e-01,Q:000020 (cl,P):(000,0.90) (004,0.10) (003,0.00)
RG 12 H:1.147e-37,Q:000216 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 13 H:9.332e-01,Q:000011 (cl,P):(000,0.55) (004,0.45) (003,0.00)
RQ 15 H:3.897e-01,Q:000096 (cl,P):(000,0.92) (001,0.08) (004,0.00)
R- 19 H:2.317e+00,Q:009309 tau:0.509391784668 theta:[ 0.]
L- 18 H:2.311e+00,Q:008045 tau:0.635049104691 theta:[ 1.]
L- 17 H:2.306e+00,Q:007273 tau:0.226831242442 theta:[ 1.]
L- 16 H:2.308e+00,Q:005463 tau:-0.540481507778 theta:[ 0.]
L- 15 H:1.553e+00,Q:000829 tau:-0.798543930054 theta:[ 0.]
L- 14 H:2.241e-01,Q:000261 tau:-0.816352725029 theta:[ 0.]
L- 13 H:1.117e-01,Q:000246 tau:-0.958009839058 theta:[ 0.]
LQ 12 H:5.233e-01,Q:000019 (cl,P):(003,0.84) (002,0.16) (004,0.00)
R- 12 H:3.222e-02,Q:000227 tau:0.199973627925 theta:[ 1.]
LG 11 H:1.147e-37,Q:000216 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 11 H:2.504e-01,Q:000011 (cl,P):(003,0.91) (002,0.09) (004,0.00)
RQ 13 H:5.903e-01,Q:000015 (cl,P):(003,0.67) (004,0.33) (002,0.00)
R- 14 H:1.109e+00,Q:000568 tau:-0.604116022587 theta:[ 0.]
L- 13 H:6.215e-01,Q:000372 tau:-0.670504689217 theta:[ 0.]
L- 12 H:4.122e-01,Q:000269 tau:-0.549998342991 theta:[ 1.]
LQ 11 H:5.430e-01,Q:000018 (cl,P):(003,0.83) (004,0.17) (002,0.00)
R- 11 H:2.004e-01,Q:000251 tau:-0.775105893612 theta:[ 0.]
LQ 10 H:7.625e-01,Q:000025 (cl,P):(004,0.76) (003,0.24) (002,0.00)
R- 10 H:6.465e-02,Q:000226 tau:-0.419636309147 theta:[ 1.]
LQ 09 H:2.163e-01,Q:000015 (cl,P):(004,0.93) (003,0.07) (002,0.00)
R- 09 H:3.338e-02,Q:000211 tau:-0.7560313344 theta:[ 0.]
LQ 08 H:1.183e-01,Q:000035 (cl,P):(004,0.97) (003,0.03) (002,0.00)
RG 08 H:1.405e-37,Q:000176 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 12 H:7.052e-01,Q:000103 (cl,P):(004,0.81) (000,0.19) (003,0.00)
R- 13 H:7.336e-01,Q:000196 tau:-0.459757775068 theta:[ 1.]
LQ 12 H:2.260e-01,Q:000039 (cl,P):(004,0.92) (000,0.08) (003,0.00)
RQ 12 H:1.816e-01,Q:000157 (cl,P):(000,0.97) (004,0.03) (003,0.00)
R- 15 H:2.270e+00,Q:004634 tau:-0.135360553861 theta:[ 0.]
L- 14 H:1.602e+00,Q:001582 tau:-0.270326554775 theta:[ 0.]
L- 13 H:1.170e+00,Q:000835 tau:-0.444893628359 theta:[ 0.]
L- 12 H:2.937e-01,Q:000249 tau:-0.517760813236 theta:[ 1.]
LQ 11 H:9.333e-01,Q:000015 (cl,P):(000,0.53) (004,0.47) (003,0.00)
R- 11 H:1.162e-01,Q:000234 tau:-0.44991979003 theta:[ 0.]
L- 10 H:8.730e-02,Q:000228 tau:-0.45599681139 theta:[ 0.]
L- 09 H:5.777e-02,Q:000219 tau:-0.490905940533 theta:[ 0.]
LQ 08 H:1.583e-37,Q:000156 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 08 H:1.596e-01,Q:000063 (cl,P):(000,0.97) (001,0.03) (004,0.00)
RQ 09 H:3.061e-01,Q:000009 (cl,P):(000,0.89) (001,0.11) (004,0.00)
RQ 10 H:3.965e-36,Q:000006 (cl,P):(000,0.83) (001,0.17) (004,0.00)
R- 12 H:9.059e-01,Q:000586 tau:-0.530362486839 theta:[ 1.]
LQ 11 H:3.879e-37,Q:000063 (cl,P):(000,1.00) (004,0.00) (003,0.00)
R- 11 H:6.952e-01,Q:000523 tau:-0.356610566378 theta:[ 0.]
L- 10 H:7.230e-01,Q:000271 tau:0.113993316889 theta:[ 1.]
L- 09 H:5.125e-01,Q:000241 tau:-0.332224100828 theta:[ 1.]
LQ 08 H:8.196e-01,Q:000035 (cl,P):(000,0.74) (001,0.26) (004,0.00)
R- 08 H:6.986e-02,Q:000206 tau:-0.421923309565 theta:[ 0.]
LQ 07 H:2.990e-01,Q:000029 (cl,P):(001,0.93) (000,0.07) (004,0.00)
RG 07 H:1.397e-37,Q:000177 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 09 H:4.340e-01,Q:000030 (cl,P):(000,0.90) (001,0.10) (004,0.00)
R- 10 H:4.261e-01,Q:000252 tau:-0.311722964048 theta:[ 0.]
LQ 09 H:1.950e-01,Q:000150 (cl,P):(001,0.97) (000,0.02) (002,0.01)
RQ 09 H:5.651e-01,Q:000102 (cl,P):(001,0.85) (002,0.15) (004,0.00)
R- 13 H:1.089e+00,Q:000747 tau:0.120427936316 theta:[ 1.]
L- 12 H:9.287e-01,Q:000612 tau:-0.439001053572 theta:[ 1.]
LQ 11 H:5.733e-01,Q:000096 (cl,P):(001,0.89) (000,0.08) (002,0.03)
R- 11 H:5.213e-01,Q:000516 tau:-0.219281688333 theta:[ 0.]
L- 10 H:7.136e-01,Q:000176 tau:-0.352194398642 theta:[ 1.]
LQ 09 H:1.049e-36,Q:000023 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 09 H:4.213e-01,Q:000153 (cl,P):(002,0.91) (001,0.09) (004,0.00)
R- 10 H:2.749e-01,Q:000340 tau:-0.387966185808 theta:[ 1.]
LQ 09 H:5.510e-01,Q:000005 (cl,P):(001,0.60) (002,0.40) (004,0.00)
R- 09 H:2.223e-01,Q:000335 tau:0.080593585968 theta:[ 1.]
L- 08 H:1.628e-01,Q:000327 tau:-0.175473451614 theta:[ 0.]
LG 07 H:1.296e-37,Q:000191 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 07 H:3.147e-01,Q:000136 (cl,P):(002,0.94) (003,0.06) (004,0.00)
RQ 08 H:3.444e-01,Q:000008 (cl,P):(002,0.62) (001,0.38) (004,0.00)
RQ 12 H:1.826e-37,Q:000135 (cl,P):(001,1.00) (004,0.00) (003,0.00)
R- 14 H:2.262e+00,Q:003052 tau:-0.360864549875 theta:[ 1.]
L- 13 H:1.021e+00,Q:000652 tau:-0.478363335133 theta:[ 1.]
L- 12 H:2.646e-01,Q:000331 tau:-0.51578605175 theta:[ 1.]
L- 11 H:2.662e-02,Q:000290 tau:0.425249546766 theta:[ 0.]
LG 10 H:9.372e-38,Q:000265 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 10 H:1.657e-01,Q:000025 (cl,P):(001,0.96) (000,0.04) (004,0.00)
RQ 11 H:8.586e-01,Q:000041 (cl,P):(001,0.66) (002,0.34) (004,0.00)
R- 12 H:1.910e-01,Q:000321 tau:0.494571238756 theta:[ 0.]
L- 11 H:1.318e-01,Q:000318 tau:0.447821229696 theta:[ 0.]
L- 10 H:8.445e-02,Q:000316 tau:-0.451214075089 theta:[ 1.]
LQ 09 H:2.532e-01,Q:000044 (cl,P):(002,0.93) (001,0.07) (004,0.00)
R- 09 H:2.824e-02,Q:000272 tau:-0.370255202055 theta:[ 1.]
LG 08 H:9.967e-38,Q:000249 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 08 H:1.891e-01,Q:000023 (cl,P):(002,0.96) (003,0.04) (004,0.00)
RQ 10 H:5.924e-36,Q:000002 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 11 H:3.965e-36,Q:000003 (cl,P):(001,1.00) (004,0.00) (003,0.00)
R- 13 H:2.141e+00,Q:002400 tau:-0.0225597918034 theta:[ 1.]
L- 12 H:1.364e+00,Q:001302 tau:-0.186463400722 theta:[ 1.]
L- 11 H:8.146e-01,Q:000544 tau:0.35803848505 theta:[ 0.]
L- 10 H:5.272e-01,Q:000449 tau:-0.302767455578 theta:[ 1.]
LQ 09 H:8.248e-01,Q:000065 (cl,P):(002,0.69) (003,0.31) (004,0.00)
R- 09 H:1.499e-01,Q:000384 tau:-0.0624792911112 theta:[ 0.]
LQ 08 H:7.145e-01,Q:000031 (cl,P):(003,0.74) (002,0.26) (004,0.00)
R- 08 H:2.217e-02,Q:000353 tau:-0.299720615149 theta:[ 1.]
LQ 07 H:3.574e-36,Q:000005 (cl,P):(003,0.80) (002,0.20) (004,0.00)
RG 07 H:7.158e-38,Q:000348 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 10 H:4.204e-01,Q:000095 (cl,P):(002,0.89) (003,0.11) (004,0.00)
R- 11 H:1.100e+00,Q:000758 tau:-0.02706576325 theta:[ 0.]
L- 10 H:7.071e-02,Q:000200 tau:-0.0321599170566 theta:[ 0.]
LG 09 H:1.276e-37,Q:000194 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 09 H:3.333e-01,Q:000006 (cl,P):(003,0.67) (004,0.33) (002,0.00)
R- 10 H:1.023e+00,Q:000558 tau:0.321507126093 theta:[ 0.]
L- 09 H:5.275e-01,Q:000414 tau:-0.141094684601 theta:[ 1.]
LQ 08 H:9.133e-01,Q:000064 (cl,P):(004,0.61) (003,0.39) (002,0.00)
R- 08 H:2.626e-01,Q:000350 tau:-0.0644503831863 theta:[ 1.]
L- 07 H:7.712e-02,Q:000289 tau:-0.122672520578 theta:[ 1.]
LQ 06 H:2.270e-01,Q:000051 (cl,P):(004,0.92) (003,0.08) (002,0.00)
RG 06 H:1.042e-37,Q:000238 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 07 H:6.622e-01,Q:000061 (cl,P):(004,0.82) (000,0.18) (003,0.00)
RQ 09 H:9.143e-02,Q:000144 (cl,P):(003,0.99) (004,0.01) (002,0.01)
R- 12 H:2.188e+00,Q:001098 tau:0.173992037773 theta:[ 0.]
L- 11 H:1.454e+00,Q:000693 tau:0.0890478491783 theta:[ 0.]
L- 10 H:1.004e+00,Q:000403 tau:0.102935224771 theta:[ 1.]
LQ 09 H:6.696e-01,Q:000119 (cl,P):(002,0.82) (001,0.18) (004,0.00)
R- 09 H:3.626e-01,Q:000284 tau:0.0483255051076 theta:[ 0.]
L- 08 H:1.635e-01,Q:000262 tau:0.116220265627 theta:[ 1.]
LQ 07 H:5.417e-01,Q:000026 (cl,P):(001,0.85) (002,0.15) (004,0.00)
R- 07 H:6.258e-02,Q:000236 tau:0.214942723513 theta:[ 1.]
LG 06 H:1.107e-37,Q:000224 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 06 H:6.258e-01,Q:000012 (cl,P):(001,0.83) (000,0.17) (004,0.00)
RQ 08 H:9.755e-01,Q:000022 (cl,P):(001,0.50) (000,0.50) (004,0.00)
RG 10 H:8.572e-38,Q:000290 (cl,P):(000,1.00) (004,0.00) (003,0.00)
R- 11 H:1.296e+00,Q:000405 tau:0.378229409456 theta:[ 0.]
L- 10 H:6.063e-01,Q:000276 tau:0.220858022571 theta:[ 0.]
LQ 09 H:6.564e-37,Q:000037 (cl,P):(000,1.00) (004,0.00) (003,0.00)
R- 09 H:9.220e-02,Q:000239 tau:0.2373290658 theta:[ 0.]
LQ 08 H:3.444e-01,Q:000008 (cl,P):(004,0.75) (000,0.25) (003,0.00)
R- 08 H:3.122e-02,Q:000231 tau:0.367419779301 theta:[ 0.]
LG 07 H:1.102e-37,Q:000225 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 07 H:3.333e-01,Q:000006 (cl,P):(004,0.83) (003,0.17) (002,0.00)
RQ 10 H:1.012e-01,Q:000129 (cl,P):(003,0.98) (004,0.02) (002,0.00)
R- 16 H:1.847e+00,Q:001810 tau:0.396842598915 theta:[ 1.]
L- 15 H:1.665e+00,Q:000948 tau:0.16501249373 theta:[ 0.]
L- 14 H:1.386e+00,Q:000690 tau:-0.40640181303 theta:[ 0.]
L- 13 H:1.501e+00,Q:000206 tau:-0.658288061619 theta:[ 0.]
LQ 12 H:7.069e-01,Q:000087 (cl,P):(003,0.85) (002,0.10) (004,0.05)
RQ 12 H:5.934e-01,Q:000119 (cl,P):(004,0.85) (000,0.15) (003,0.00)
R- 13 H:3.919e-01,Q:000484 tau:0.261732280254 theta:[ 1.]
LQ 12 H:7.796e-01,Q:000093 (cl,P):(000,0.76) (001,0.24) (004,0.00)
R- 12 H:1.626e-01,Q:000391 tau:0.132076844573 theta:[ 0.]
L- 11 H:7.879e-02,Q:000381 tau:0.35280919075 theta:[ 1.]
L- 10 H:2.575e-02,Q:000296 tau:0.27855014801 theta:[ 1.]
LQ 09 H:1.161e-01,Q:000039 (cl,P):(000,0.97) (001,0.03) (004,0.00)
RG 09 H:9.660e-38,Q:000257 (cl,P):(000,1.00) (004,0.00) (003,0.00)
RQ 10 H:1.743e-01,Q:000085 (cl,P):(000,0.96) (004,0.04) (003,0.00)
RQ 11 H:7.510e-01,Q:000010 (cl,P):(004,0.50) (000,0.50) (003,0.00)
R- 14 H:1.034e+00,Q:000258 tau:0.387344270945 theta:[ 0.]
LQ 13 H:5.070e-01,Q:000159 (cl,P):(004,0.89) (003,0.10) (000,0.01)
RQ 13 H:1.221e-01,Q:000099 (cl,P):(003,0.98) (004,0.01) (002,0.01)
R- 15 H:1.428e+00,Q:000862 tau:0.194467380643 theta:[ 0.]
L- 14 H:1.273e+00,Q:000664 tau:-0.440580666065 theta:[ 0.]
L- 13 H:1.206e+00,Q:000219 tau:-0.68847745657 theta:[ 0.]
LQ 12 H:9.799e-01,Q:000069 (cl,P):(002,0.54) (003,0.46) (004,0.00)
RQ 12 H:7.572e-01,Q:000150 (cl,P):(003,0.77) (004,0.23) (002,0.00)
R- 13 H:6.631e-01,Q:000445 tau:0.604109048843 theta:[ 1.]
L- 12 H:3.264e-01,Q:000404 tau:0.441635280848 theta:[ 1.]
LQ 11 H:6.826e-01,Q:000070 (cl,P):(004,0.81) (000,0.19) (003,0.00)
R- 11 H:1.438e-01,Q:000334 tau:0.578687906265 theta:[ 1.]
L- 10 H:2.387e-02,Q:000315 tau:0.562226116657 theta:[ 1.]
LG 09 H:8.076e-38,Q:000308 (cl,P):(004,1.00) (003,0.00) (002,0.00)
RQ 09 H:3.936e-01,Q:000007 (cl,P):(004,0.86) (003,0.14) (002,0.00)
RQ 10 H:6.971e-01,Q:000019 (cl,P):(004,0.68) (003,0.32) (002,0.00)
RQ 12 H:2.372e-01,Q:000041 (cl,P):(003,0.95) (004,0.05) (002,0.00)
R- 14 H:9.612e-01,Q:000198 tau:0.42234519124 theta:[ 0.]
LQ 13 H:4.753e-01,Q:000151 (cl,P):(003,0.91) (004,0.07) (002,0.02)
RQ 13 H:7.036e-01,Q:000047 (cl,P):(002,0.79) (003,0.21) (004,0.00)
R- 17 H:1.331e+00,Q:000772 tau:0.748506069183 theta:[ 1.]
L- 16 H:9.499e-01,Q:000350 tau:0.279334098101 theta:[ 0.]
L- 15 H:6.942e-01,Q:000288 tau:-0.449340671301 theta:[ 0.]
LQ 14 H:4.343e-01,Q:000053 (cl,P):(002,0.89) (003,0.11) (004,0.00)
R- 14 H:2.055e-01,Q:000235 tau:0.129781678319 theta:[ 0.]
L- 13 H:3.457e-02,Q:000207 tau:-0.40372261405 theta:[ 0.]
LQ 12 H:3.936e-01,Q:000007 (cl,P):(003,0.86) (002,0.14) (004,0.00)
RG 12 H:1.238e-37,Q:000200 (cl,P):(003,1.00) (004,0.00) (002,0.00)
RQ 13 H:6.653e-01,Q:000028 (cl,P):(003,0.75) (002,0.25) (004,0.00)
RQ 15 H:1.740e-01,Q:000062 (cl,P):(002,0.97) (001,0.03) (004,0.00)
R- 16 H:8.314e-01,Q:000422 tau:0.329549670219 theta:[ 0.]
L- 15 H:5.681e-01,Q:000367 tau:0.885086894035 theta:[ 1.]
L- 14 H:1.944e-01,Q:000285 tau:0.778123378754 theta:[ 1.]
LQ 13 H:4.892e-01,Q:000059 (cl,P):(002,0.86) (003,0.14) (004,0.00)
R- 13 H:3.278e-02,Q:000226 tau:0.264445483685 theta:[ 0.]
LG 12 H:1.152e-37,Q:000215 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 12 H:3.281e-01,Q:000011 (cl,P):(002,0.91) (001,0.09) (004,0.00)
RQ 14 H:9.611e-01,Q:000082 (cl,P):(002,0.61) (001,0.39) (004,0.00)
RQ 15 H:3.902e-01,Q:000055 (cl,P):(001,0.91) (002,0.09) (004,0.00)
R- 18 H:1.641e+00,Q:001264 tau:0.705081224442 theta:[ 0.]
L- 17 H:1.472e+00,Q:000719 tau:-0.426131308079 theta:[ 1.]
LQ 16 H:9.505e-01,Q:000102 (cl,P):(000,0.59) (001,0.41) (004,0.00)
R- 16 H:1.124e+00,Q:000617 tau:0.53605723381 theta:[ 1.]
L- 15 H:9.595e-01,Q:000499 tau:-0.262299269438 theta:[ 1.]
LQ 14 H:4.960e-01,Q:000089 (cl,P):(001,0.88) (002,0.12) (004,0.00)
R- 14 H:5.169e-01,Q:000410 tau:0.550821661949 theta:[ 0.]
LQ 13 H:8.982e-01,Q:000078 (cl,P):(002,0.67) (003,0.33) (004,0.00)
R- 13 H:2.281e-01,Q:000332 tau:0.665768742561 theta:[ 0.]
L- 12 H:8.093e-02,Q:000279 tau:0.438179343939 theta:[ 1.]
L- 11 H:2.961e-02,Q:000251 tau:0.561247289181 theta:[ 0.]
LQ 10 H:2.777e-01,Q:000013 (cl,P):(002,0.92) (003,0.08) (004,0.00)
RG 10 H:1.042e-37,Q:000238 (cl,P):(002,1.00) (004,0.00) (003,0.00)
RQ 11 H:2.886e-01,Q:000028 (cl,P):(002,0.93) (001,0.07) (004,0.00)
RQ 12 H:6.089e-01,Q:000053 (cl,P):(002,0.83) (001,0.17) (004,0.00)
RQ 15 H:5.696e-01,Q:000118 (cl,P):(001,0.86) (002,0.14) (004,0.00)
R- 17 H:1.081e+00,Q:000545 tau:0.889273703098 theta:[ 0.]
L- 16 H:9.441e-01,Q:000436 tau:-0.300151973963 theta:[ 1.]
LQ 15 H:2.940e-01,Q:000096 (cl,P):(000,0.95) (001,0.05) (004,0.00)
R- 15 H:4.405e-01,Q:000340 tau:0.856172978878 theta:[ 0.]
L- 14 H:3.022e-01,Q:000310 tau:0.736774921417 theta:[ 0.]
LQ 13 H:7.196e-01,Q:000054 (cl,P):(001,0.80) (002,0.19) (000,0.02)
R- 13 H:1.149e-01,Q:000256 tau:0.592301309109 theta:[ 1.]
L- 12 H:8.657e-02,Q:000254 tau:0.842199385166 theta:[ 0.]
L- 11 H:6.235e-02,Q:000230 tau:0.509778022766 theta:[ 1.]
L- 10 H:3.272e-02,Q:000217 tau:0.765484571457 theta:[ 0.]
LQ 09 H:1.316e-01,Q:000041 (cl,P):(001,0.98) (002,0.02) (004,0.00)
RG 09 H:1.405e-37,Q:000176 (cl,P):(001,1.00) (004,0.00) (003,0.00)
RQ 10 H:2.119e-01,Q:000013 (cl,P):(001,0.92) (000,0.08) (004,0.00)
RQ 11 H:1.148e-01,Q:000024 (cl,P):(001,0.96) (000,0.04) (004,0.00)
RQ 12 H:5.924e-36,Q:000002 (cl,P):(001,0.50) (000,0.50) (004,0.00)
RQ 14 H:9.021e-01,Q:000030 (cl,P):(001,0.67) (000,0.33) (004,0.00)
RQ 16 H:1.863e-01,Q:000109 (cl,P):(000,0.96) (001,0.04) (004,0.00)

Recall rate

Loading a new dataset, the last on, for computing a recall rate



In [9]:

    
#load the last dataset that never use for training
dset=dataset(8)
correct=0;
for x in xrange(dset.size):
    L=t.getL(np.array([x]),dset)
    if dset.getL(x) == L:
        correct=correct+1
    dset.setL(x,L)
print("recall rate: {}%".format(correct/float(dset.size)*100))









    



recall rate: 91.76%

Labelling

The computer use the decision tree to classify the unknown feature vector u



In [10]:

    
#setup the new test-set
#load dataset     
dset=dataset(8)
d=0.05
y, x = np.mgrid[slice(-1, 1+d, d), slice(-1, 1+d, d)]

#start labeling
L=np.zeros(x.shape,dtype=int)
for r in xrange(x.shape[0]):
    for c in xrange(x.shape[1]):
        u=( x[r,c],y[r,c] )
        Prob=t.classify(u)
        L[r,c]=np.argmax(Prob)

2D space partitioning by the decision tree

Displaying the labelled result



In [11]:

    
%matplotlib inline
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis([-1,1,-1,1])
ax.pcolor(x,y,L)
ax.hold(True)

Overlay the dataset



In [14]:

    
z=np.random.randint(0,dset.size,1000)
for i in z:
    ax.plot(dset.I[i,0],dset.I[i,1],marker[dset.samples[i]])
fig









    Out[14]:



In [16]:

    
t.classify([1,1])









    Out[16]:





array([ 0.96330275,  0.03669725,  0.        ,  0.        ,  0.        ])



In [ ]: