L-BFGS Method based Logistic Regression

In this tutorial, we'll explore training and evaluation of L-BFGS method based Logitistic Regression Classifiers.

To start, we import the standard BIDMach class definitions.



In [1]:

    
import BIDMat.{CMat,CSMat,DMat,Dict,IDict,FMat,GMat,GIMat,GSMat,HMat,IMat,Mat,SMat,SBMat,SDMat}
import BIDMat.MatFunctions._
import BIDMat.SciFunctions._
import BIDMat.Solvers._
import BIDMat.Plotting._
import BIDMach.Learner
import BIDMach.models.{FM,GLM,KMeans,LDA,LDAgibbs,NMF,SFA}
import BIDMach.datasources.{MatDS,FilesDS,SFilesDS}
import BIDMach.mixins.{CosineSim,Perplexity,Top,L1Regularizer,L2Regularizer}
import BIDMach.updaters.{ADAGrad,Batch,BatchNorm,IncMult,IncNorm,Telescoping}
import BIDMach.causal.{IPTW}

Mat.checkMKL
Mat.checkCUDA
if (Mat.hasCUDA > 0) GPUmem









    



Cant find native HDF5 library
Couldnt load JCuda






    Out[1]:





()

Now we load some training and test data, and some category labels. The data come from a news collection from Reuters, and is a "classic" test set for classification. Each article belongs to one or more of 103 categories. The articles are represented as Bag-of-Words (BoW) column vectors. For a data matrix A, element A(i,j) holds the count of word i in document j.

The category matrices have 103 rows, and a category matrix C has a one in position C(i,j) if document j is tagged with category i, or zero otherwise.

To reduce the computing time and memory footprint, the training data have been sampled. The full collection has about 700k documents. Our training set has 60k.

Since the document matrices contain counts of words, we use a min function to limit the count to "1", i.e. because we need binary features for naive Bayes.



In [2]:

    
val dict = "/Users/Anna/workspace/BIDMach/data/"
val a = loadFMat(dict+"arabic.fmat.lz4")
val c = loadFMat(dict+"arabic_cats.fmat.lz4")
//val aa = loadFMat(dict+"a.txt")
//val c = loadFMat(dict+"alabel.txt")
//val a = aa + 0.5 
val atrain =a(?,(100->a.ncols))
val atest =a(?,(0->100))
val ctrain =c(?,(100->a.ncols))
val ctest =c(?,(0->100))
max(atrain, 0.001, atrain)                       // the first "traindata" argument is the input, the other is output
max(atest, 0.001, atest)

//val dict = "/Users/Anna/workspace/BIDMach/data/class2.txt"
//val rpath = "/Users/Anna/workspace/BIDMach/data/cat2.txt"
//var aa=loadFMat(dict)
//var c =loadFMat(rpath)
//var b=aa.t
//var d = c
//var index = 1;

//for(index <- 1 to 171){
//var aa = loadFMat(dict+"fc8/"+index+".txt")
//var c = loadFMat(dict+"fc8/c"+index+".txt")
//var b = loadFMat(dict+"testfc8.txt")
//var d = loadFMat(dict+"wltestcatall.txt")
//val dict = "/Users/Anna/workspace/BIDMach_1.0.0-full-linux-x86_64/data/uci/"
//val aa = loadFMat(dict+"arabic.fmat.lz4")
//val c = loadFMat(dict+"arabic_cats.fmat.lz4")
//var a = aa.t   
//var atrain = a //a(?,(100->a.ncols))
//var atest =  b //a(?,(0->100))
//var ctrain = c //c(?,(100->a.ncols))
//var ctest = d //c(?,(0->100))
//max(atrain, 0.001, atrain)                       // the first "traindata" argument is the input, the other is output
//max(atest, 0.001, atest)

val cx=zeros(2,ctest.ncols)
val (mm,mopts,nn,nopts)=GLM.LBFGSlearner(atrain,ctrain,atest,cx)
mopts.autoReset=false
mopts.useGPU=false
mopts.lrate=0.1
mopts.batchSize=2
mopts.dim=256
mopts.startBlock=0
mopts.npasses=10
mopts.updateAll=false
mm.train;
nn.predict;
atrain.ncols
//saveFMat(rpath+index+".txt",cx);
//}
//cx.ncols









    



corpus perplexity=11.924807
pass= 0
 1.00%, ll=-1.14872, gf=0.113, secs=0.7, GB=0.00, MB/s= 0.57
 2.00%, ll=-512.49626, gf=0.155, secs=1.1, GB=0.00, MB/s= 0.78
 3.00%, ll=-580.19754, gf=0.177, secs=1.4, GB=0.00, MB/s= 0.88
 4.00%, ll=-335.49322, gf=0.195, secs=1.7, GB=0.00, MB/s= 0.97
 5.00%, ll=-131.44034, gf=0.209, secs=2.0, GB=0.00, MB/s= 1.04
 6.00%, ll=-24.27121, gf=0.214, secs=2.3, GB=0.00, MB/s= 1.06
 7.00%, ll=-2.30999, gf=0.223, secs=2.6, GB=0.00, MB/s= 1.11
 8.00%, ll=-2.46396, gf=0.231, secs=2.9, GB=0.00, MB/s= 1.15
 9.00%, ll=-1.33129, gf=0.236, secs=3.1, GB=0.00, MB/s= 1.18
10.00%, ll=-0.85965, gf=0.243, secs=3.4, GB=0.00, MB/s= 1.21
11.00%, ll=-0.00136, gf=0.249, secs=3.6, GB=0.00, MB/s= 1.24
12.00%, ll=0.00000, gf=0.254, secs=3.9, GB=0.00, MB/s= 1.27
13.00%, ll=-0.02861, gf=0.258, secs=4.2, GB=0.01, MB/s= 1.28
14.00%, ll=-19.90062, gf=0.262, secs=4.4, GB=0.01, MB/s= 1.31
15.00%, ll=-6.02461, gf=0.267, secs=4.6, GB=0.01, MB/s= 1.33
16.00%, ll=-171.73542, gf=0.270, secs=4.9, GB=0.01, MB/s= 1.35
17.00%, ll=-157.84812, gf=0.273, secs=5.1, GB=0.01, MB/s= 1.36
18.00%, ll=-86.64165, gf=0.275, secs=5.4, GB=0.01, MB/s= 1.37
19.00%, ll=-30.54422, gf=0.278, secs=5.6, GB=0.01, MB/s= 1.39
20.00%, ll=-18.60019, gf=0.280, secs=5.9, GB=0.01, MB/s= 1.40
21.00%, ll=-9.18192, gf=0.282, secs=6.1, GB=0.01, MB/s= 1.41
22.00%, ll=-3.51230, gf=0.283, secs=6.4, GB=0.01, MB/s= 1.41
23.00%, ll=-3.23538, gf=0.285, secs=6.6, GB=0.01, MB/s= 1.42
24.00%, ll=-51.45923, gf=0.287, secs=6.9, GB=0.01, MB/s= 1.43
25.00%, ll=-51.45929, gf=0.288, secs=7.1, GB=0.01, MB/s= 1.44
26.00%, ll=-30.95603, gf=0.290, secs=7.4, GB=0.01, MB/s= 1.44
27.00%, ll=-32.31539, gf=0.291, secs=7.6, GB=0.01, MB/s= 1.45
28.00%, ll=-17.34102, gf=0.293, secs=7.9, GB=0.01, MB/s= 1.46
29.00%, ll=-12.40876, gf=0.294, secs=8.1, GB=0.01, MB/s= 1.47
30.00%, ll=-7.78442, gf=0.295, secs=8.4, GB=0.01, MB/s= 1.47
31.00%, ll=-6.46077, gf=0.296, secs=8.6, GB=0.01, MB/s= 1.48
32.00%, ll=-5.60764, gf=0.297, secs=8.9, GB=0.01, MB/s= 1.48
33.00%, ll=-2.76664, gf=0.298, secs=9.1, GB=0.01, MB/s= 1.49
34.00%, ll=-3.78964, gf=0.300, secs=9.3, GB=0.01, MB/s= 1.49
35.00%, ll=-3.30305, gf=0.300, secs=9.6, GB=0.01, MB/s= 1.50
36.00%, ll=-55.91283, gf=0.301, secs=9.9, GB=0.01, MB/s= 1.50
37.00%, ll=-500.14483, gf=0.302, secs=10.1, GB=0.02, MB/s= 1.50
38.00%, ll=-474.48741, gf=0.303, secs=10.3, GB=0.02, MB/s= 1.51
39.00%, ll=-399.60940, gf=0.303, secs=10.6, GB=0.02, MB/s= 1.51
40.00%, ll=-103.77494, gf=0.304, secs=10.8, GB=0.02, MB/s= 1.51
41.00%, ll=-16.08423, gf=0.305, secs=11.1, GB=0.02, MB/s= 1.52
42.00%, ll=-23.39155, gf=0.305, secs=11.4, GB=0.02, MB/s= 1.52
43.00%, ll=-6.26146, gf=0.305, secs=11.6, GB=0.02, MB/s= 1.52
44.00%, ll=-8.45534, gf=0.306, secs=11.8, GB=0.02, MB/s= 1.52
45.00%, ll=-10.59097, gf=0.307, secs=12.1, GB=0.02, MB/s= 1.53
46.00%, ll=-0.96926, gf=0.307, secs=12.3, GB=0.02, MB/s= 1.53
47.00%, ll=-1.35261, gf=0.308, secs=12.6, GB=0.02, MB/s= 1.53
48.00%, ll=-1.42654, gf=0.308, secs=12.9, GB=0.02, MB/s= 1.53
49.00%, ll=-1.97775, gf=0.308, secs=13.1, GB=0.02, MB/s= 1.53
50.00%, ll=-2.63714, gf=0.309, secs=13.3, GB=0.02, MB/s= 1.54
51.00%, ll=-35.90422, gf=0.309, secs=13.6, GB=0.02, MB/s= 1.54
52.00%, ll=-95.15880, gf=0.309, secs=13.8, GB=0.02, MB/s= 1.54
53.00%, ll=-75.40914, gf=0.310, secs=14.1, GB=0.02, MB/s= 1.54
54.00%, ll=-51.93669, gf=0.311, secs=14.3, GB=0.02, MB/s= 1.55
55.00%, ll=-27.19405, gf=0.311, secs=14.6, GB=0.02, MB/s= 1.55
56.00%, ll=-35.40561, gf=0.311, secs=14.8, GB=0.02, MB/s= 1.55
57.00%, ll=-38.89381, gf=0.311, secs=15.1, GB=0.02, MB/s= 1.55
58.00%, ll=-37.71455, gf=0.312, secs=15.3, GB=0.02, MB/s= 1.55
59.00%, ll=-42.06237, gf=0.312, secs=15.6, GB=0.02, MB/s= 1.56
60.00%, ll=-27.96083, gf=0.312, secs=15.8, GB=0.02, MB/s= 1.56
61.00%, ll=-24.96176, gf=0.313, secs=16.1, GB=0.03, MB/s= 1.56
62.00%, ll=-16.41766, gf=0.313, secs=16.3, GB=0.03, MB/s= 1.56
63.00%, ll=-10.24271, gf=0.313, secs=16.6, GB=0.03, MB/s= 1.56
64.00%, ll=-7.86382, gf=0.313, secs=16.9, GB=0.03, MB/s= 1.56
65.00%, ll=-3.97772, gf=0.313, secs=17.1, GB=0.03, MB/s= 1.56
66.00%, ll=-2.99713, gf=0.314, secs=17.3, GB=0.03, MB/s= 1.56
67.00%, ll=-1014.74832, gf=0.314, secs=17.6, GB=0.03, MB/s= 1.56
68.00%, ll=-907.24366, gf=0.314, secs=17.8, GB=0.03, MB/s= 1.57
69.00%, ll=-880.12370, gf=0.314, secs=18.1, GB=0.03, MB/s= 1.57
70.00%, ll=-527.40457, gf=0.315, secs=18.3, GB=0.03, MB/s= 1.57
71.00%, ll=-371.25709, gf=0.315, secs=18.6, GB=0.03, MB/s= 1.57
72.00%, ll=-442.79871, gf=0.315, secs=18.8, GB=0.03, MB/s= 1.57
73.00%, ll=-361.26251, gf=0.315, secs=19.1, GB=0.03, MB/s= 1.57
74.00%, ll=-433.71932, gf=0.315, secs=19.3, GB=0.03, MB/s= 1.57
75.00%, ll=-337.64090, gf=0.316, secs=19.6, GB=0.03, MB/s= 1.57
76.00%, ll=-251.29260, gf=0.316, secs=19.8, GB=0.03, MB/s= 1.57
77.00%, ll=-191.20815, gf=0.316, secs=20.1, GB=0.03, MB/s= 1.58
78.00%, ll=-162.31523, gf=0.316, secs=20.3, GB=0.03, MB/s= 1.58
79.00%, ll=-144.88985, gf=0.316, secs=20.6, GB=0.03, MB/s= 1.58
80.00%, ll=-29.71847, gf=0.317, secs=20.8, GB=0.03, MB/s= 1.58
81.00%, ll=-54.80950, gf=0.316, secs=21.1, GB=0.03, MB/s= 1.58
82.00%, ll=-39.46392, gf=0.317, secs=21.3, GB=0.03, MB/s= 1.58
83.00%, ll=-37.41061, gf=0.317, secs=21.6, GB=0.03, MB/s= 1.58
84.00%, ll=-34.28977, gf=0.317, secs=21.8, GB=0.03, MB/s= 1.58
85.00%, ll=-89.00587, gf=0.317, secs=22.1, GB=0.03, MB/s= 1.58
86.00%, ll=-32.33935, gf=0.317, secs=22.3, GB=0.04, MB/s= 1.58
87.00%, ll=-47.88466, gf=0.317, secs=22.6, GB=0.04, MB/s= 1.58
88.00%, ll=-52.98359, gf=0.318, secs=22.8, GB=0.04, MB/s= 1.58
89.00%, ll=-37.58659, gf=0.317, secs=23.2, GB=0.04, MB/s= 1.58
90.00%, ll=-34.50275, gf=0.316, secs=23.5, GB=0.04, MB/s= 1.57
91.00%, ll=-32.84385, gf=0.316, secs=23.7, GB=0.04, MB/s= 1.57
92.00%, ll=-50.65752, gf=0.316, secs=24.0, GB=0.04, MB/s= 1.57
93.00%, ll=-50.24549, gf=0.316, secs=24.2, GB=0.04, MB/s= 1.58
94.00%, ll=-37.29997, gf=0.316, secs=24.5, GB=0.04, MB/s= 1.58
95.00%, ll=-41.65459, gf=0.317, secs=24.7, GB=0.04, MB/s= 1.58
96.00%, ll=-54.50206, gf=0.317, secs=25.0, GB=0.04, MB/s= 1.58
97.00%, ll=-37.09835, gf=0.317, secs=25.2, GB=0.04, MB/s= 1.58
98.00%, ll=-25.00205, gf=0.317, secs=25.5, GB=0.04, MB/s= 1.58
99.00%, ll=-21.53054, gf=0.317, secs=25.8, GB=0.04, MB/s= 1.58
100.00%, ll=-16.38595, gf=0.317, secs=26.0, GB=0.04, MB/s= 1.58
pass= 1
 1.00%, ll=-14.66289, gf=0.317, secs=26.3, GB=0.04, MB/s= 1.58
 2.00%, ll=-12.79378, gf=0.317, secs=26.5, GB=0.04, MB/s= 1.58
 3.00%, ll=-1221.01571, gf=0.317, secs=26.8, GB=0.04, MB/s= 1.58
 4.00%, ll=-16031.73688, gf=0.317, secs=27.0, GB=0.04, MB/s= 1.58
 5.00%, ll=-15270.42154, gf=0.317, secs=27.2, GB=0.04, MB/s= 1.58
 6.00%, ll=-10378.64567, gf=0.318, secs=27.5, GB=0.04, MB/s= 1.58
 7.00%, ll=-7765.38070, gf=0.318, secs=27.7, GB=0.04, MB/s= 1.58
 8.00%, ll=-6885.27104, gf=0.318, secs=28.0, GB=0.04, MB/s= 1.58
 9.00%, ll=-8827.93116, gf=0.318, secs=28.3, GB=0.04, MB/s= 1.58
10.00%, ll=-9232.91019, gf=0.318, secs=28.5, GB=0.05, MB/s= 1.58
11.00%, ll=-6680.49921, gf=0.318, secs=28.7, GB=0.05, MB/s= 1.59
12.00%, ll=-7370.33698, gf=0.318, secs=29.0, GB=0.05, MB/s= 1.59
13.00%, ll=-8161.20399, gf=0.318, secs=29.2, GB=0.05, MB/s= 1.59
14.00%, ll=-5659.77812, gf=0.319, secs=29.5, GB=0.05, MB/s= 1.59
15.00%, ll=-5578.11923, gf=0.319, secs=29.7, GB=0.05, MB/s= 1.59
16.00%, ll=-4334.79393, gf=0.319, secs=30.0, GB=0.05, MB/s= 1.59
17.00%, ll=-4459.85777, gf=0.319, secs=30.2, GB=0.05, MB/s= 1.59
18.00%, ll=-3396.71886, gf=0.319, secs=30.5, GB=0.05, MB/s= 1.59
19.00%, ll=-1595.86255, gf=0.319, secs=30.7, GB=0.05, MB/s= 1.59
20.00%, ll=-1615.81897, gf=0.319, secs=31.0, GB=0.05, MB/s= 1.59
21.00%, ll=-1570.83185, gf=0.319, secs=31.3, GB=0.05, MB/s= 1.59
22.00%, ll=-1727.53970, gf=0.319, secs=31.5, GB=0.05, MB/s= 1.59
23.00%, ll=-1608.53660, gf=0.319, secs=31.8, GB=0.05, MB/s= 1.59
24.00%, ll=-1044.10456, gf=0.319, secs=32.0, GB=0.05, MB/s= 1.59
25.00%, ll=-1068.65347, gf=0.319, secs=32.3, GB=0.05, MB/s= 1.59
26.00%, ll=-887.46727, gf=0.319, secs=32.5, GB=0.05, MB/s= 1.59
27.00%, ll=-798.65758, gf=0.319, secs=32.8, GB=0.05, MB/s= 1.59
28.00%, ll=-706.52822, gf=0.319, secs=33.0, GB=0.05, MB/s= 1.59
29.00%, ll=-827.21372, gf=0.319, secs=33.3, GB=0.05, MB/s= 1.59
30.00%, ll=-668.36056, gf=0.319, secs=33.5, GB=0.05, MB/s= 1.59
31.00%, ll=-602.52203, gf=0.319, secs=33.8, GB=0.05, MB/s= 1.59
32.00%, ll=-631.25018, gf=0.319, secs=34.1, GB=0.05, MB/s= 1.59
33.00%, ll=-664.10085, gf=0.319, secs=34.3, GB=0.05, MB/s= 1.59
34.00%, ll=-641.71525, gf=0.320, secs=34.5, GB=0.06, MB/s= 1.59
35.00%, ll=-398.90220, gf=0.320, secs=34.8, GB=0.06, MB/s= 1.59
36.00%, ll=-417.24358, gf=0.320, secs=35.1, GB=0.06, MB/s= 1.59
37.00%, ll=-319.99369, gf=0.320, secs=35.3, GB=0.06, MB/s= 1.59
38.00%, ll=-327.71260, gf=0.320, secs=35.6, GB=0.06, MB/s= 1.59
39.00%, ll=-306.06002, gf=0.320, secs=35.8, GB=0.06, MB/s= 1.59
40.00%, ll=-278.22633, gf=0.319, secs=36.1, GB=0.06, MB/s= 1.59
41.00%, ll=-216.62836, gf=0.319, secs=36.4, GB=0.06, MB/s= 1.59
42.00%, ll=-182.43108, gf=0.319, secs=36.7, GB=0.06, MB/s= 1.59
43.00%, ll=-205.94191, gf=0.319, secs=37.0, GB=0.06, MB/s= 1.59
44.00%, ll=-226.99477, gf=0.319, secs=37.2, GB=0.06, MB/s= 1.59
45.00%, ll=-170.14721, gf=0.319, secs=37.5, GB=0.06, MB/s= 1.59
46.00%, ll=-142.84854, gf=0.319, secs=37.7, GB=0.06, MB/s= 1.59
47.00%, ll=-114.57476, gf=0.319, secs=38.0, GB=0.06, MB/s= 1.59
48.00%, ll=-94.87685, gf=0.319, secs=38.2, GB=0.06, MB/s= 1.59
49.00%, ll=-117.89929, gf=0.319, secs=38.5, GB=0.06, MB/s= 1.59
50.00%, ll=-416.89685, gf=0.319, secs=38.7, GB=0.06, MB/s= 1.59
51.00%, ll=-756.18060, gf=0.319, secs=39.0, GB=0.06, MB/s= 1.59
52.00%, ll=-437.71479, gf=0.319, secs=39.2, GB=0.06, MB/s= 1.59
53.00%, ll=-656.66506, gf=0.319, secs=39.5, GB=0.06, MB/s= 1.59
54.00%, ll=-637.01681, gf=0.319, secs=39.8, GB=0.06, MB/s= 1.59
55.00%, ll=-614.52628, gf=0.319, secs=40.0, GB=0.06, MB/s= 1.59
56.00%, ll=-708.49725, gf=0.319, secs=40.3, GB=0.06, MB/s= 1.59
57.00%, ll=-607.26227, gf=0.319, secs=40.5, GB=0.06, MB/s= 1.59
58.00%, ll=-243.79403, gf=0.319, secs=40.8, GB=0.06, MB/s= 1.59
59.00%, ll=-331.82698, gf=0.320, secs=41.0, GB=0.07, MB/s= 1.59
60.00%, ll=-139.34635, gf=0.320, secs=41.2, GB=0.07, MB/s= 1.59
61.00%, ll=-172.45419, gf=0.320, secs=41.5, GB=0.07, MB/s= 1.59
62.00%, ll=-234.61018, gf=0.320, secs=41.8, GB=0.07, MB/s= 1.59
63.00%, ll=-168.23249, gf=0.319, secs=42.1, GB=0.07, MB/s= 1.59
64.00%, ll=-221.21087, gf=0.319, secs=42.3, GB=0.07, MB/s= 1.59
65.00%, ll=-167.68707, gf=0.319, secs=42.6, GB=0.07, MB/s= 1.59
66.00%, ll=-227.10013, gf=0.318, secs=43.0, GB=0.07, MB/s= 1.59
67.00%, ll=-163.47838, gf=0.318, secs=43.2, GB=0.07, MB/s= 1.59
68.00%, ll=-109.72068, gf=0.318, secs=43.5, GB=0.07, MB/s= 1.59
69.00%, ll=-113.18961, gf=0.318, secs=43.7, GB=0.07, MB/s= 1.59
70.00%, ll=-72.94882, gf=0.318, secs=44.0, GB=0.07, MB/s= 1.59
71.00%, ll=-43.47061, gf=0.318, secs=44.2, GB=0.07, MB/s= 1.59
72.00%, ll=-68.54942, gf=0.318, secs=44.5, GB=0.07, MB/s= 1.59
73.00%, ll=-45.34280, gf=0.318, secs=44.8, GB=0.07, MB/s= 1.59
74.00%, ll=-65.60710, gf=0.318, secs=45.1, GB=0.07, MB/s= 1.59
75.00%, ll=-33.25086, gf=0.318, secs=45.3, GB=0.07, MB/s= 1.59
76.00%, ll=-42.29474, gf=0.318, secs=45.6, GB=0.07, MB/s= 1.59
77.00%, ll=-43.68575, gf=0.318, secs=45.8, GB=0.07, MB/s= 1.58
78.00%, ll=-19.22573, gf=0.318, secs=46.1, GB=0.07, MB/s= 1.58
79.00%, ll=-984.16693, gf=0.318, secs=46.4, GB=0.07, MB/s= 1.58
80.00%, ll=-1371.85068, gf=0.318, secs=46.6, GB=0.07, MB/s= 1.58
81.00%, ll=-1233.89192, gf=0.318, secs=46.9, GB=0.07, MB/s= 1.58
82.00%, ll=-1062.01692, gf=0.318, secs=47.2, GB=0.07, MB/s= 1.58
83.00%, ll=-1154.86642, gf=0.318, secs=47.5, GB=0.08, MB/s= 1.58
84.00%, ll=-1254.69416, gf=0.318, secs=47.7, GB=0.08, MB/s= 1.58
85.00%, ll=-1075.32074, gf=0.318, secs=48.0, GB=0.08, MB/s= 1.58
86.00%, ll=-667.95506, gf=0.318, secs=48.2, GB=0.08, MB/s= 1.58
87.00%, ll=-873.43670, gf=0.318, secs=48.5, GB=0.08, MB/s= 1.58
88.00%, ll=-662.78516, gf=0.318, secs=48.8, GB=0.08, MB/s= 1.58
89.00%, ll=-533.77660, gf=0.318, secs=49.0, GB=0.08, MB/s= 1.58
90.00%, ll=-616.42464, gf=0.317, secs=49.3, GB=0.08, MB/s= 1.58
91.00%, ll=-483.22471, gf=0.317, secs=49.6, GB=0.08, MB/s= 1.58
92.00%, ll=-3.32617, gf=0.317, secs=49.9, GB=0.08, MB/s= 1.58
93.00%, ll=-3.63072, gf=0.317, secs=50.1, GB=0.08, MB/s= 1.58
94.00%, ll=-3.56535, gf=0.317, secs=50.4, GB=0.08, MB/s= 1.58
95.00%, ll=-1.52007, gf=0.317, secs=50.6, GB=0.08, MB/s= 1.58
96.00%, ll=-6.80332, gf=0.317, secs=50.9, GB=0.08, MB/s= 1.58
97.00%, ll=-5.95466, gf=0.317, secs=51.2, GB=0.08, MB/s= 1.58
98.00%, ll=-5.36539, gf=0.317, secs=51.4, GB=0.08, MB/s= 1.58
99.00%, ll=-5.33001, gf=0.317, secs=51.7, GB=0.08, MB/s= 1.58
100.00%, ll=-3.90117, gf=0.317, secs=51.9, GB=0.08, MB/s= 1.58
pass= 2
 1.00%, ll=-8.71211, gf=0.317, secs=52.2, GB=0.08, MB/s= 1.58
 2.00%, ll=-6.50329, gf=0.317, secs=52.4, GB=0.08, MB/s= 1.58
 3.00%, ll=-5.04205, gf=0.317, secs=52.7, GB=0.08, MB/s= 1.58
 4.00%, ll=-3.09802, gf=0.317, secs=53.0, GB=0.08, MB/s= 1.58
 5.00%, ll=-5.20730, gf=0.317, secs=53.2, GB=0.08, MB/s= 1.58
 6.00%, ll=-5.55460, gf=0.317, secs=53.5, GB=0.08, MB/s= 1.58
 7.00%, ll=-3.49589, gf=0.318, secs=53.7, GB=0.08, MB/s= 1.58
 8.00%, ll=-3.12580, gf=0.318, secs=54.0, GB=0.09, MB/s= 1.58
 9.00%, ll=-43.51551, gf=0.318, secs=54.2, GB=0.09, MB/s= 1.58
10.00%, ll=-258.02687, gf=0.318, secs=54.5, GB=0.09, MB/s= 1.58
11.00%, ll=-356.48877, gf=0.318, secs=54.7, GB=0.09, MB/s= 1.58
12.00%, ll=-402.12422, gf=0.318, secs=55.0, GB=0.09, MB/s= 1.58
13.00%, ll=-440.61029, gf=0.318, secs=55.2, GB=0.09, MB/s= 1.58
14.00%, ll=-299.26174, gf=0.318, secs=55.5, GB=0.09, MB/s= 1.58
15.00%, ll=-299.69775, gf=0.318, secs=55.7, GB=0.09, MB/s= 1.58
16.00%, ll=-237.46327, gf=0.318, secs=56.0, GB=0.09, MB/s= 1.58
17.00%, ll=-244.30888, gf=0.318, secs=56.2, GB=0.09, MB/s= 1.58
18.00%, ll=-156.62379, gf=0.318, secs=56.5, GB=0.09, MB/s= 1.58
19.00%, ll=-50.95429, gf=0.318, secs=56.7, GB=0.09, MB/s= 1.58
20.00%, ll=-49.34627, gf=0.318, secs=57.0, GB=0.09, MB/s= 1.59
21.00%, ll=-51.38906, gf=0.318, secs=57.2, GB=0.09, MB/s= 1.59
22.00%, ll=-144.95095, gf=0.318, secs=57.5, GB=0.09, MB/s= 1.59
23.00%, ll=-140.72177, gf=0.318, secs=57.7, GB=0.09, MB/s= 1.59
24.00%, ll=-91.52305, gf=0.318, secs=58.0, GB=0.09, MB/s= 1.59
25.00%, ll=-77.95444, gf=0.318, secs=58.3, GB=0.09, MB/s= 1.59
26.00%, ll=-94.72608, gf=0.318, secs=58.5, GB=0.09, MB/s= 1.59
27.00%, ll=-79.30152, gf=0.318, secs=58.8, GB=0.09, MB/s= 1.59
28.00%, ll=-75.98710, gf=0.318, secs=59.0, GB=0.09, MB/s= 1.59
29.00%, ll=-106.01610, gf=0.318, secs=59.3, GB=0.09, MB/s= 1.59
30.00%, ll=-92.45236, gf=0.318, secs=59.5, GB=0.09, MB/s= 1.59
31.00%, ll=-87.75554, gf=0.318, secs=59.8, GB=0.09, MB/s= 1.59
32.00%, ll=-85.30041, gf=0.318, secs=60.0, GB=0.10, MB/s= 1.59
33.00%, ll=-96.73228, gf=0.318, secs=60.3, GB=0.10, MB/s= 1.59
34.00%, ll=-89.52049, gf=0.318, secs=60.5, GB=0.10, MB/s= 1.59
35.00%, ll=-53.07807, gf=0.319, secs=60.8, GB=0.10, MB/s= 1.59
36.00%, ll=-58.91474, gf=0.319, secs=61.0, GB=0.10, MB/s= 1.59
37.00%, ll=-50.89544, gf=0.319, secs=61.3, GB=0.10, MB/s= 1.59
38.00%, ll=-48.76169, gf=0.319, secs=61.5, GB=0.10, MB/s= 1.59
39.00%, ll=-45.70095, gf=0.319, secs=61.8, GB=0.10, MB/s= 1.59
40.00%, ll=-39.22151, gf=0.319, secs=62.1, GB=0.10, MB/s= 1.59
41.00%, ll=-25.89944, gf=0.319, secs=62.3, GB=0.10, MB/s= 1.59
42.00%, ll=-23.20065, gf=0.319, secs=62.6, GB=0.10, MB/s= 1.59
43.00%, ll=-26.40041, gf=0.319, secs=62.9, GB=0.10, MB/s= 1.59
44.00%, ll=-27.83815, gf=0.319, secs=63.1, GB=0.10, MB/s= 1.59
45.00%, ll=-16.45190, gf=0.319, secs=63.4, GB=0.10, MB/s= 1.59
46.00%, ll=-13.14060, gf=0.319, secs=63.6, GB=0.10, MB/s= 1.59
47.00%, ll=-21.47491, gf=0.319, secs=63.9, GB=0.10, MB/s= 1.59
48.00%, ll=-24.18132, gf=0.319, secs=64.1, GB=0.10, MB/s= 1.59
49.00%, ll=-91.16539, gf=0.319, secs=64.4, GB=0.10, MB/s= 1.59
50.00%, ll=-173.99020, gf=0.319, secs=64.6, GB=0.10, MB/s= 1.59
51.00%, ll=-15.02484, gf=0.319, secs=64.9, GB=0.10, MB/s= 1.59
52.00%, ll=-14.53127, gf=0.319, secs=65.2, GB=0.10, MB/s= 1.59
53.00%, ll=-10.72194, gf=0.319, secs=65.4, GB=0.10, MB/s= 1.59
54.00%, ll=-10.26457, gf=0.319, secs=65.7, GB=0.10, MB/s= 1.59
55.00%, ll=-16.59847, gf=0.319, secs=65.9, GB=0.10, MB/s= 1.59
56.00%, ll=-19.62934, gf=0.319, secs=66.2, GB=0.11, MB/s= 1.59
57.00%, ll=-27.84986, gf=0.319, secs=66.4, GB=0.11, MB/s= 1.59
58.00%, ll=-45.32184, gf=0.319, secs=66.7, GB=0.11, MB/s= 1.59
59.00%, ll=-74.69407, gf=0.319, secs=67.0, GB=0.11, MB/s= 1.59
60.00%, ll=-55.98292, gf=0.319, secs=67.2, GB=0.11, MB/s= 1.59
61.00%, ll=-55.38607, gf=0.319, secs=67.5, GB=0.11, MB/s= 1.59
62.00%, ll=-50.99880, gf=0.319, secs=67.7, GB=0.11, MB/s= 1.59
63.00%, ll=-55.27645, gf=0.319, secs=68.0, GB=0.11, MB/s= 1.59
64.00%, ll=-47.12440, gf=0.319, secs=68.2, GB=0.11, MB/s= 1.59
65.00%, ll=-41.90715, gf=0.319, secs=68.5, GB=0.11, MB/s= 1.59
66.00%, ll=-32.01160, gf=0.319, secs=68.7, GB=0.11, MB/s= 1.59
67.00%, ll=-30.23642, gf=0.319, secs=69.0, GB=0.11, MB/s= 1.59
68.00%, ll=-31.88588, gf=0.319, secs=69.3, GB=0.11, MB/s= 1.59
69.00%, ll=-35.63775, gf=0.319, secs=69.5, GB=0.11, MB/s= 1.59
70.00%, ll=-84.30203, gf=0.319, secs=69.8, GB=0.11, MB/s= 1.59
71.00%, ll=-77.16696, gf=0.319, secs=70.1, GB=0.11, MB/s= 1.59
72.00%, ll=-67.51027, gf=0.319, secs=70.3, GB=0.11, MB/s= 1.59
73.00%, ll=-70.51536, gf=0.319, secs=70.6, GB=0.11, MB/s= 1.59
74.00%, ll=-54.48104, gf=0.319, secs=70.8, GB=0.11, MB/s= 1.59
75.00%, ll=-29.99108, gf=0.319, secs=71.1, GB=0.11, MB/s= 1.59
76.00%, ll=-30.90556, gf=0.319, secs=71.3, GB=0.11, MB/s= 1.59
77.00%, ll=-24.37481, gf=0.319, secs=71.6, GB=0.11, MB/s= 1.59
78.00%, ll=-19.64175, gf=0.319, secs=71.9, GB=0.11, MB/s= 1.59
79.00%, ll=-27.59511, gf=0.319, secs=72.1, GB=0.11, MB/s= 1.59
80.00%, ll=-37.34544, gf=0.319, secs=72.4, GB=0.11, MB/s= 1.59
81.00%, ll=-27.71591, gf=0.319, secs=72.6, GB=0.12, MB/s= 1.59
82.00%, ll=-47.56291, gf=0.319, secs=72.9, GB=0.12, MB/s= 1.59
83.00%, ll=-51.80321, gf=0.319, secs=73.2, GB=0.12, MB/s= 1.59
84.00%, ll=-70.26992, gf=0.319, secs=73.4, GB=0.12, MB/s= 1.59
85.00%, ll=-103.80816, gf=0.319, secs=73.7, GB=0.12, MB/s= 1.59
86.00%, ll=-65.29522, gf=0.319, secs=74.0, GB=0.12, MB/s= 1.59
87.00%, ll=-91.87321, gf=0.319, secs=74.2, GB=0.12, MB/s= 1.59
88.00%, ll=-72.20689, gf=0.319, secs=74.5, GB=0.12, MB/s= 1.59
89.00%, ll=-65.06143, gf=0.319, secs=74.7, GB=0.12, MB/s= 1.59
90.00%, ll=-68.89834, gf=0.319, secs=75.0, GB=0.12, MB/s= 1.59
91.00%, ll=-250.87706, gf=0.318, secs=75.3, GB=0.12, MB/s= 1.59
92.00%, ll=-1487.52918, gf=0.318, secs=75.5, GB=0.12, MB/s= 1.59
93.00%, ll=-1259.59466, gf=0.318, secs=75.8, GB=0.12, MB/s= 1.59
94.00%, ll=-1340.89347, gf=0.318, secs=76.1, GB=0.12, MB/s= 1.59
95.00%, ll=-1154.31919, gf=0.318, secs=76.3, GB=0.12, MB/s= 1.59
96.00%, ll=-959.80090, gf=0.318, secs=76.6, GB=0.12, MB/s= 1.59
97.00%, ll=-763.00857, gf=0.318, secs=76.9, GB=0.12, MB/s= 1.59
98.00%, ll=-609.96684, gf=0.318, secs=77.1, GB=0.12, MB/s= 1.59
99.00%, ll=-704.23714, gf=0.318, secs=77.4, GB=0.12, MB/s= 1.59
100.00%, ll=-445.63839, gf=0.318, secs=77.7, GB=0.12, MB/s= 1.59
pass= 3
 1.00%, ll=-717.76972, gf=0.318, secs=77.9, GB=0.12, MB/s= 1.59
 2.00%, ll=-679.72499, gf=0.318, secs=78.2, GB=0.12, MB/s= 1.59
 3.00%, ll=-653.90310, gf=0.318, secs=78.5, GB=0.12, MB/s= 1.59
 4.00%, ll=-595.31197, gf=0.318, secs=78.7, GB=0.12, MB/s= 1.59
 5.00%, ll=-516.43180, gf=0.318, secs=79.0, GB=0.13, MB/s= 1.59
 6.00%, ll=-337.27244, gf=0.318, secs=79.2, GB=0.13, MB/s= 1.59
 7.00%, ll=-323.29019, gf=0.318, secs=79.5, GB=0.13, MB/s= 1.59
 8.00%, ll=-300.03517, gf=0.318, secs=79.8, GB=0.13, MB/s= 1.58
 9.00%, ll=-206.51879, gf=0.318, secs=80.0, GB=0.13, MB/s= 1.58
10.00%, ll=-287.47154, gf=0.318, secs=80.3, GB=0.13, MB/s= 1.58
11.00%, ll=-304.95757, gf=0.318, secs=80.6, GB=0.13, MB/s= 1.58
12.00%, ll=-318.80215, gf=0.318, secs=80.9, GB=0.13, MB/s= 1.58
13.00%, ll=-223.91136, gf=0.318, secs=81.1, GB=0.13, MB/s= 1.58
14.00%, ll=-89.74234, gf=0.318, secs=81.4, GB=0.13, MB/s= 1.58
15.00%, ll=-128.30940, gf=0.318, secs=81.7, GB=0.13, MB/s= 1.58
16.00%, ll=-90.56816, gf=0.318, secs=81.9, GB=0.13, MB/s= 1.58
17.00%, ll=-85.45535, gf=0.318, secs=82.2, GB=0.13, MB/s= 1.58
18.00%, ll=-267.57348, gf=0.318, secs=82.5, GB=0.13, MB/s= 1.58
19.00%, ll=-361.26716, gf=0.318, secs=82.8, GB=0.13, MB/s= 1.58
20.00%, ll=-324.55186, gf=0.318, secs=83.0, GB=0.13, MB/s= 1.58
21.00%, ll=-349.20210, gf=0.317, secs=83.4, GB=0.13, MB/s= 1.58
22.00%, ll=-268.31432, gf=0.317, secs=83.7, GB=0.13, MB/s= 1.58
23.00%, ll=-240.47721, gf=0.317, secs=84.0, GB=0.13, MB/s= 1.58
24.00%, ll=-172.94295, gf=0.317, secs=84.2, GB=0.13, MB/s= 1.58
25.00%, ll=-180.53126, gf=0.317, secs=84.5, GB=0.13, MB/s= 1.58
26.00%, ll=-164.02624, gf=0.317, secs=84.8, GB=0.13, MB/s= 1.58
27.00%, ll=-178.32199, gf=0.317, secs=85.1, GB=0.13, MB/s= 1.58
28.00%, ll=-123.12511, gf=0.317, secs=85.3, GB=0.13, MB/s= 1.58
29.00%, ll=-42.91422, gf=0.317, secs=85.6, GB=0.14, MB/s= 1.58
30.00%, ll=-35.60046, gf=0.317, secs=85.9, GB=0.14, MB/s= 1.58
31.00%, ll=-34.41886, gf=0.317, secs=86.2, GB=0.14, MB/s= 1.58
32.00%, ll=-45.11995, gf=0.316, secs=86.4, GB=0.14, MB/s= 1.58
33.00%, ll=-40.07172, gf=0.316, secs=86.7, GB=0.14, MB/s= 1.58
34.00%, ll=-46.95006, gf=0.316, secs=87.1, GB=0.14, MB/s= 1.57
35.00%, ll=-39.13320, gf=0.316, secs=87.4, GB=0.14, MB/s= 1.57
36.00%, ll=-39.74008, gf=0.316, secs=87.7, GB=0.14, MB/s= 1.57
37.00%, ll=-27.33822, gf=0.316, secs=88.0, GB=0.14, MB/s= 1.57
38.00%, ll=-28.01219, gf=0.315, secs=88.3, GB=0.14, MB/s= 1.57
39.00%, ll=-22.51178, gf=0.315, secs=88.6, GB=0.14, MB/s= 1.57
40.00%, ll=-32.48937, gf=0.315, secs=88.9, GB=0.14, MB/s= 1.57
41.00%, ll=-23.42847, gf=0.315, secs=89.2, GB=0.14, MB/s= 1.57
42.00%, ll=-28.31828, gf=0.315, secs=89.5, GB=0.14, MB/s= 1.57
43.00%, ll=-21.22943, gf=0.315, secs=89.8, GB=0.14, MB/s= 1.57
44.00%, ll=-20.49073, gf=0.315, secs=90.1, GB=0.14, MB/s= 1.57
45.00%, ll=-20.79633, gf=0.314, secs=90.4, GB=0.14, MB/s= 1.57
46.00%, ll=-10.23617, gf=0.314, secs=90.7, GB=0.14, MB/s= 1.57
47.00%, ll=-9.92851, gf=0.314, secs=91.0, GB=0.14, MB/s= 1.57
48.00%, ll=-7.97194, gf=0.314, secs=91.2, GB=0.14, MB/s= 1.57
49.00%, ll=-9.27835, gf=0.314, secs=91.5, GB=0.14, MB/s= 1.57
50.00%, ll=-19.43092, gf=0.314, secs=91.8, GB=0.14, MB/s= 1.56
51.00%, ll=-49.37904, gf=0.314, secs=92.1, GB=0.14, MB/s= 1.56
52.00%, ll=-44.34407, gf=0.314, secs=92.5, GB=0.14, MB/s= 1.56
53.00%, ll=-37.51638, gf=0.313, secs=92.8, GB=0.14, MB/s= 1.56
54.00%, ll=-30.49244, gf=0.313, secs=93.1, GB=0.15, MB/s= 1.56
55.00%, ll=-20.81293, gf=0.313, secs=93.4, GB=0.15, MB/s= 1.56
56.00%, ll=-28.47211, gf=0.313, secs=93.7, GB=0.15, MB/s= 1.56
57.00%, ll=-45.44551, gf=0.313, secs=94.0, GB=0.15, MB/s= 1.56
58.00%, ll=-118.53058, gf=0.313, secs=94.2, GB=0.15, MB/s= 1.56
59.00%, ll=-176.68209, gf=0.313, secs=94.5, GB=0.15, MB/s= 1.56
60.00%, ll=-194.34716, gf=0.313, secs=94.8, GB=0.15, MB/s= 1.56
61.00%, ll=-212.77463, gf=0.313, secs=95.2, GB=0.15, MB/s= 1.56
62.00%, ll=-208.93902, gf=0.312, secs=95.4, GB=0.15, MB/s= 1.56
63.00%, ll=-181.04550, gf=0.312, secs=95.7, GB=0.15, MB/s= 1.56
64.00%, ll=-180.60658, gf=0.312, secs=96.0, GB=0.15, MB/s= 1.56
65.00%, ll=-131.39041, gf=0.312, secs=96.3, GB=0.15, MB/s= 1.56
66.00%, ll=-132.88622, gf=0.312, secs=96.6, GB=0.15, MB/s= 1.56
67.00%, ll=-113.63589, gf=0.312, secs=97.0, GB=0.15, MB/s= 1.55
68.00%, ll=-114.54951, gf=0.311, secs=97.4, GB=0.15, MB/s= 1.55
69.00%, ll=-93.78247, gf=0.311, secs=97.7, GB=0.15, MB/s= 1.55
70.00%, ll=-490.52911, gf=0.311, secs=98.0, GB=0.15, MB/s= 1.55
71.00%, ll=-1248.49225, gf=0.311, secs=98.3, GB=0.15, MB/s= 1.55
72.00%, ll=-1184.47296, gf=0.311, secs=98.6, GB=0.15, MB/s= 1.55
73.00%, ll=-1317.66102, gf=0.311, secs=98.9, GB=0.15, MB/s= 1.55
74.00%, ll=-1295.09448, gf=0.310, secs=99.2, GB=0.15, MB/s= 1.55
75.00%, ll=-935.67637, gf=0.310, secs=99.6, GB=0.15, MB/s= 1.55
76.00%, ll=-907.52849, gf=0.310, secs=99.9, GB=0.15, MB/s= 1.55
77.00%, ll=-739.84591, gf=0.310, secs=100.2, GB=0.15, MB/s= 1.54
78.00%, ll=-822.83125, gf=0.310, secs=100.6, GB=0.16, MB/s= 1.54
79.00%, ll=-847.94860, gf=0.309, secs=100.9, GB=0.16, MB/s= 1.54
80.00%, ll=-653.84570, gf=0.309, secs=101.2, GB=0.16, MB/s= 1.54
81.00%, ll=-597.11397, gf=0.309, secs=101.6, GB=0.16, MB/s= 1.54
82.00%, ll=-555.78954, gf=0.309, secs=101.9, GB=0.16, MB/s= 1.54
83.00%, ll=-629.58093, gf=0.309, secs=102.2, GB=0.16, MB/s= 1.54
84.00%, ll=-623.66228, gf=0.309, secs=102.5, GB=0.16, MB/s= 1.54
85.00%, ll=-595.66524, gf=0.308, secs=102.8, GB=0.16, MB/s= 1.54
86.00%, ll=-354.71881, gf=0.308, secs=103.1, GB=0.16, MB/s= 1.54
87.00%, ll=-466.93917, gf=0.308, secs=103.5, GB=0.16, MB/s= 1.54
88.00%, ll=-369.63253, gf=0.308, secs=103.8, GB=0.16, MB/s= 1.53
89.00%, ll=-307.81707, gf=0.308, secs=104.1, GB=0.16, MB/s= 1.53
90.00%, ll=-343.04807, gf=0.308, secs=104.4, GB=0.16, MB/s= 1.53
91.00%, ll=-302.69681, gf=0.308, secs=104.7, GB=0.16, MB/s= 1.53
92.00%, ll=-297.84823, gf=0.308, secs=105.0, GB=0.16, MB/s= 1.53
93.00%, ll=-275.21403, gf=0.307, secs=105.3, GB=0.16, MB/s= 1.53
94.00%, ll=-268.06623, gf=0.307, secs=105.7, GB=0.16, MB/s= 1.53
95.00%, ll=-275.75463, gf=0.307, secs=106.0, GB=0.16, MB/s= 1.53
96.00%, ll=-228.49878, gf=0.307, secs=106.3, GB=0.16, MB/s= 1.53
97.00%, ll=-190.80151, gf=0.307, secs=106.6, GB=0.16, MB/s= 1.53
98.00%, ll=-167.38585, gf=0.307, secs=106.9, GB=0.16, MB/s= 1.53
99.00%, ll=-150.11938, gf=0.306, secs=107.3, GB=0.16, MB/s= 1.53
100.00%, ll=-155.31689, gf=0.306, secs=107.6, GB=0.16, MB/s= 1.53
pass= 4
 1.00%, ll=-129.20811, gf=0.306, secs=107.9, GB=0.16, MB/s= 1.52
 2.00%, ll=-108.16590, gf=0.306, secs=108.3, GB=0.17, MB/s= 1.52
 3.00%, ll=-108.28791, gf=0.306, secs=108.6, GB=0.17, MB/s= 1.52
 4.00%, ll=-112.76614, gf=0.306, secs=108.9, GB=0.17, MB/s= 1.52
 5.00%, ll=-114.94918, gf=0.305, secs=109.3, GB=0.17, MB/s= 1.52
 6.00%, ll=-98.06148, gf=0.305, secs=109.6, GB=0.17, MB/s= 1.52
 7.00%, ll=-70.98461, gf=0.305, secs=109.9, GB=0.17, MB/s= 1.52
 8.00%, ll=-70.54982, gf=0.305, secs=110.2, GB=0.17, MB/s= 1.52
 9.00%, ll=-69.69235, gf=0.305, secs=110.5, GB=0.17, MB/s= 1.52
10.00%, ll=-13.22109, gf=0.305, secs=110.8, GB=0.17, MB/s= 1.52
11.00%, ll=-11.91460, gf=0.305, secs=111.1, GB=0.17, MB/s= 1.52
12.00%, ll=-11.86538, gf=0.305, secs=111.4, GB=0.17, MB/s= 1.52
13.00%, ll=-14.81396, gf=0.305, secs=111.7, GB=0.17, MB/s= 1.52
14.00%, ll=-7.15441, gf=0.304, secs=112.0, GB=0.17, MB/s= 1.52
15.00%, ll=-5.33730, gf=0.304, secs=112.3, GB=0.17, MB/s= 1.52
16.00%, ll=-3.15092, gf=0.304, secs=112.7, GB=0.17, MB/s= 1.52
17.00%, ll=-3.01839, gf=0.304, secs=113.0, GB=0.17, MB/s= 1.51
18.00%, ll=-24.50730, gf=0.304, secs=113.3, GB=0.17, MB/s= 1.51
19.00%, ll=-32.79653, gf=0.304, secs=113.6, GB=0.17, MB/s= 1.51
20.00%, ll=-31.97763, gf=0.304, secs=114.0, GB=0.17, MB/s= 1.51
21.00%, ll=-32.83305, gf=0.303, secs=114.3, GB=0.17, MB/s= 1.51
22.00%, ll=-32.29772, gf=0.303, secs=114.6, GB=0.17, MB/s= 1.51
23.00%, ll=-41.22037, gf=0.303, secs=114.9, GB=0.17, MB/s= 1.51
24.00%, ll=-27.47338, gf=0.303, secs=115.2, GB=0.17, MB/s= 1.51
25.00%, ll=-27.27996, gf=0.303, secs=115.6, GB=0.17, MB/s= 1.51
26.00%, ll=-21.88946, gf=0.303, secs=115.9, GB=0.17, MB/s= 1.51
27.00%, ll=-21.69312, gf=0.303, secs=116.2, GB=0.18, MB/s= 1.51
28.00%, ll=-18.20340, gf=0.303, secs=116.5, GB=0.18, MB/s= 1.51
29.00%, ll=-25.93993, gf=0.302, secs=116.9, GB=0.18, MB/s= 1.51
30.00%, ll=-19.16774, gf=0.302, secs=117.2, GB=0.18, MB/s= 1.51
31.00%, ll=-18.58393, gf=0.302, secs=117.5, GB=0.18, MB/s= 1.51
32.00%, ll=-20.41869, gf=0.302, secs=117.9, GB=0.18, MB/s= 1.50
33.00%, ll=-17.43607, gf=0.302, secs=118.2, GB=0.18, MB/s= 1.50
34.00%, ll=-27.31348, gf=0.302, secs=118.5, GB=0.18, MB/s= 1.50
35.00%, ll=-11.08048, gf=0.302, secs=118.8, GB=0.18, MB/s= 1.50
36.00%, ll=-11.04952, gf=0.302, secs=119.1, GB=0.18, MB/s= 1.50
37.00%, ll=-7.52791, gf=0.302, secs=119.4, GB=0.18, MB/s= 1.50
38.00%, ll=-7.27772, gf=0.301, secs=119.7, GB=0.18, MB/s= 1.50
39.00%, ll=-10.93575, gf=0.301, secs=120.0, GB=0.18, MB/s= 1.50
40.00%, ll=-15.75243, gf=0.301, secs=120.3, GB=0.18, MB/s= 1.50
41.00%, ll=-9.88059, gf=0.301, secs=120.7, GB=0.18, MB/s= 1.50
42.00%, ll=-9.31790, gf=0.301, secs=121.0, GB=0.18, MB/s= 1.50
43.00%, ll=-5.34069, gf=0.301, secs=121.3, GB=0.18, MB/s= 1.50
44.00%, ll=-3.69781, gf=0.301, secs=121.6, GB=0.18, MB/s= 1.50
45.00%, ll=-151.21700, gf=0.301, secs=122.0, GB=0.18, MB/s= 1.50
46.00%, ll=-149.43375, gf=0.300, secs=122.3, GB=0.18, MB/s= 1.50
47.00%, ll=-127.27114, gf=0.300, secs=122.7, GB=0.18, MB/s= 1.50
48.00%, ll=-111.40674, gf=0.300, secs=123.0, GB=0.18, MB/s= 1.50
49.00%, ll=-111.34736, gf=0.300, secs=123.3, GB=0.18, MB/s= 1.49
50.00%, ll=-93.72431, gf=0.300, secs=123.7, GB=0.18, MB/s= 1.49
51.00%, ll=-10.06171, gf=0.300, secs=124.0, GB=0.19, MB/s= 1.49
52.00%, ll=-8.92431, gf=0.300, secs=124.3, GB=0.19, MB/s= 1.49
53.00%, ll=-7.61122, gf=0.299, secs=124.6, GB=0.19, MB/s= 1.49
54.00%, ll=-9.27828, gf=0.299, secs=125.0, GB=0.19, MB/s= 1.49
55.00%, ll=-7.14078, gf=0.299, secs=125.3, GB=0.19, MB/s= 1.49
56.00%, ll=-163.77565, gf=0.299, secs=125.6, GB=0.19, MB/s= 1.49
57.00%, ll=-153.68428, gf=0.299, secs=125.9, GB=0.19, MB/s= 1.49
58.00%, ll=-230.11629, gf=0.299, secs=126.2, GB=0.19, MB/s= 1.49
59.00%, ll=-253.63467, gf=0.299, secs=126.5, GB=0.19, MB/s= 1.49
60.00%, ll=-184.97159, gf=0.299, secs=126.9, GB=0.19, MB/s= 1.49
61.00%, ll=-207.65201, gf=0.299, secs=127.2, GB=0.19, MB/s= 1.49
62.00%, ll=-281.17592, gf=0.299, secs=127.5, GB=0.19, MB/s= 1.49
63.00%, ll=-164.88195, gf=0.298, secs=127.8, GB=0.19, MB/s= 1.49
64.00%, ll=-185.25961, gf=0.298, secs=128.1, GB=0.19, MB/s= 1.49
65.00%, ll=-84.38912, gf=0.298, secs=128.5, GB=0.19, MB/s= 1.49
66.00%, ll=-79.65956, gf=0.298, secs=128.8, GB=0.19, MB/s= 1.49
67.00%, ll=-85.60461, gf=0.298, secs=129.1, GB=0.19, MB/s= 1.48
68.00%, ll=-106.51467, gf=0.298, secs=129.4, GB=0.19, MB/s= 1.48
69.00%, ll=-123.90778, gf=0.298, secs=129.7, GB=0.19, MB/s= 1.48
70.00%, ll=-131.03039, gf=0.298, secs=130.0, GB=0.19, MB/s= 1.48
71.00%, ll=-75.51138, gf=0.298, secs=130.4, GB=0.19, MB/s= 1.48
72.00%, ll=-77.68046, gf=0.298, secs=130.7, GB=0.19, MB/s= 1.48
73.00%, ll=-113.12371, gf=0.298, secs=131.0, GB=0.19, MB/s= 1.48
74.00%, ll=-80.04567, gf=0.297, secs=131.3, GB=0.19, MB/s= 1.48
75.00%, ll=-38.99007, gf=0.297, secs=131.7, GB=0.19, MB/s= 1.48
76.00%, ll=-48.54033, gf=0.297, secs=132.0, GB=0.20, MB/s= 1.48
77.00%, ll=-52.04978, gf=0.297, secs=132.3, GB=0.20, MB/s= 1.48
78.00%, ll=-56.29169, gf=0.297, secs=132.7, GB=0.20, MB/s= 1.48
79.00%, ll=-125.50292, gf=0.297, secs=133.0, GB=0.20, MB/s= 1.48
80.00%, ll=-185.11482, gf=0.296, secs=133.4, GB=0.20, MB/s= 1.48
81.00%, ll=-101.57185, gf=0.296, secs=133.7, GB=0.20, MB/s= 1.48
82.00%, ll=-124.28298, gf=0.296, secs=134.1, GB=0.20, MB/s= 1.48
83.00%, ll=-168.67059, gf=0.296, secs=134.4, GB=0.20, MB/s= 1.48
84.00%, ll=-126.00529, gf=0.296, secs=134.7, GB=0.20, MB/s= 1.48
85.00%, ll=-86.34231, gf=0.296, secs=135.1, GB=0.20, MB/s= 1.47
86.00%, ll=-55.46620, gf=0.296, secs=135.4, GB=0.20, MB/s= 1.47
87.00%, ll=-70.09693, gf=0.296, secs=135.7, GB=0.20, MB/s= 1.47
88.00%, ll=-70.52486, gf=0.296, secs=136.1, GB=0.20, MB/s= 1.47
89.00%, ll=-67.45006, gf=0.295, secs=136.4, GB=0.20, MB/s= 1.47
90.00%, ll=-61.13628, gf=0.295, secs=136.7, GB=0.20, MB/s= 1.47
91.00%, ll=-51.37682, gf=0.295, secs=137.0, GB=0.20, MB/s= 1.47
92.00%, ll=-55.57420, gf=0.295, secs=137.4, GB=0.20, MB/s= 1.47
93.00%, ll=-40.75160, gf=0.295, secs=137.7, GB=0.20, MB/s= 1.47
94.00%, ll=-43.62139, gf=0.295, secs=138.0, GB=0.20, MB/s= 1.47
95.00%, ll=-41.50815, gf=0.295, secs=138.3, GB=0.20, MB/s= 1.47
96.00%, ll=-22.62469, gf=0.295, secs=138.7, GB=0.20, MB/s= 1.47
97.00%, ll=-16.99770, gf=0.295, secs=139.0, GB=0.20, MB/s= 1.47
98.00%, ll=-20.46793, gf=0.295, secs=139.3, GB=0.20, MB/s= 1.47
99.00%, ll=-17.72652, gf=0.294, secs=139.6, GB=0.20, MB/s= 1.47
100.00%, ll=-14.15976, gf=0.294, secs=140.0, GB=0.21, MB/s= 1.47
pass= 5
 1.00%, ll=-30.55226, gf=0.294, secs=140.3, GB=0.21, MB/s= 1.47
 2.00%, ll=-25.35961, gf=0.294, secs=140.6, GB=0.21, MB/s= 1.47
 3.00%, ll=-31.07155, gf=0.294, secs=140.9, GB=0.21, MB/s= 1.47
 4.00%, ll=-29.20745, gf=0.294, secs=141.3, GB=0.21, MB/s= 1.46
 5.00%, ll=-30.68203, gf=0.294, secs=141.6, GB=0.21, MB/s= 1.46
 6.00%, ll=-22.28540, gf=0.294, secs=141.9, GB=0.21, MB/s= 1.46
 7.00%, ll=-22.07111, gf=0.294, secs=142.3, GB=0.21, MB/s= 1.46
 8.00%, ll=-20.24498, gf=0.293, secs=142.6, GB=0.21, MB/s= 1.46
 9.00%, ll=-19.29648, gf=0.293, secs=142.9, GB=0.21, MB/s= 1.46
10.00%, ll=-16.41680, gf=0.293, secs=143.4, GB=0.21, MB/s= 1.46
11.00%, ll=-16.08284, gf=0.293, secs=143.8, GB=0.21, MB/s= 1.46
12.00%, ll=-16.29440, gf=0.293, secs=144.1, GB=0.21, MB/s= 1.46
13.00%, ll=-12.67494, gf=0.293, secs=144.4, GB=0.21, MB/s= 1.46
14.00%, ll=-7.69833, gf=0.293, secs=144.8, GB=0.21, MB/s= 1.46
15.00%, ll=-8.67508, gf=0.292, secs=145.1, GB=0.21, MB/s= 1.46
16.00%, ll=-7.86995, gf=0.292, secs=145.4, GB=0.21, MB/s= 1.46
17.00%, ll=-6.32880, gf=0.292, secs=145.8, GB=0.21, MB/s= 1.46
18.00%, ll=-13.50471, gf=0.292, secs=146.1, GB=0.21, MB/s= 1.46
19.00%, ll=-15.30005, gf=0.292, secs=146.4, GB=0.21, MB/s= 1.46
20.00%, ll=-11.49187, gf=0.292, secs=146.8, GB=0.21, MB/s= 1.45
21.00%, ll=-14.43030, gf=0.292, secs=147.1, GB=0.21, MB/s= 1.45
22.00%, ll=-10.96418, gf=0.292, secs=147.4, GB=0.21, MB/s= 1.45
23.00%, ll=-6.45981, gf=0.292, secs=147.8, GB=0.21, MB/s= 1.45
24.00%, ll=-9.34359, gf=0.291, secs=148.1, GB=0.22, MB/s= 1.45
25.00%, ll=-10.46101, gf=0.291, secs=148.4, GB=0.22, MB/s= 1.45
26.00%, ll=-9.81766, gf=0.291, secs=148.8, GB=0.22, MB/s= 1.45
27.00%, ll=-10.42491, gf=0.291, secs=149.1, GB=0.22, MB/s= 1.45
28.00%, ll=-6.35330, gf=0.291, secs=149.4, GB=0.22, MB/s= 1.45
29.00%, ll=-10.60782, gf=0.291, secs=149.8, GB=0.22, MB/s= 1.45
30.00%, ll=-8.21619, gf=0.291, secs=150.1, GB=0.22, MB/s= 1.45
31.00%, ll=-7.54014, gf=0.291, secs=150.5, GB=0.22, MB/s= 1.45
32.00%, ll=-8.33126, gf=0.291, secs=150.8, GB=0.22, MB/s= 1.45
33.00%, ll=-19.54793, gf=0.291, secs=151.1, GB=0.22, MB/s= 1.45
34.00%, ll=-34.57597, gf=0.290, secs=151.5, GB=0.22, MB/s= 1.45
35.00%, ll=-19.16981, gf=0.290, secs=151.8, GB=0.22, MB/s= 1.45
36.00%, ll=-29.28189, gf=0.290, secs=152.1, GB=0.22, MB/s= 1.45
37.00%, ll=-37.60513, gf=0.290, secs=152.5, GB=0.22, MB/s= 1.45
38.00%, ll=-26.98699, gf=0.290, secs=152.9, GB=0.22, MB/s= 1.44
39.00%, ll=-37.89952, gf=0.290, secs=153.3, GB=0.22, MB/s= 1.44
40.00%, ll=-13.81167, gf=0.290, secs=153.6, GB=0.22, MB/s= 1.44
41.00%, ll=-13.22977, gf=0.289, secs=154.0, GB=0.22, MB/s= 1.44
42.00%, ll=-12.29633, gf=0.289, secs=154.3, GB=0.22, MB/s= 1.44
43.00%, ll=-7.61264, gf=0.289, secs=154.7, GB=0.22, MB/s= 1.44
44.00%, ll=-15.38875, gf=0.289, secs=155.1, GB=0.22, MB/s= 1.44
45.00%, ll=-30.24482, gf=0.289, secs=155.4, GB=0.22, MB/s= 1.44
46.00%, ll=-28.82513, gf=0.289, secs=155.8, GB=0.22, MB/s= 1.44
47.00%, ll=-33.51351, gf=0.289, secs=156.2, GB=0.22, MB/s= 1.44
48.00%, ll=-29.59997, gf=0.288, secs=156.5, GB=0.22, MB/s= 1.44
49.00%, ll=-30.47285, gf=0.288, secs=156.9, GB=0.23, MB/s= 1.44
50.00%, ll=-31.16506, gf=0.288, secs=157.2, GB=0.23, MB/s= 1.44
51.00%, ll=-8.63290, gf=0.288, secs=157.5, GB=0.23, MB/s= 1.44
52.00%, ll=-4.38140, gf=0.288, secs=157.9, GB=0.23, MB/s= 1.44
53.00%, ll=-4.89603, gf=0.288, secs=158.2, GB=0.23, MB/s= 1.43
54.00%, ll=-43.63346, gf=0.288, secs=158.6, GB=0.23, MB/s= 1.43
55.00%, ll=-29.74943, gf=0.288, secs=159.0, GB=0.23, MB/s= 1.43
56.00%, ll=-35.10405, gf=0.288, secs=159.3, GB=0.23, MB/s= 1.43
57.00%, ll=-30.44280, gf=0.287, secs=159.6, GB=0.23, MB/s= 1.43
58.00%, ll=-33.42212, gf=0.287, secs=160.0, GB=0.23, MB/s= 1.43
59.00%, ll=-9.40967, gf=0.287, secs=160.4, GB=0.23, MB/s= 1.43
60.00%, ll=-5.28213, gf=0.287, secs=160.7, GB=0.23, MB/s= 1.43
61.00%, ll=-6.32175, gf=0.287, secs=161.0, GB=0.23, MB/s= 1.43
62.00%, ll=-4.73522, gf=0.287, secs=161.4, GB=0.23, MB/s= 1.43
63.00%, ll=-5.26327, gf=0.287, secs=161.8, GB=0.23, MB/s= 1.43
64.00%, ll=-8.74722, gf=0.287, secs=162.1, GB=0.23, MB/s= 1.43
65.00%, ll=-5.54065, gf=0.287, secs=162.5, GB=0.23, MB/s= 1.43
66.00%, ll=-5.63826, gf=0.286, secs=162.8, GB=0.23, MB/s= 1.43
67.00%, ll=-5.93234, gf=0.286, secs=163.2, GB=0.23, MB/s= 1.43
68.00%, ll=-4.66043, gf=0.286, secs=163.5, GB=0.23, MB/s= 1.43
69.00%, ll=-5.61765, gf=0.286, secs=163.8, GB=0.23, MB/s= 1.43
70.00%, ll=-24.04232, gf=0.286, secs=164.2, GB=0.23, MB/s= 1.43
71.00%, ll=-20.85380, gf=0.286, secs=164.5, GB=0.23, MB/s= 1.42
72.00%, ll=-18.55257, gf=0.286, secs=164.9, GB=0.23, MB/s= 1.42
73.00%, ll=-21.63049, gf=0.286, secs=165.2, GB=0.24, MB/s= 1.42
74.00%, ll=-21.76684, gf=0.286, secs=165.6, GB=0.24, MB/s= 1.42
75.00%, ll=-16.03278, gf=0.286, secs=165.9, GB=0.24, MB/s= 1.42
76.00%, ll=-23.86850, gf=0.285, secs=166.3, GB=0.24, MB/s= 1.42
77.00%, ll=-29.25077, gf=0.285, secs=166.6, GB=0.24, MB/s= 1.42
78.00%, ll=-18.02007, gf=0.285, secs=167.0, GB=0.24, MB/s= 1.42
79.00%, ll=-377.16455, gf=0.285, secs=167.4, GB=0.24, MB/s= 1.42
80.00%, ll=-663.42876, gf=0.285, secs=167.8, GB=0.24, MB/s= 1.42
81.00%, ll=-399.38690, gf=0.285, secs=168.2, GB=0.24, MB/s= 1.42
82.00%, ll=-560.86771, gf=0.285, secs=168.5, GB=0.24, MB/s= 1.42
83.00%, ll=-670.21245, gf=0.284, secs=168.9, GB=0.24, MB/s= 1.42
84.00%, ll=-458.91245, gf=0.284, secs=169.2, GB=0.24, MB/s= 1.42
85.00%, ll=-521.71987, gf=0.284, secs=169.6, GB=0.24, MB/s= 1.42
86.00%, ll=-203.15596, gf=0.284, secs=169.9, GB=0.24, MB/s= 1.42
87.00%, ll=-279.35469, gf=0.284, secs=170.3, GB=0.24, MB/s= 1.42
88.00%, ll=-241.65518, gf=0.284, secs=170.6, GB=0.24, MB/s= 1.41
89.00%, ll=-238.40516, gf=0.284, secs=171.0, GB=0.24, MB/s= 1.41
90.00%, ll=-196.47455, gf=0.284, secs=171.3, GB=0.24, MB/s= 1.41
91.00%, ll=-171.28639, gf=0.284, secs=171.7, GB=0.24, MB/s= 1.41
92.00%, ll=-251.20678, gf=0.284, secs=172.0, GB=0.24, MB/s= 1.41
93.00%, ll=-223.65543, gf=0.283, secs=172.4, GB=0.24, MB/s= 1.41
94.00%, ll=-228.05365, gf=0.283, secs=172.8, GB=0.24, MB/s= 1.41
95.00%, ll=-200.96891, gf=0.283, secs=173.1, GB=0.24, MB/s= 1.41
96.00%, ll=-227.61247, gf=0.283, secs=173.5, GB=0.24, MB/s= 1.41
97.00%, ll=-161.64934, gf=0.283, secs=173.8, GB=0.25, MB/s= 1.41
98.00%, ll=-133.02177, gf=0.282, secs=174.6, GB=0.25, MB/s= 1.41
99.00%, ll=-131.55660, gf=0.282, secs=175.0, GB=0.25, MB/s= 1.40
100.00%, ll=-119.75572, gf=0.282, secs=175.4, GB=0.25, MB/s= 1.40
pass= 6
 1.00%, ll=-154.17286, gf=0.282, secs=175.8, GB=0.25, MB/s= 1.40
 2.00%, ll=-144.79610, gf=0.282, secs=176.1, GB=0.25, MB/s= 1.40
 3.00%, ll=-133.08462, gf=0.281, secs=176.5, GB=0.25, MB/s= 1.40
 4.00%, ll=-131.76184, gf=0.281, secs=176.9, GB=0.25, MB/s= 1.40
 5.00%, ll=-177.93181, gf=0.281, secs=177.2, GB=0.25, MB/s= 1.40
 6.00%, ll=-110.27848, gf=0.281, secs=177.6, GB=0.25, MB/s= 1.40
 7.00%, ll=-84.59204, gf=0.281, secs=178.0, GB=0.25, MB/s= 1.40
 8.00%, ll=-64.43133, gf=0.281, secs=178.4, GB=0.25, MB/s= 1.40
 9.00%, ll=-65.18812, gf=0.281, secs=178.7, GB=0.25, MB/s= 1.40
10.00%, ll=-75.31402, gf=0.281, secs=179.1, GB=0.25, MB/s= 1.40
11.00%, ll=-75.20522, gf=0.281, secs=179.5, GB=0.25, MB/s= 1.40
12.00%, ll=-78.88138, gf=0.280, secs=179.8, GB=0.25, MB/s= 1.40
13.00%, ll=-80.01564, gf=0.280, secs=180.2, GB=0.25, MB/s= 1.40
14.00%, ll=-41.23900, gf=0.280, secs=180.5, GB=0.25, MB/s= 1.40
15.00%, ll=-40.65094, gf=0.280, secs=180.9, GB=0.25, MB/s= 1.40
16.00%, ll=-29.85025, gf=0.280, secs=181.3, GB=0.25, MB/s= 1.40
17.00%, ll=-23.73754, gf=0.280, secs=181.6, GB=0.25, MB/s= 1.39
18.00%, ll=-486.43860, gf=0.280, secs=182.0, GB=0.25, MB/s= 1.39
19.00%, ll=-723.00179, gf=0.280, secs=182.4, GB=0.25, MB/s= 1.39
20.00%, ll=-688.90591, gf=0.280, secs=182.7, GB=0.25, MB/s= 1.39
21.00%, ll=-745.57021, gf=0.279, secs=183.1, GB=0.25, MB/s= 1.39
22.00%, ll=-733.21551, gf=0.279, secs=183.5, GB=0.26, MB/s= 1.39
23.00%, ll=-700.48228, gf=0.279, secs=183.9, GB=0.26, MB/s= 1.39
24.00%, ll=-452.08118, gf=0.279, secs=184.2, GB=0.26, MB/s= 1.39
25.00%, ll=-491.58726, gf=0.279, secs=184.6, GB=0.26, MB/s= 1.39
26.00%, ll=-473.56434, gf=0.279, secs=185.0, GB=0.26, MB/s= 1.39
27.00%, ll=-482.35010, gf=0.279, secs=185.4, GB=0.26, MB/s= 1.39
28.00%, ll=-382.56636, gf=0.279, secs=185.7, GB=0.26, MB/s= 1.39
29.00%, ll=-427.15014, gf=0.278, secs=186.1, GB=0.26, MB/s= 1.39
30.00%, ll=-332.11426, gf=0.278, secs=186.5, GB=0.26, MB/s= 1.39
31.00%, ll=-336.57182, gf=0.278, secs=186.8, GB=0.26, MB/s= 1.39
32.00%, ll=-396.96341, gf=0.278, secs=187.2, GB=0.26, MB/s= 1.39
33.00%, ll=-388.98645, gf=0.278, secs=187.6, GB=0.26, MB/s= 1.39
34.00%, ll=-443.94859, gf=0.278, secs=187.9, GB=0.26, MB/s= 1.38
35.00%, ll=-275.72223, gf=0.278, secs=188.3, GB=0.26, MB/s= 1.38
36.00%, ll=-304.66066, gf=0.278, secs=188.7, GB=0.26, MB/s= 1.38
37.00%, ll=-252.14711, gf=0.278, secs=189.1, GB=0.26, MB/s= 1.38
38.00%, ll=-268.16705, gf=0.277, secs=189.4, GB=0.26, MB/s= 1.38
39.00%, ll=-253.50570, gf=0.277, secs=189.8, GB=0.26, MB/s= 1.38
40.00%, ll=-272.60178, gf=0.277, secs=190.2, GB=0.26, MB/s= 1.38
41.00%, ll=-219.57196, gf=0.277, secs=190.6, GB=0.26, MB/s= 1.38
42.00%, ll=-223.83199, gf=0.277, secs=190.9, GB=0.26, MB/s= 1.38
43.00%, ll=-238.05156, gf=0.277, secs=191.3, GB=0.26, MB/s= 1.38
44.00%, ll=-272.30666, gf=0.277, secs=191.7, GB=0.26, MB/s= 1.38
45.00%, ll=-230.84580, gf=0.277, secs=192.1, GB=0.26, MB/s= 1.38
46.00%, ll=-169.81763, gf=0.277, secs=192.4, GB=0.27, MB/s= 1.38
47.00%, ll=-147.93731, gf=0.276, secs=192.8, GB=0.27, MB/s= 1.38
48.00%, ll=-136.57014, gf=0.276, secs=193.2, GB=0.27, MB/s= 1.38
49.00%, ll=-154.20353, gf=0.276, secs=193.6, GB=0.27, MB/s= 1.38
50.00%, ll=-194.12947, gf=0.276, secs=194.0, GB=0.27, MB/s= 1.38
51.00%, ll=-720.02216, gf=0.276, secs=194.4, GB=0.27, MB/s= 1.37
52.00%, ll=-911.49659, gf=0.276, secs=194.8, GB=0.27, MB/s= 1.37
53.00%, ll=-872.39119, gf=0.276, secs=195.1, GB=0.27, MB/s= 1.37
54.00%, ll=-899.36995, gf=0.276, secs=195.5, GB=0.27, MB/s= 1.37
55.00%, ll=-649.18823, gf=0.275, secs=195.9, GB=0.27, MB/s= 1.37
56.00%, ll=-856.28914, gf=0.275, secs=196.3, GB=0.27, MB/s= 1.37
57.00%, ll=-925.41908, gf=0.275, secs=196.6, GB=0.27, MB/s= 1.37
58.00%, ll=-558.18419, gf=0.275, secs=197.0, GB=0.27, MB/s= 1.37
59.00%, ll=-437.38161, gf=0.275, secs=197.4, GB=0.27, MB/s= 1.37
60.00%, ll=-421.62524, gf=0.275, secs=197.8, GB=0.27, MB/s= 1.37
61.00%, ll=-394.29156, gf=0.275, secs=198.2, GB=0.27, MB/s= 1.37
62.00%, ll=-344.81093, gf=0.275, secs=198.6, GB=0.27, MB/s= 1.37
63.00%, ll=-329.62713, gf=0.274, secs=199.0, GB=0.27, MB/s= 1.37
64.00%, ll=-404.47019, gf=0.274, secs=199.4, GB=0.27, MB/s= 1.37
65.00%, ll=-272.90317, gf=0.274, secs=199.8, GB=0.27, MB/s= 1.37
66.00%, ll=-340.26099, gf=0.274, secs=200.2, GB=0.27, MB/s= 1.37
67.00%, ll=-261.69351, gf=0.274, secs=200.6, GB=0.27, MB/s= 1.36
68.00%, ll=-253.50855, gf=0.274, secs=201.0, GB=0.27, MB/s= 1.36
69.00%, ll=-249.19748, gf=0.274, secs=201.4, GB=0.27, MB/s= 1.36
70.00%, ll=-207.32645, gf=0.274, secs=201.8, GB=0.28, MB/s= 1.36
71.00%, ll=-226.79980, gf=0.273, secs=202.2, GB=0.28, MB/s= 1.36
72.00%, ll=-235.06178, gf=0.273, secs=202.6, GB=0.28, MB/s= 1.36
73.00%, ll=-193.11035, gf=0.273, secs=203.0, GB=0.28, MB/s= 1.36
74.00%, ll=-193.95610, gf=0.273, secs=203.5, GB=0.28, MB/s= 1.36
75.00%, ll=-121.57894, gf=0.272, secs=204.1, GB=0.28, MB/s= 1.36
76.00%, ll=-128.72828, gf=0.272, secs=204.5, GB=0.28, MB/s= 1.36
77.00%, ll=-123.37432, gf=0.272, secs=204.9, GB=0.28, MB/s= 1.36
78.00%, ll=-84.55543, gf=0.272, secs=205.4, GB=0.28, MB/s= 1.36
79.00%, ll=-61.35545, gf=0.272, secs=205.8, GB=0.28, MB/s= 1.35
80.00%, ll=-32.81521, gf=0.272, secs=206.2, GB=0.28, MB/s= 1.35
81.00%, ll=-45.08668, gf=0.271, secs=206.7, GB=0.28, MB/s= 1.35
82.00%, ll=-40.17471, gf=0.271, secs=207.1, GB=0.28, MB/s= 1.35
83.00%, ll=-34.33367, gf=0.271, secs=207.6, GB=0.28, MB/s= 1.35
84.00%, ll=-28.18283, gf=0.271, secs=208.0, GB=0.28, MB/s= 1.35
85.00%, ll=-52.08568, gf=0.271, secs=208.4, GB=0.28, MB/s= 1.35
86.00%, ll=-16.91041, gf=0.270, secs=209.0, GB=0.28, MB/s= 1.35
87.00%, ll=-16.76242, gf=0.270, secs=209.4, GB=0.28, MB/s= 1.35
88.00%, ll=-21.42530, gf=0.270, secs=209.9, GB=0.28, MB/s= 1.35
89.00%, ll=-14.46404, gf=0.270, secs=210.3, GB=0.28, MB/s= 1.35
90.00%, ll=-9.28817, gf=0.270, secs=210.7, GB=0.28, MB/s= 1.34
91.00%, ll=-14.64216, gf=0.270, secs=211.1, GB=0.28, MB/s= 1.34
92.00%, ll=-1318.92362, gf=0.270, secs=211.4, GB=0.28, MB/s= 1.34
93.00%, ll=-1312.35418, gf=0.270, secs=211.8, GB=0.28, MB/s= 1.34
94.00%, ll=-1343.32492, gf=0.269, secs=212.3, GB=0.28, MB/s= 1.34
95.00%, ll=-966.46738, gf=0.269, secs=212.7, GB=0.29, MB/s= 1.34
96.00%, ll=-1246.61441, gf=0.269, secs=213.1, GB=0.29, MB/s= 1.34
97.00%, ll=-1111.97282, gf=0.269, secs=213.5, GB=0.29, MB/s= 1.34
98.00%, ll=-811.53502, gf=0.269, secs=213.9, GB=0.29, MB/s= 1.34
99.00%, ll=-956.12944, gf=0.269, secs=214.3, GB=0.29, MB/s= 1.34
100.00%, ll=-724.34294, gf=0.269, secs=214.8, GB=0.29, MB/s= 1.34
pass= 7
 1.00%, ll=-1152.39681, gf=0.268, secs=215.2, GB=0.29, MB/s= 1.34
 2.00%, ll=-1202.52120, gf=0.268, secs=215.6, GB=0.29, MB/s= 1.34
 3.00%, ll=-982.83737, gf=0.268, secs=216.0, GB=0.29, MB/s= 1.34
 4.00%, ll=-815.56123, gf=0.268, secs=216.4, GB=0.29, MB/s= 1.34
 5.00%, ll=-823.64406, gf=0.268, secs=216.8, GB=0.29, MB/s= 1.33
 6.00%, ll=-4042.00209, gf=0.268, secs=217.3, GB=0.29, MB/s= 1.33
 7.00%, ll=-7000.37197, gf=0.268, secs=217.6, GB=0.29, MB/s= 1.33
 8.00%, ll=-6972.73328, gf=0.268, secs=218.0, GB=0.29, MB/s= 1.33
 9.00%, ll=-4989.18904, gf=0.267, secs=218.4, GB=0.29, MB/s= 1.33
10.00%, ll=-7190.61860, gf=0.267, secs=218.9, GB=0.29, MB/s= 1.33
11.00%, ll=-7578.47723, gf=0.267, secs=219.3, GB=0.29, MB/s= 1.33
12.00%, ll=-7188.37739, gf=0.267, secs=219.7, GB=0.29, MB/s= 1.33
13.00%, ll=-6468.15683, gf=0.267, secs=220.1, GB=0.29, MB/s= 1.33
14.00%, ll=-3297.25419, gf=0.267, secs=220.5, GB=0.29, MB/s= 1.33
15.00%, ll=-3199.91743, gf=0.267, secs=220.9, GB=0.29, MB/s= 1.33
16.00%, ll=-2484.41402, gf=0.267, secs=221.3, GB=0.29, MB/s= 1.33
17.00%, ll=-1876.90416, gf=0.266, secs=221.7, GB=0.29, MB/s= 1.33
18.00%, ll=-4143.74474, gf=0.266, secs=222.1, GB=0.29, MB/s= 1.33
19.00%, ll=-6426.16285, gf=0.266, secs=222.5, GB=0.30, MB/s= 1.33
20.00%, ll=-6017.95514, gf=0.266, secs=223.0, GB=0.30, MB/s= 1.33
21.00%, ll=-5930.97900, gf=0.266, secs=223.4, GB=0.30, MB/s= 1.33
22.00%, ll=-5016.34269, gf=0.266, secs=223.8, GB=0.30, MB/s= 1.32
23.00%, ll=-4598.51625, gf=0.266, secs=224.2, GB=0.30, MB/s= 1.32
24.00%, ll=-3408.46913, gf=0.266, secs=224.6, GB=0.30, MB/s= 1.32
25.00%, ll=-3505.86681, gf=0.265, secs=225.1, GB=0.30, MB/s= 1.32
26.00%, ll=-3493.52008, gf=0.265, secs=225.5, GB=0.30, MB/s= 1.32
27.00%, ll=-3891.53540, gf=0.265, secs=225.9, GB=0.30, MB/s= 1.32
28.00%, ll=-2411.50811, gf=0.265, secs=226.3, GB=0.30, MB/s= 1.32
29.00%, ll=-2502.17065, gf=0.265, secs=226.7, GB=0.30, MB/s= 1.32
30.00%, ll=-1573.96038, gf=0.265, secs=227.2, GB=0.30, MB/s= 1.32
31.00%, ll=-1901.20104, gf=0.265, secs=227.6, GB=0.30, MB/s= 1.32
32.00%, ll=-2469.09587, gf=0.265, secs=228.0, GB=0.30, MB/s= 1.32
33.00%, ll=-1428.19830, gf=0.264, secs=228.4, GB=0.30, MB/s= 1.32
34.00%, ll=-1888.79772, gf=0.264, secs=228.8, GB=0.30, MB/s= 1.32
35.00%, ll=-1240.78369, gf=0.264, secs=229.3, GB=0.30, MB/s= 1.32
36.00%, ll=-1237.76201, gf=0.264, secs=229.7, GB=0.30, MB/s= 1.32
37.00%, ll=-1066.66657, gf=0.264, secs=230.1, GB=0.30, MB/s= 1.31
38.00%, ll=-961.97374, gf=0.264, secs=230.5, GB=0.30, MB/s= 1.31
39.00%, ll=-648.13604, gf=0.264, secs=231.0, GB=0.30, MB/s= 1.31
40.00%, ll=-731.24432, gf=0.263, secs=231.4, GB=0.30, MB/s= 1.31
41.00%, ll=-523.57990, gf=0.263, secs=231.8, GB=0.30, MB/s= 1.31
42.00%, ll=-856.73146, gf=0.263, secs=232.3, GB=0.30, MB/s= 1.31
43.00%, ll=-560.52278, gf=0.263, secs=232.7, GB=0.31, MB/s= 1.31
44.00%, ll=-512.32057, gf=0.263, secs=233.1, GB=0.31, MB/s= 1.31
45.00%, ll=-598.20200, gf=0.263, secs=233.6, GB=0.31, MB/s= 1.31
46.00%, ll=-192.91058, gf=0.263, secs=234.0, GB=0.31, MB/s= 1.31
47.00%, ll=-195.89268, gf=0.263, secs=234.4, GB=0.31, MB/s= 1.31
48.00%, ll=-190.91232, gf=0.262, secs=234.8, GB=0.31, MB/s= 1.31
49.00%, ll=-267.05667, gf=0.262, secs=235.3, GB=0.31, MB/s= 1.31
50.00%, ll=-540.30056, gf=0.262, secs=235.8, GB=0.31, MB/s= 1.31
51.00%, ll=-1411.16935, gf=0.262, secs=236.2, GB=0.31, MB/s= 1.31
52.00%, ll=-1349.35283, gf=0.262, secs=236.6, GB=0.31, MB/s= 1.30
53.00%, ll=-1188.33202, gf=0.262, secs=237.1, GB=0.31, MB/s= 1.30
54.00%, ll=-1070.91843, gf=0.261, secs=237.6, GB=0.31, MB/s= 1.30
55.00%, ll=-646.05891, gf=0.261, secs=238.1, GB=0.31, MB/s= 1.30
56.00%, ll=-921.27919, gf=0.261, secs=238.5, GB=0.31, MB/s= 1.30
57.00%, ll=-830.77609, gf=0.261, secs=239.0, GB=0.31, MB/s= 1.30
58.00%, ll=-663.91189, gf=0.261, secs=239.4, GB=0.31, MB/s= 1.30
59.00%, ll=-991.54128, gf=0.261, secs=239.9, GB=0.31, MB/s= 1.30
60.00%, ll=-1060.47502, gf=0.260, secs=240.4, GB=0.31, MB/s= 1.30
61.00%, ll=-1065.44921, gf=0.260, secs=240.8, GB=0.31, MB/s= 1.30
62.00%, ll=-971.76359, gf=0.260, secs=241.3, GB=0.31, MB/s= 1.30
63.00%, ll=-865.20853, gf=0.260, secs=241.9, GB=0.31, MB/s= 1.29
64.00%, ll=-939.39208, gf=0.260, secs=242.4, GB=0.31, MB/s= 1.29
65.00%, ll=-640.24136, gf=0.259, secs=242.9, GB=0.31, MB/s= 1.29
66.00%, ll=-725.86586, gf=0.259, secs=243.4, GB=0.31, MB/s= 1.29
67.00%, ll=-593.16520, gf=0.259, secs=243.9, GB=0.31, MB/s= 1.29
68.00%, ll=-579.85193, gf=0.259, secs=244.4, GB=0.32, MB/s= 1.29
69.00%, ll=-470.50328, gf=0.259, secs=244.9, GB=0.32, MB/s= 1.29
70.00%, ll=-106.00969, gf=0.259, secs=245.4, GB=0.32, MB/s= 1.29
71.00%, ll=-92.88879, gf=0.258, secs=245.9, GB=0.32, MB/s= 1.29
72.00%, ll=-83.85033, gf=0.258, secs=246.5, GB=0.32, MB/s= 1.29
73.00%, ll=-71.31951, gf=0.258, secs=247.1, GB=0.32, MB/s= 1.28
74.00%, ll=-102.29053, gf=0.258, secs=247.6, GB=0.32, MB/s= 1.28
75.00%, ll=-74.18225, gf=0.257, secs=248.2, GB=0.32, MB/s= 1.28
76.00%, ll=-93.29465, gf=0.257, secs=248.7, GB=0.32, MB/s= 1.28
77.00%, ll=-83.16320, gf=0.257, secs=249.3, GB=0.32, MB/s= 1.28
78.00%, ll=-86.75419, gf=0.256, secs=249.9, GB=0.32, MB/s= 1.28
79.00%, ll=-185.47900, gf=0.256, secs=250.4, GB=0.32, MB/s= 1.28
80.00%, ll=-353.85114, gf=0.256, secs=250.8, GB=0.32, MB/s= 1.28
81.00%, ll=-183.82860, gf=0.256, secs=251.3, GB=0.32, MB/s= 1.28
82.00%, ll=-296.91219, gf=0.256, secs=251.9, GB=0.32, MB/s= 1.27
83.00%, ll=-310.93603, gf=0.255, secs=252.5, GB=0.32, MB/s= 1.27
84.00%, ll=-194.42783, gf=0.255, secs=253.1, GB=0.32, MB/s= 1.27
85.00%, ll=-169.05789, gf=0.255, secs=253.6, GB=0.32, MB/s= 1.27
86.00%, ll=-97.20523, gf=0.255, secs=254.2, GB=0.32, MB/s= 1.27
87.00%, ll=-97.69001, gf=0.254, secs=254.8, GB=0.32, MB/s= 1.27
88.00%, ll=-126.73048, gf=0.254, secs=255.4, GB=0.32, MB/s= 1.27
89.00%, ll=-121.48938, gf=0.254, secs=255.8, GB=0.32, MB/s= 1.27
90.00%, ll=-92.87545, gf=0.254, secs=256.4, GB=0.32, MB/s= 1.26
91.00%, ll=-71.14594, gf=0.254, secs=256.9, GB=0.32, MB/s= 1.26
92.00%, ll=-152.63973, gf=0.254, secs=257.4, GB=0.33, MB/s= 1.26
93.00%, ll=-149.04703, gf=0.253, secs=257.8, GB=0.33, MB/s= 1.26
94.00%, ll=-135.98257, gf=0.253, secs=258.3, GB=0.33, MB/s= 1.26
95.00%, ll=-112.21918, gf=0.253, secs=258.7, GB=0.33, MB/s= 1.26
96.00%, ll=-114.16524, gf=0.253, secs=259.2, GB=0.33, MB/s= 1.26
97.00%, ll=-94.23502, gf=0.253, secs=259.7, GB=0.33, MB/s= 1.26
98.00%, ll=-79.47303, gf=0.253, secs=260.2, GB=0.33, MB/s= 1.26
99.00%, ll=-91.08668, gf=0.252, secs=260.7, GB=0.33, MB/s= 1.26
100.00%, ll=-64.77074, gf=0.252, secs=261.2, GB=0.33, MB/s= 1.26
pass= 8
 1.00%, ll=-114.81418, gf=0.252, secs=261.7, GB=0.33, MB/s= 1.26
 2.00%, ll=-119.25808, gf=0.252, secs=262.3, GB=0.33, MB/s= 1.26
 3.00%, ll=-96.55835, gf=0.252, secs=262.8, GB=0.33, MB/s= 1.25
 4.00%, ll=-86.13565, gf=0.252, secs=263.3, GB=0.33, MB/s= 1.25
 5.00%, ll=-90.32670, gf=0.251, secs=263.8, GB=0.33, MB/s= 1.25
 6.00%, ll=-72.21573, gf=0.251, secs=264.4, GB=0.33, MB/s= 1.25
 7.00%, ll=-60.10094, gf=0.251, secs=265.0, GB=0.33, MB/s= 1.25
 8.00%, ll=-60.80119, gf=0.251, secs=265.6, GB=0.33, MB/s= 1.25
 9.00%, ll=-50.02823, gf=0.250, secs=266.2, GB=0.33, MB/s= 1.25
10.00%, ll=-138.64071, gf=0.250, secs=266.8, GB=0.33, MB/s= 1.25
11.00%, ll=-156.56273, gf=0.250, secs=267.2, GB=0.33, MB/s= 1.25
12.00%, ll=-151.25363, gf=0.250, secs=267.9, GB=0.33, MB/s= 1.24
13.00%, ll=-164.66062, gf=0.250, secs=268.3, GB=0.33, MB/s= 1.24
14.00%, ll=-77.63577, gf=0.249, secs=268.9, GB=0.33, MB/s= 1.24
15.00%, ll=-73.24009, gf=0.249, secs=269.4, GB=0.33, MB/s= 1.24
16.00%, ll=-58.51324, gf=0.249, secs=269.9, GB=0.33, MB/s= 1.24
17.00%, ll=-53.23059, gf=0.249, secs=270.4, GB=0.34, MB/s= 1.24
18.00%, ll=-76.17949, gf=0.249, secs=271.0, GB=0.34, MB/s= 1.24
19.00%, ll=-100.35674, gf=0.248, secs=271.6, GB=0.34, MB/s= 1.24
20.00%, ll=-96.66929, gf=0.248, secs=272.1, GB=0.34, MB/s= 1.24
21.00%, ll=-94.71548, gf=0.248, secs=272.7, GB=0.34, MB/s= 1.24
22.00%, ll=-81.80150, gf=0.248, secs=273.1, GB=0.34, MB/s= 1.24
23.00%, ll=-78.06013, gf=0.248, secs=273.7, GB=0.34, MB/s= 1.23
24.00%, ll=-57.21400, gf=0.248, secs=274.2, GB=0.34, MB/s= 1.23
25.00%, ll=-56.65159, gf=0.247, secs=274.6, GB=0.34, MB/s= 1.23
26.00%, ll=-57.24205, gf=0.247, secs=275.1, GB=0.34, MB/s= 1.23
27.00%, ll=-60.03454, gf=0.247, secs=275.6, GB=0.34, MB/s= 1.23
28.00%, ll=-40.43767, gf=0.247, secs=276.2, GB=0.34, MB/s= 1.23
29.00%, ll=-45.72845, gf=0.247, secs=276.7, GB=0.34, MB/s= 1.23
30.00%, ll=-27.61619, gf=0.247, secs=277.2, GB=0.34, MB/s= 1.23
31.00%, ll=-32.71811, gf=0.247, secs=277.7, GB=0.34, MB/s= 1.23
32.00%, ll=-40.00929, gf=0.246, secs=278.2, GB=0.34, MB/s= 1.23
33.00%, ll=-24.60121, gf=0.246, secs=278.8, GB=0.34, MB/s= 1.23
34.00%, ll=-37.74259, gf=0.246, secs=279.2, GB=0.34, MB/s= 1.23
35.00%, ll=-25.78172, gf=0.246, secs=279.8, GB=0.34, MB/s= 1.22
36.00%, ll=-27.64751, gf=0.246, secs=280.3, GB=0.34, MB/s= 1.22
37.00%, ll=-22.95135, gf=0.245, secs=280.9, GB=0.34, MB/s= 1.22
38.00%, ll=-21.80322, gf=0.245, secs=281.4, GB=0.34, MB/s= 1.22
39.00%, ll=-16.14663, gf=0.245, secs=281.9, GB=0.34, MB/s= 1.22
40.00%, ll=-40.09220, gf=0.245, secs=282.4, GB=0.34, MB/s= 1.22
41.00%, ll=-34.84094, gf=0.245, secs=282.9, GB=0.35, MB/s= 1.22
42.00%, ll=-36.41733, gf=0.245, secs=283.4, GB=0.35, MB/s= 1.22
43.00%, ll=-35.00133, gf=0.245, secs=283.9, GB=0.35, MB/s= 1.22
44.00%, ll=-38.76471, gf=0.244, secs=284.5, GB=0.35, MB/s= 1.22
45.00%, ll=-32.27815, gf=0.244, secs=285.0, GB=0.35, MB/s= 1.22
46.00%, ll=-24.22671, gf=0.244, secs=285.6, GB=0.35, MB/s= 1.22
47.00%, ll=-21.75957, gf=0.244, secs=286.1, GB=0.35, MB/s= 1.22
48.00%, ll=-18.66357, gf=0.244, secs=286.6, GB=0.35, MB/s= 1.21
49.00%, ll=-18.60332, gf=0.244, secs=287.1, GB=0.35, MB/s= 1.21
50.00%, ll=-41.78505, gf=0.243, secs=287.6, GB=0.35, MB/s= 1.21
51.00%, ll=-81.92023, gf=0.243, secs=288.2, GB=0.35, MB/s= 1.21
52.00%, ll=-83.48793, gf=0.243, secs=288.7, GB=0.35, MB/s= 1.21
53.00%, ll=-68.56807, gf=0.243, secs=289.2, GB=0.35, MB/s= 1.21
54.00%, ll=-68.28362, gf=0.243, secs=289.7, GB=0.35, MB/s= 1.21
55.00%, ll=-54.51916, gf=0.243, secs=290.2, GB=0.35, MB/s= 1.21
56.00%, ll=-60.24733, gf=0.243, secs=290.7, GB=0.35, MB/s= 1.21
57.00%, ll=-61.19616, gf=0.242, secs=291.2, GB=0.35, MB/s= 1.21
58.00%, ll=-40.92040, gf=0.242, secs=291.7, GB=0.35, MB/s= 1.21
59.00%, ll=-14.54211, gf=0.242, secs=292.4, GB=0.35, MB/s= 1.21
60.00%, ll=-15.17246, gf=0.242, secs=292.9, GB=0.35, MB/s= 1.21
61.00%, ll=-14.53813, gf=0.242, secs=293.6, GB=0.35, MB/s= 1.20
62.00%, ll=-11.79827, gf=0.242, secs=294.1, GB=0.35, MB/s= 1.20
63.00%, ll=-9.82544, gf=0.241, secs=294.6, GB=0.35, MB/s= 1.20
64.00%, ll=-11.23041, gf=0.241, secs=295.1, GB=0.35, MB/s= 1.20
65.00%, ll=-7.78573, gf=0.241, secs=295.6, GB=0.36, MB/s= 1.20
66.00%, ll=-9.36032, gf=0.241, secs=296.2, GB=0.36, MB/s= 1.20
67.00%, ll=-6.90359, gf=0.241, secs=296.7, GB=0.36, MB/s= 1.20
68.00%, ll=-6.53759, gf=0.241, secs=297.3, GB=0.36, MB/s= 1.20
69.00%, ll=-5.26348, gf=0.240, secs=297.8, GB=0.36, MB/s= 1.20
70.00%, ll=-4.73811, gf=0.240, secs=298.2, GB=0.36, MB/s= 1.20
71.00%, ll=-4.70066, gf=0.240, secs=298.9, GB=0.36, MB/s= 1.20
72.00%, ll=-4.21879, gf=0.240, secs=299.4, GB=0.36, MB/s= 1.20
73.00%, ll=-2.70254, gf=0.240, secs=300.0, GB=0.36, MB/s= 1.19
74.00%, ll=-2.68771, gf=0.240, secs=300.5, GB=0.36, MB/s= 1.19
75.00%, ll=-1.63498, gf=0.239, secs=301.1, GB=0.36, MB/s= 1.19
76.00%, ll=-1.72821, gf=0.239, secs=301.6, GB=0.36, MB/s= 1.19
77.00%, ll=-1.68484, gf=0.239, secs=302.1, GB=0.36, MB/s= 1.19
78.00%, ll=-0.99461, gf=0.239, secs=302.7, GB=0.36, MB/s= 1.19
79.00%, ll=-2.88990, gf=0.239, secs=303.4, GB=0.36, MB/s= 1.19
80.00%, ll=-3.55453, gf=0.239, secs=303.9, GB=0.36, MB/s= 1.19
81.00%, ll=-1.94180, gf=0.238, secs=304.4, GB=0.36, MB/s= 1.19
82.00%, ll=-1.83365, gf=0.238, secs=305.1, GB=0.36, MB/s= 1.19
83.00%, ll=-1.36074, gf=0.238, secs=305.6, GB=0.36, MB/s= 1.19
84.00%, ll=-0.61791, gf=0.238, secs=306.2, GB=0.36, MB/s= 1.19
85.00%, ll=-0.85358, gf=0.238, secs=306.8, GB=0.36, MB/s= 1.18
86.00%, ll=-0.42939, gf=0.238, secs=307.3, GB=0.36, MB/s= 1.18
87.00%, ll=-0.13409, gf=0.237, secs=307.8, GB=0.36, MB/s= 1.18
88.00%, ll=-0.40665, gf=0.237, secs=308.3, GB=0.36, MB/s= 1.18
89.00%, ll=-0.31414, gf=0.237, secs=308.8, GB=0.36, MB/s= 1.18
90.00%, ll=-0.14237, gf=0.237, secs=309.3, GB=0.37, MB/s= 1.18
91.00%, ll=-0.40347, gf=0.237, secs=309.8, GB=0.37, MB/s= 1.18
92.00%, ll=-2.80531, gf=0.237, secs=310.3, GB=0.37, MB/s= 1.18
93.00%, ll=-3.74947, gf=0.237, secs=310.8, GB=0.37, MB/s= 1.18
94.00%, ll=-3.07905, gf=0.237, secs=311.3, GB=0.37, MB/s= 1.18
95.00%, ll=-2.09755, gf=0.236, secs=311.9, GB=0.37, MB/s= 1.18
96.00%, ll=-271.53002, gf=0.236, secs=312.5, GB=0.37, MB/s= 1.18
97.00%, ll=-558.82652, gf=0.236, secs=313.0, GB=0.37, MB/s= 1.18
98.00%, ll=-537.15936, gf=0.236, secs=313.6, GB=0.37, MB/s= 1.18
99.00%, ll=-446.62072, gf=0.236, secs=314.1, GB=0.37, MB/s= 1.18
100.00%, ll=-520.43841, gf=0.236, secs=314.6, GB=0.37, MB/s= 1.17
pass= 9
 1.00%, ll=-366.29108, gf=0.236, secs=315.1, GB=0.37, MB/s= 1.17
 2.00%, ll=-307.77598, gf=0.235, secs=315.6, GB=0.37, MB/s= 1.17
 3.00%, ll=-334.66002, gf=0.235, secs=316.1, GB=0.37, MB/s= 1.17
 4.00%, ll=-382.03089, gf=0.235, secs=316.7, GB=0.37, MB/s= 1.17
 5.00%, ll=-445.33339, gf=0.235, secs=317.2, GB=0.37, MB/s= 1.17
 6.00%, ll=-369.77279, gf=0.235, secs=317.8, GB=0.37, MB/s= 1.17
 7.00%, ll=-265.74005, gf=0.235, secs=318.2, GB=0.37, MB/s= 1.17
 8.00%, ll=-255.10237, gf=0.235, secs=318.8, GB=0.37, MB/s= 1.17
 9.00%, ll=-709.87209, gf=0.235, secs=319.3, GB=0.37, MB/s= 1.17
10.00%, ll=-978.32931, gf=0.234, secs=319.8, GB=0.37, MB/s= 1.17
11.00%, ll=-1429.39377, gf=0.234, secs=320.3, GB=0.37, MB/s= 1.17
12.00%, ll=-1202.91564, gf=0.234, secs=320.8, GB=0.37, MB/s= 1.17
13.00%, ll=-698.20295, gf=0.234, secs=321.3, GB=0.37, MB/s= 1.17
14.00%, ll=-195.30974, gf=0.234, secs=321.8, GB=0.38, MB/s= 1.17
15.00%, ll=-368.59152, gf=0.234, secs=322.3, GB=0.38, MB/s= 1.17
16.00%, ll=-377.40328, gf=0.234, secs=322.9, GB=0.38, MB/s= 1.16
17.00%, ll=-171.97670, gf=0.234, secs=323.4, GB=0.38, MB/s= 1.16
18.00%, ll=-1412.24722, gf=0.233, secs=324.0, GB=0.38, MB/s= 1.16
19.00%, ll=-2350.55721, gf=0.233, secs=324.5, GB=0.38, MB/s= 1.16
20.00%, ll=-2202.11660, gf=0.233, secs=325.1, GB=0.38, MB/s= 1.16
21.00%, ll=-2284.02007, gf=0.233, secs=325.6, GB=0.38, MB/s= 1.16
22.00%, ll=-1846.95702, gf=0.233, secs=326.1, GB=0.38, MB/s= 1.16
23.00%, ll=-1310.02909, gf=0.233, secs=326.6, GB=0.38, MB/s= 1.16
24.00%, ll=-304189.58281, gf=0.233, secs=327.3, GB=0.38, MB/s= 1.16
25.00%, ll=-315004.77714, gf=0.232, secs=328.0, GB=0.38, MB/s= 1.16
26.00%, ll=-302674.70593, gf=0.232, secs=328.5, GB=0.38, MB/s= 1.16
27.00%, ll=-282707.29536, gf=0.232, secs=329.2, GB=0.38, MB/s= 1.16
28.00%, ll=-280515.99092, gf=0.232, secs=329.7, GB=0.38, MB/s= 1.16
29.00%, ll=-394632.54657, gf=0.232, secs=330.3, GB=0.38, MB/s= 1.15
30.00%, ll=-342695.04056, gf=0.232, secs=330.8, GB=0.38, MB/s= 1.15
31.00%, ll=-327095.22680, gf=0.232, secs=331.2, GB=0.38, MB/s= 1.15
32.00%, ll=-343217.22068, gf=0.231, secs=331.7, GB=0.38, MB/s= 1.15
33.00%, ll=-420743.69770, gf=0.231, secs=332.2, GB=0.38, MB/s= 1.15
34.00%, ll=-554735.01649, gf=0.231, secs=332.6, GB=0.38, MB/s= 1.15
35.00%, ll=-309364.02704, gf=0.231, secs=333.1, GB=0.38, MB/s= 1.15
36.00%, ll=-344430.36484, gf=0.231, secs=333.7, GB=0.38, MB/s= 1.15
37.00%, ll=-281950.20494, gf=0.231, secs=334.2, GB=0.38, MB/s= 1.15
38.00%, ll=-315945.91837, gf=0.231, secs=334.7, GB=0.39, MB/s= 1.15
39.00%, ll=-319774.86975, gf=0.231, secs=335.2, GB=0.39, MB/s= 1.15
40.00%, ll=-286205.29058, gf=0.231, secs=335.6, GB=0.39, MB/s= 1.15
41.00%, ll=-250931.21777, gf=0.231, secs=336.1, GB=0.39, MB/s= 1.15
42.00%, ll=-223045.38772, gf=0.231, secs=336.6, GB=0.39, MB/s= 1.15
43.00%, ll=-274945.94326, gf=0.230, secs=337.2, GB=0.39, MB/s= 1.15
44.00%, ll=-340195.16989, gf=0.230, secs=337.7, GB=0.39, MB/s= 1.15
45.00%, ll=-313975.16028, gf=0.230, secs=338.2, GB=0.39, MB/s= 1.15
46.00%, ll=-263752.98390, gf=0.230, secs=338.7, GB=0.39, MB/s= 1.15
47.00%, ll=-220480.93354, gf=0.230, secs=339.2, GB=0.39, MB/s= 1.15
48.00%, ll=-203065.01623, gf=0.230, secs=339.7, GB=0.39, MB/s= 1.15
49.00%, ll=-205721.31894, gf=0.230, secs=340.2, GB=0.39, MB/s= 1.15
50.00%, ll=-210444.61659, gf=0.230, secs=340.7, GB=0.39, MB/s= 1.14
51.00%, ll=-100534.22998, gf=0.230, secs=341.2, GB=0.39, MB/s= 1.14
52.00%, ll=-104506.59544, gf=0.230, secs=341.7, GB=0.39, MB/s= 1.14
53.00%, ll=-115391.94471, gf=0.229, secs=342.1, GB=0.39, MB/s= 1.14
54.00%, ll=-159068.61363, gf=0.229, secs=342.6, GB=0.39, MB/s= 1.14
55.00%, ll=-79025.64796, gf=0.229, secs=343.1, GB=0.39, MB/s= 1.14
56.00%, ll=-70982.67897, gf=0.229, secs=343.5, GB=0.39, MB/s= 1.14
57.00%, ll=-84214.44642, gf=0.229, secs=344.0, GB=0.39, MB/s= 1.14
58.00%, ll=-86631.73022, gf=0.229, secs=344.5, GB=0.39, MB/s= 1.14
59.00%, ll=-85108.40269, gf=0.229, secs=344.9, GB=0.39, MB/s= 1.14
60.00%, ll=-63732.57190, gf=0.229, secs=345.4, GB=0.39, MB/s= 1.14
61.00%, ll=-70935.77755, gf=0.229, secs=345.9, GB=0.39, MB/s= 1.14
62.00%, ll=-62459.05768, gf=0.229, secs=346.3, GB=0.39, MB/s= 1.14
63.00%, ll=-92121.68931, gf=0.229, secs=346.8, GB=0.40, MB/s= 1.14
64.00%, ll=-76134.17000, gf=0.229, secs=347.3, GB=0.40, MB/s= 1.14
65.00%, ll=-57150.38296, gf=0.229, secs=347.8, GB=0.40, MB/s= 1.14
66.00%, ll=-46001.89302, gf=0.228, secs=348.3, GB=0.40, MB/s= 1.14
67.00%, ll=-46969.76829, gf=0.228, secs=348.8, GB=0.40, MB/s= 1.14
68.00%, ll=-62068.04870, gf=0.228, secs=349.3, GB=0.40, MB/s= 1.14
69.00%, ll=-59078.13815, gf=0.228, secs=349.8, GB=0.40, MB/s= 1.14
70.00%, ll=-77616.74337, gf=0.228, secs=350.3, GB=0.40, MB/s= 1.14
71.00%, ll=-76401.66609, gf=0.228, secs=350.9, GB=0.40, MB/s= 1.14
72.00%, ll=-76068.52902, gf=0.228, secs=351.4, GB=0.40, MB/s= 1.14
73.00%, ll=-86523.77007, gf=0.228, secs=351.9, GB=0.40, MB/s= 1.14
74.00%, ll=-98984.54267, gf=0.228, secs=352.4, GB=0.40, MB/s= 1.13
75.00%, ll=-67950.39835, gf=0.228, secs=352.9, GB=0.40, MB/s= 1.13
76.00%, ll=-64780.16756, gf=0.228, secs=353.4, GB=0.40, MB/s= 1.13
77.00%, ll=-55172.21175, gf=0.227, secs=353.9, GB=0.40, MB/s= 1.13
78.00%, ll=-65437.68864, gf=0.227, secs=354.4, GB=0.40, MB/s= 1.13
79.00%, ll=-58233.18559, gf=0.227, secs=354.9, GB=0.40, MB/s= 1.13
80.00%, ll=-28300.72562, gf=0.227, secs=355.4, GB=0.40, MB/s= 1.13
81.00%, ll=-40456.33580, gf=0.227, secs=355.9, GB=0.40, MB/s= 1.13
82.00%, ll=-32859.08428, gf=0.227, secs=356.4, GB=0.40, MB/s= 1.13
83.00%, ll=-28509.41140, gf=0.227, secs=356.9, GB=0.40, MB/s= 1.13
84.00%, ll=-45222.33812, gf=0.227, secs=357.4, GB=0.40, MB/s= 1.13
85.00%, ll=-54952.61864, gf=0.227, secs=357.9, GB=0.40, MB/s= 1.13
86.00%, ll=-31280.35201, gf=0.227, secs=358.4, GB=0.40, MB/s= 1.13
87.00%, ll=-40391.24491, gf=0.227, secs=359.0, GB=0.41, MB/s= 1.13
88.00%, ll=-31742.08747, gf=0.226, secs=359.5, GB=0.41, MB/s= 1.13
89.00%, ll=-24313.43038, gf=0.226, secs=360.0, GB=0.41, MB/s= 1.13
90.00%, ll=-34382.93989, gf=0.226, secs=360.5, GB=0.41, MB/s= 1.13
91.00%, ll=-31732.80484, gf=0.226, secs=361.1, GB=0.41, MB/s= 1.13
92.00%, ll=-28009.16360, gf=0.226, secs=361.6, GB=0.41, MB/s= 1.13
93.00%, ll=-27979.92489, gf=0.226, secs=362.1, GB=0.41, MB/s= 1.13
94.00%, ll=-25570.91474, gf=0.226, secs=362.7, GB=0.41, MB/s= 1.13
95.00%, ll=-29872.71675, gf=0.226, secs=363.2, GB=0.41, MB/s= 1.12
96.00%, ll=-25655.00337, gf=0.226, secs=363.8, GB=0.41, MB/s= 1.12
97.00%, ll=-21589.97939, gf=0.225, secs=364.4, GB=0.41, MB/s= 1.12
98.00%, ll=-20418.30583, gf=0.225, secs=364.9, GB=0.41, MB/s= 1.12
99.00%, ll=-16555.00601, gf=0.225, secs=365.5, GB=0.41, MB/s= 1.12
100.00%, ll=-20653.33467, gf=0.225, secs=366.0, GB=0.41, MB/s= 1.12
Time=366.3240 secs, gflops=0.22
corpus perplexity=7.583303
Predicting
100.00%, ll=-16422.49023, gf=0.005, secs=0.0, GB=0.00, MB/s= 2.60
Time=0.0060 secs, gflops=0.01






    Out[2]:





263156

Get the word and document counts from the data. This turns out to be equivalent to a matrix multiply. For a data matrix A and category matrix C, we want all (cat, word) pairs (i,j) such that C(i,k) and A(j,k) are both 1 - this means that document k contains word j, and is also tagged with category i. Summing over all documents gives us

$${\rm wordcatCounts(i,j)} = \sum_{k=1}^N C(i,k) A(j,k) = C * A^T$$

Because we are doing independent binary classifiers for each class, we need to construct the counts for words not in the class (negwcounts).

Finally, we add a smoothing count 0.5 to counts that could be zero.



In [45]:

    
val cx=zeros(ctest.nrows,ctest.ncols)
val (mm,mopts,nn,nopts)=GLM.learner(atrain,ctrain,atest,cx,0)
mopts.autoReset=false
mopts.useGPU=false
mopts.lrate=0.1
mopts.batchSize=2
mopts.dim=256
mopts.startBlock=0
mopts.npasses=10
mopts.updateAll=false
//mopts.what









    









    Out[45]:





false

Now compute the probabilities

pwordcat = probability that a word is in a cat, given the cat.
pwordncat = probability of a word, given the complement of the cat.
pcat = probability that doc is in a given cat.
spcat = sum of pcat probabilities (> 1 because docs can be in multiple cats)



In [11]:

    
//mm.train
//nn.predict
atrain.ncols









    









    



java.lang.NoClassDefFoundError: Could not initialize class

Now take the logs of those probabilities. Here we're using the formula presented here to match Naive Bayes to Logistic Regression for independent data.

For each word, we compute the log of the ratio of the complementary word probability over the in-class word probability.

For each category, we compute the log of the ratio of the complementary category probability over the current category probability.

lpwordcat(j,i) represents $\log\left(\frac{{\rm Pr}(X_i|\neg c_j)}{{\rm Pr}(X_i|c_j)}\right)$

while lpcat(j) represents $\log\left(\frac{{\rm Pr}(\neg c)}{{\rm Pr}(c)}\right)$



In [3]:

    
val cx1=cx
min(cx1, 1, cx1)                       // the first "traindata" argument is the input, the other is output
max(cx1, 0, cx1) 
val p=ctest *@cx1 +(1-ctest) *@(1-cx1)
mean(p,2)
//saveFMat(rpath+index+".txt",cx)









    









    



java.lang.RuntimeException: dims incompatible
    BIDMat.DenseMat$mcF$sp.ggMatOpStrictv$mcF$sp(DenseMat.scala:963)
    BIDMat.DenseMat$mcF$sp.ggMatOpv$mcF$sp(DenseMat.scala:951)
    BIDMat.FMat.ffMatOpv(FMat.scala:206)
    BIDMat.FMat.$times$at(FMat.scala:799)

Here's where we apply Naive Bayes. The formula we're using is borrowed from here.

$${\rm Pr}(c|X_1,\ldots,X_k) = \frac{1}{1 + \frac{{\rm Pr}(\neg c)}{{\rm Pr}(c)}\prod_{i-1}^k\frac{{\rm Pr}(X_i|\neg c)}{{\rm Pr}(X_i|c)}}$$

and we can rewrite

$$\frac{{\rm Pr}(\neg c)}{{\rm Pr}(c)}\prod_{i-1}^k\frac{{\rm Pr}(X_i|\neg c)}{{\rm Pr}(X_i|c)}$$

$$\exp\left(\log\left(\frac{{\rm Pr}(\neg c)}{{\rm Pr}(c)}\right) + \sum_{i=1}^k\log\left(\frac{{\rm Pr}(X_i|\neg c)}{{\rm Pr}(X_i|c)}\right)\right) = \exp({\rm lpcat(j)} + {\rm lpwordcat(j,?)} * X)$$

for class number j and an input column $X$. This follows because an input column $X$ is a sparse vector with ones in the positions of the input features. The product ${\rm lpwordcat(i,?)} * X$ picks out the features occuring in the input document and adds the corresponding logs from lpwordcat.

Finally, we take the exponential above and fold it into the formula $P(c_j|X_1,\ldots,X_k) = 1/(1+\exp(\cdots))$. This gives us a matrix of predictions. preds(i,j) = prediction of membership in category i for test document j.



In [6]:

    
val lacc = (cx1 ∙→ ctest + (1-cx1) ∙→ (1-ctest))/cx1.ncols
lacc.t
mean(lacc)

To measure the accuracy of the predictions above, we can compute the probability that the classifier outputs the right label. We used this formula in class for the expected accuracy for logistic regression. The "dot arrow" operator takes dot product along rows:



In [7]:

    
val model = mm.model
val (nn1, nopts1) = GLM.LBFGSpredictor(model, atest, cx)









    









    Out[7]:





BIDMach.models.GLM$LearnLBFGSOptions@8eb62c4

Raw accuracy is not a good measure in most cases. When there are few positives (instances in the class vs. its complement), accuracy simply drives down false-positive rate at the expense of false-negative rate. In the worst case, the learner may always predict "no" and still achieve high accuracy.

ROC curves and ROC Area Under the Curve (AUC) are much better. Here we compute the ROC curves from the predictions above. We need:

scores - the predicted quality from the formula above.
good - 1 for positive instances, 0 for negative instances.
bad - complement of good.
npoints (100) - specifies the number of X-axis points for the ROC plot.

itest specifies which of the categories to plot for. We chose itest=6 because that category has one of the highest positive rates, and gives the most stable accuracy plots.



In [8]:

    
nn1.predict









    



corpus perplexity=1.970211
Predicting
 4.00%, ll=-0.88782, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
 8.00%, ll=-0.87371, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.19
12.00%, ll=-0.94405, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.14
16.00%, ll=-0.97745, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.19
20.00%, ll=-0.91692, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.24
24.00%, ll=-0.95906, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.19
28.00%, ll=-0.91178, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.22
32.00%, ll=-0.80047, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.26
36.00%, ll=-0.92667, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.22
40.00%, ll=-0.91742, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.24
44.00%, ll=-0.92421, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.26
48.00%, ll=-0.94550, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.23
51.00%, ll=-0.81677, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.25
56.00%, ll=-0.91331, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.27
60.00%, ll=-0.93091, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.24
64.00%, ll=-0.85525, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.26
68.00%, ll=-0.87134, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.27
72.00%, ll=-0.94563, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.25
76.00%, ll=-0.78031, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.26
80.00%, ll=-0.94823, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.27
83.00%, ll=-0.95096, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.29
88.00%, ll=-0.86657, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.26
92.00%, ll=-0.97759, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.28
96.00%, ll=-0.75901, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.29
100.00%, ll=-0.93326, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.27
Time=0.0090 secs, gflops=0.00

TODO 1: In the cell below, write an expression to derive the ROC Area under the curve (AUC) given the curve rr. rr gives the ROC curve y-coordinates at 100 evenly-spaced X-values from 0 to 1.0.



In [9]:

    
val cx1=cx*10
min(cx1, 1, cx1)                       // the first "traindata" argument is the input, the other is output
max(cx1, 0, cx1) 
val p=ctest *@cx1 +(1-ctest) *@(1-cx1)
mean(p,2)
val lacc = (cx1 ∙→ ctest + (1-cx1) ∙→ (1-ctest))/cx1.ncols
lacc.t
mean(lacc)

TODO 2: In the cell below, write the value of AUC returned by the expression above.



In [10]:

    
cx1









    









    Out[10]:





        1  0.97167  0.32712        1        1        0  0.19741        1...
        0        0        0        0        0        1        0        0...



In [11]:

    
saveFMat(dict+"moonresult.fmat.txt",cx)

Logistic Regression

Now lets train a logistic classifier on the same data. BIDMach has an umbrella classifier called GLM for Generalized Linear Model. GLM includes linear regression, logistic regression (with log accuracy or direct accuracy optimization), and SVM.

The learner function accepts these arguments:

traindata: the training data in the same format as for Naive Bayes
traincats: the training category labels
testdata: the test input data
predcats: a container for the predictions generated by the model
modeltype (GLM.logistic here): an integer that specifies the type of model (0=linear, 1=logistic log accuracy, 2=logistic accuracy, 3=SVM).

We'll construct the learner and then look at its options:



In [4]:

    
val dict = "/Users/Anna/workspace/BIDMach/data/wildlife/"
val aa = loadFMat(dict+"1.txt")
val c = loadFMat(dict+"c1.txt")
val b = loadFMat("/Users/Anna/workspace/BIDMach/data/wildlife/wltestall.txt")
val d = loadFMat("/Users/Anna/workspace/BIDMach/data/wildlife/wltestcatall.txt")
//val dict = "/Users/Anna/workspace/BIDMach_1.0.0-full-linux-x86_64/data/uci/"
//val aa = loadFMat(dict+"arabic.fmat.lz4")
//val c = loadFMat(dict+"arabic_cats.fmat.lz4")
val a = aa *10  
val atrain = a //a(?,(100->a.ncols))
val atest =  a //a(?,(0->100))
val ctrain = c //c(?,(100->a.ncols))
val ctest = c //c(?,(0->100))
//max(atrain, 0.001, atrain)                       // the first "traindata" argument is the input, the other is output
//max(atest, 0.001, atest)
atrain









    









    Out[4]:





          0          0          0          0          0          0...
          0          0          0          0          0          0...
    0.10306   0.098660   0.020530  0.0033600    0.45147          0...
          0   0.034910   0.048020    0.17789    0.13800    0.23622...
          0    0.53282   0.022220    0.66129   0.016920    0.50571...
    0.14581          0          0          0          0          0...
          0          0          0          0          0          0...
          0          0          0          0          0   0.058240...
         ..         ..         ..         ..         ..         ..

The most important options are:

lrate: the learning rate
batchSize: the minibatch size
npasses: the number of passes over the dataset

We'll use the following parameters for this training run.



In [10]:

    
val cx=zeros(2,ctest.ncols)
val (mm,mopts,nn,nopts)=GLM.LBFGSlearner(atrain,ctrain,atest,cx)
mopts.autoReset=false
mopts.useGPU=false
mopts.lrate=1
mopts.batchSize=2
mopts.dim=256
mopts.startBlock=0
mopts.npasses=10
mopts.updateAll=false
mopts.what









    



Option Name       Type          Value
===========       ====          =====
addConstFeat      boolean       false
autoReset         boolean       false
batchSize         int           2
dim               int           256
doubleScore       boolean       false
epsilon           float         1.0E-5
evalStep          int           11
featThreshold     Mat           null
featType          int           1
hashFeatures      boolean       false
initmys           float         0.1
initsumsq         float         1.0E-5
iweight           FMat          null
lim               float         0.0
links             IMat             4
   4

lrate             FMat          1
mask              FMat          null
npasses           int           10
nzPerColumn       int           0
pstep             float         0.01
putBack           int           -1
r1nmats           int           1
r2nmats           int           1
reg1weight        FMat          0.10000
reg2weight        FMat          0.10000
resFile           String        null
rmask             FMat          null
sample            float         1.0
sizeMargin        float         3.0
startBlock        int           0
targets           FMat          null
targmap           FMat          null
texp              FMat          0.50000
updateAll         boolean       false
useCache          boolean       true
useDouble         boolean       false
useGPU            boolean       false
vexp              FMat          0.50000
waitsteps         int           2



In [11]:

    
mm.train
nn.predict
val cx1=cx
min(cx1, 1, cx1)                       // the first "traindata" argument is the input, the other is output
max(cx1, 0, cx1) 
val p=ctest *@cx1 +(1-ctest) *@(1-cx1)
mean(p,2)









    



corpus perplexity=3886.147682
pass= 0
 6.00%, ll=-1.00000, gf=0.015, secs=0.0, GB=0.00, MB/s= 4.52
41.00%, ll=-44.57259, gf=96.317, secs=25.6, GB=0.00, MB/s= 0.01






    



java.lang.OutOfMemoryError: Java heap space
    scala.reflect.ManifestFactory$$anon$11.newArray(Manifest.scala:133)
    scala.reflect.ManifestFactory$$anon$11.newArray(Manifest.scala:131)
    BIDMat.DenseMat.<init>(DenseMat.scala:12)
    BIDMat.DenseMat$.newOrCheck(DenseMat.scala:1807)
    BIDMat.DenseMat$.newOrCheck(DenseMat.scala:1851)
    BIDMat.DenseMat$mcF$sp.ggMatOpStrictv$mcF$sp(DenseMat.scala:958)
    BIDMat.DenseMat$mcF$sp.ggMatOpv$mcF$sp(DenseMat.scala:951)
    BIDMat.FMat.ffMatOpv(FMat.scala:206)
    BIDMat.FPair.$plus(FMat.scala:1119)
    BIDMat.Mop_Plus$.op(Operators.scala:309)
    BIDMat.Mop$class.op(Operators.scala:44)
    BIDMat.Mop_Plus$.op(Operators.scala:307)
    BIDMat.FMat.$plus(FMat.scala:1060)
    BIDMach.updaters.LBFGS$$anonfun$update$1$$anonfun$apply$mcVI$sp$1.apply(LBFGS.scala:174)
    BIDMach.updaters.LBFGS$$anonfun$update$1$$anonfun$apply$mcVI$sp$1.apply(LBFGS.scala:162)
    scala.collection.immutable.Range.foreach(Range.scala:160)
    BIDMach.updaters.LBFGS$$anonfun$update$1.apply$mcVI$sp(LBFGS.scala:162)
    scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
    BIDMach.updaters.LBFGS.update(LBFGS.scala:130)
    BIDMach.Learner.retrain(Learner.scala:85)
    BIDMach.Learner.train(Learner.scala:49)

Since we have the accuracy scores for both Naive Bayes and Logistic regression, we can plot both of them on the same axes. Naive Bayes is red, Logistic regression is blue. The x-axis is the category number from 0 to 102. The y-axis is the absolute accuracy of the predictor for that category.



In [7]:

    
val lacc = (cx1 ∙→ ctest + (1-cx1) ∙→ (1-ctest))/cx1.ncols
lacc.t
mean(lacc)

TODO 3: With the full training set (700k training documents), Logistic Regression is noticeably more accurate than Naive Bayes in every category. What do you observe in the plot above? Why do you think this is?

Next we'll compute the ROC plot and ROC area (AUC) for Logistic regression for category itest.



In [40]:

    
saveFMat(dict+"wildliferesults.fmat.txt",cx)

We computed the ROC curve for Naive Bayes earlier, so now we can plot them on the same axes. Naive Bayes is once again in red, Logistic regression in blue.



In [51]:

    
cx1









    









    Out[51]:





  1.1398e-13  1.0409e-13  3.1763e-16  6.3486e-17  3.7628e-18  6.3368e-16...
  1.0488e-14  3.4730e-14  1.2205e-16  1.6546e-17  7.8444e-19  1.5367e-16...
  4.9068e-14  1.3641e-12  5.0059e-15  1.0316e-15  4.3107e-18  2.8967e-15...
  3.2656e-14  5.3654e-14  1.1086e-16  1.9517e-17  8.0842e-19  3.9087e-17...
  8.4080e-11  2.6030e-12  1.4020e-14  1.3571e-15  2.5275e-15  8.0566e-13...
  1.8410e-13  2.3721e-13  2.6570e-16  6.7162e-17  5.4204e-18  8.2176e-16...
  6.5161e-11  1.3945e-10  2.1820e-12  7.0878e-14  1.5250e-15  2.8615e-12...
  1.6085e-13  1.9523e-13  3.2423e-16  5.6627e-17  7.7785e-18  1.7874e-15...
          ..          ..          ..          ..          ..          ..

TODO 4: In the cell below, compute and plot lift curves from the ROC curves for Naive Bayes and Logistic regression. The lift curves should show the ratio of ROC y-values over a unit slope diagonal line (Y=X). The X-values should be the same as for the ROC plots, except that X=0 will be omitted since the lift will be undefined.



In [2]:

    
val dict = "/Users/Anna/workspace/BIDMach_1.0.0-full-linux-x86_64/data/"
//val a = loadFMat(dict+"arabic.fmat.lz4")
//val c = loadFMat(dict+"arabic_cats.fmat.lz4")
val aa = loadFMat(dict+"a.txt")
val c = loadFMat(dict+"alabel.txt")
val a = aa + 0.5 
val atrain =a(?,(100->a.ncols))
val atest =a(?,(0->100))
val ctrain =c(?,(100->a.ncols))
val ctest =c(?,(0->100))
max(atrain, 0.001, atrain)                       // the first "traindata" argument is the input, the other is output
max(atest, 0.001, atest)
atest









    









    Out[2]:





   0.075172    0.42627    0.64251    0.36696    0.19400    0.58963...
    0.64074    0.42242    0.41518    0.87276    0.77227  0.0010000...

TODO 5: Experiment with different values for learning rate and batchSize to get the best performance for absolute accuracy and ROC area on category 6. Write your optimal values below:



In [41]:

    
val cx=zeros(ctest.nrows,ctest.ncols)
val (mm,mopts,nn,nopts)=GLM.SVMlearner(atrain,ctrain,atest,cx)
mopts.autoReset=false
mopts.useGPU=false
mopts.lrate=0.1
mopts.batchSize=2
mopts.dim=256
mopts.startBlock=0
mopts.npasses=10
mopts.updateAll=false
mopts.what









    



Option Name       Type          Value
===========       ====          =====
addConstFeat      boolean       false
autoReset         boolean       false
batchSize         int           2
dim               int           256
doubleScore       boolean       false
epsilon           float         1.0E-5
evalStep          int           11
featThreshold     Mat           null
featType          int           1
hashFeatures      boolean       false
initsumsq         float         1.0E-5
iweight           FMat          null
lim               float         0.0
links             IMat             3
   3

lrate             FMat          0.10000
mask              FMat          null
npasses           int           10
nzPerColumn       int           0
pstep             float         0.01
putBack           int           -1
r1nmats           int           1
r2nmats           int           1
reg1weight        FMat          1.0000e-07
reg2weight        FMat          1
resFile           String        null
rmask             FMat          null
sample            float         1.0
sizeMargin        float         3.0
startBlock        int           0
targets           FMat          null
targmap           FMat          null
texp              FMat          0.50000
updateAll         boolean       false
useCache          boolean       true
useDouble         boolean       false
useGPU            boolean       false
vexp              FMat          0.50000
waitsteps         int           2



In [42]:

    
mm.train
nn.predict









    



corpus perplexity=992.808970
pass= 0
 6.00%, ll=-1.00000, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
41.00%, ll=-0.28270, gf=0.079, secs=0.0, GB=0.00, MB/s=39.00
77.00%, ll=-0.03527, gf=0.081, secs=0.0, GB=0.00, MB/s=38.40
100.00%, ll=-0.20491, gf=0.083, secs=0.0, GB=0.00, MB/s=39.16
pass= 1
 6.00%, ll=-1.20125, gf=0.078, secs=0.0, GB=0.00, MB/s=37.71
41.00%, ll=-0.30951, gf=0.083, secs=0.0, GB=0.00, MB/s=39.11
77.00%, ll=-0.24529, gf=0.085, secs=0.0, GB=0.00, MB/s=40.00
100.00%, ll=-0.13970, gf=0.086, secs=0.0, GB=0.00, MB/s=40.22
pass= 2
 6.00%, ll=-1.47287, gf=0.085, secs=0.0, GB=0.00, MB/s=40.42
41.00%, ll=-0.22679, gf=0.087, secs=0.0, GB=0.00, MB/s=40.91
77.00%, ll=-0.39799, gf=0.090, secs=0.0, GB=0.00, MB/s=42.12
100.00%, ll=-0.17308, gf=0.090, secs=0.1, GB=0.00, MB/s=42.11
pass= 3
 6.00%, ll=-1.47907, gf=0.090, secs=0.1, GB=0.00, MB/s=42.22
41.00%, ll=-0.30906, gf=0.092, secs=0.1, GB=0.00, MB/s=43.12
77.00%, ll=-0.49180, gf=0.093, secs=0.1, GB=0.00, MB/s=43.20
100.00%, ll=-0.23331, gf=0.094, secs=0.1, GB=0.00, MB/s=43.76
pass= 4
 6.00%, ll=-1.46579, gf=0.093, secs=0.1, GB=0.00, MB/s=43.83
41.00%, ll=-0.37264, gf=0.094, secs=0.1, GB=0.00, MB/s=43.84
77.00%, ll=-0.55793, gf=0.094, secs=0.1, GB=0.00, MB/s=43.85
100.00%, ll=-0.28842, gf=0.095, secs=0.1, GB=0.00, MB/s=44.29
pass= 5
 6.00%, ll=-1.44678, gf=0.095, secs=0.1, GB=0.00, MB/s=44.33
41.00%, ll=-0.42261, gf=0.095, secs=0.1, GB=0.00, MB/s=44.31
77.00%, ll=-0.60707, gf=0.095, secs=0.1, GB=0.00, MB/s=44.29
100.00%, ll=-0.33395, gf=0.096, secs=0.1, GB=0.00, MB/s=44.64
pass= 6
 6.00%, ll=-1.42705, gf=0.095, secs=0.1, GB=0.00, MB/s=44.67
41.00%, ll=-0.46288, gf=0.097, secs=0.1, GB=0.00, MB/s=45.06
77.00%, ll=-0.64504, gf=0.097, secs=0.1, GB=0.01, MB/s=45.40
100.00%, ll=-0.37222, gf=0.098, secs=0.1, GB=0.01, MB/s=45.68
pass= 7
 6.00%, ll=-1.40822, gf=0.098, secs=0.1, GB=0.01, MB/s=45.70
41.00%, ll=-0.49608, gf=0.099, secs=0.1, GB=0.01, MB/s=46.39
77.00%, ll=-0.67528, gf=0.099, secs=0.1, GB=0.01, MB/s=46.27
100.00%, ll=-0.40490, gf=0.100, secs=0.1, GB=0.01, MB/s=46.50
pass= 8
 6.00%, ll=-1.39079, gf=0.100, secs=0.1, GB=0.01, MB/s=46.51
41.00%, ll=-0.52395, gf=0.100, secs=0.1, GB=0.01, MB/s=46.75
77.00%, ll=-0.69995, gf=0.101, secs=0.1, GB=0.01, MB/s=46.96
100.00%, ll=-0.43319, gf=0.101, secs=0.1, GB=0.01, MB/s=47.15
pass= 9
 6.00%, ll=-1.37481, gf=0.101, secs=0.1, GB=0.01, MB/s=47.16
41.00%, ll=-0.54772, gf=0.102, secs=0.1, GB=0.01, MB/s=47.67
77.00%, ll=-0.72047, gf=0.103, secs=0.2, GB=0.01, MB/s=47.84
100.00%, ll=-0.45795, gf=0.103, secs=0.2, GB=0.01, MB/s=48.00
Time=0.1550 secs, gflops=0.10
corpus perplexity=985.317557
Predicting
 4.00%, ll=-1.00000, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
 9.00%, ll=-1.00000, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
14.00%, ll=-1.00000, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
19.00%, ll=-1.00000, gf=Infinity, secs=0.0, GB=0.00, MB/s=Infinity
24.00%, ll=-1.00000, gf=0.060, secs=0.0, GB=0.00, MB/s=180.00
29.00%, ll=-1.00000, gf=0.073, secs=0.0, GB=0.00, MB/s=216.00
33.00%, ll=-1.00000, gf=0.085, secs=0.0, GB=0.00, MB/s=252.00
38.00%, ll=-1.00000, gf=0.097, secs=0.0, GB=0.00, MB/s=288.00
43.00%, ll=-1.00000, gf=0.109, secs=0.0, GB=0.00, MB/s=324.00
48.00%, ll=-1.00000, gf=0.121, secs=0.0, GB=0.00, MB/s=360.00
53.00%, ll=-1.00000, gf=0.066, secs=0.0, GB=0.00, MB/s=198.00
58.00%, ll=-1.00000, gf=0.073, secs=0.0, GB=0.00, MB/s=216.00
62.00%, ll=-1.00000, gf=0.079, secs=0.0, GB=0.00, MB/s=234.00
67.00%, ll=-1.00000, gf=0.085, secs=0.0, GB=0.00, MB/s=251.99
72.00%, ll=-1.00000, gf=0.091, secs=0.0, GB=0.00, MB/s=269.99
77.00%, ll=-1.00000, gf=0.097, secs=0.0, GB=0.00, MB/s=287.99
82.00%, ll=-1.00000, gf=0.069, secs=0.0, GB=0.00, MB/s=204.00
87.00%, ll=-1.00000, gf=0.073, secs=0.0, GB=0.00, MB/s=216.00
91.00%, ll=-1.00000, gf=0.077, secs=0.0, GB=0.00, MB/s=228.00
96.00%, ll=-1.00000, gf=0.081, secs=0.0, GB=0.00, MB/s=240.00
100.00%, ll=-1.00000, gf=0.083, secs=0.0, GB=0.00, MB/s=248.00
Time=0.0030 secs, gflops=0.08



In [ ]:

    
val cx1=cx*5
min(cx1, 1, cx1)                       // the first "traindata" argument is the input, the other is output
max(cx1, 0, cx1) 
val p=ctest *@cx1 +(1-ctest) *@(1-cx1)
mean(p,2)



In [6]:

    
cx1









    









    Out[6]:





         1   0.48584   0.16356         1         1         0  0.098707...
         0         0         0         0         0   0.82548         0...



In [7]:

    
val lacc = (cx1 ∙→ ctest + (1-cx1) ∙→ (1-ctest))/cx1.ncols
lacc.t
mean(lacc)



In [8]:

    
val model = mm.model
val (nn1, nopts1) = GLM.LBFGSpredictor(model, atest, cx)









    









    Out[8]:





BIDMach.models.GLM$LearnLBFGSOptions@3fde0ffe



In [9]:

    
nn1.predict









    



corpus perplexity=1.970211
Predicting
 4.00%, ll=-0.88782, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.10
 8.00%, ll=-0.87371, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.19
12.00%, ll=-0.94405, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.29
16.00%, ll=-0.97745, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.38
20.00%, ll=-0.91692, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.24
24.00%, ll=-0.95906, gf=0.000, secs=0.0, GB=0.00, MB/s= 0.29
28.00%, ll=-0.91178, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.34
32.00%, ll=-0.80047, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.38
36.00%, ll=-0.92667, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.43
40.00%, ll=-0.91742, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.48
44.00%, ll=-0.92421, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.53
48.00%, ll=-0.94550, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.38
51.00%, ll=-0.81677, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.42
56.00%, ll=-0.91331, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.45
60.00%, ll=-0.93091, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.48
64.00%, ll=-0.85525, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.51
68.00%, ll=-0.87134, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.54
72.00%, ll=-0.94563, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.58
76.00%, ll=-0.78031, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.46
80.00%, ll=-0.94823, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.48
83.00%, ll=-0.95096, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.50
88.00%, ll=-0.86657, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.53
92.00%, ll=-0.97759, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.55
96.00%, ll=-0.75901, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.58
100.00%, ll=-0.93326, gf=0.001, secs=0.0, GB=0.00, MB/s= 0.60
Time=0.0050 secs, gflops=0.00



In [10]:

    
val cx1=cx*10
min(cx1, 1, cx1)                       // the first "traindata" argument is the input, the other is output
max(cx1, 0, cx1) 
val p=ctest *@cx1 +(1-ctest) *@(1-cx1)
mean(p,2)
val lacc = (cx1 ∙→ ctest + (1-cx1) ∙→ (1-ctest))/cx1.ncols
lacc.t
mean(lacc)



In [11]:

    
cx1









    









    Out[11]:





        1  0.97167  0.32712        1        1        0  0.19741        1...
        0        0        0        0        0        1        0        0...



In [23]:

    
saveFMat(dict+"moonresult.fmat.txt",cx)



In [ ]:



In [19]: