notebook.community

Edit and run

Goal

Accuracy as a function of isotope incorporation & sampling depth

acc ~ depth + incorp

Variable parameters:

  • atom % isotope incorporation
    • 0, 15, 25, 50, 100
  • sampling depth
    • 1e2, 1e3, 1e4, 1e5
  • n-reps (stocastic: taxon abundances & which incorporate)
    • 10

Setting paths





In [1]:



    


# paths
import os

workDir = '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/'
buildDir = os.path.join(workDir, 'atomIncorp_sampDepth')
R_dir = '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/SIPSimR/scripts/'

fragFile = '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/ampFrags_kde.pkl'
genome_index = '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/genome_index.txt'

Init





In [2]:



    


import glob
import itertools
import nestly



    











In [3]:



    


%load_ext pushmsg



    











In [4]:



    


if not os.path.isdir(buildDir):
    os.makedirs(buildDir)
%cd $buildDir



    


















    






/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth

BD min/max





In [5]:



    


## min G+C cutoff
min_GC = 13.5
## max G+C cutoff
max_GC = 80
## max G+C shift
max_13C_shift_in_BD = 0.036


min_BD = min_GC/100.0 * 0.098 + 1.66    
max_BD = max_GC/100.0 * 0.098 + 1.66    

max_BD = max_BD + max_13C_shift_in_BD

print('Min BD: {}'.format(min_BD))
print('Max BD: {}'.format(max_BD))



    


















    






Min BD: 1.67323
Max BD: 1.7744

Nestly





In [6]:



    


# making an experimental design file for qSIP
x = range(1,7)
y = ['control', 'treatment']

expDesignFile = os.path.join(buildDir, 'qSIP_exp_design.txt')
with open(expDesignFile, 'wb') as outFH:
    for i,z in itertools.izip(x,itertools.cycle(y)):
        line = '\t'.join([str(i),z])
        outFH.write(line + '\n')

!head $expDesignFile



    


















    






1	control
2	treatment
3	control
4	treatment
5	control
6	treatment

Nestly params





In [7]:



    


# building tree structure
nest = nestly.Nest()

# varying params: TEST
#nest.add('percIncorp', [0, 100])
#nest.add('subsample_size', [100, 10000])
#nest.add('rep', [1])

# varying params
nest.add('percIncorp', [0, 15, 25, 50, 100])
nest.add('subsample_size', [100, 1000, 10000, 100000])
nest.add('rep', range(1,11))


## set params
nest.add('percTaxa', [10], create_dir=False)
nest.add('abs', ['1e9'], create_dir=False)
#nest.add('abs', ['1e7'], create_dir=False)   # TESTING
nest.add('np', [4], create_dir=False)
nest.add('Monte_rep', [100000], create_dir=False)
nest.add('subsample_dist', ['uniform'], create_dir=False)
nest.add('min_BD', [min_BD], create_dir=False)
nest.add('max_BD', [max_BD], create_dir=False)
nest.add('DBL_scaling', [0.5], create_dir=False)
nest.add('bandwidth', [0.8], create_dir=False)
nest.add('heavy_BD_min', [1.71], create_dir=False)
nest.add('heavy_BD_max', [1.75], create_dir=False)
nest.add('topTaxaToPlot', [100], create_dir=False)
nest.add('padj', [0.1], create_dir=False)
nest.add('log2', [0.25], create_dir=False)

### input/output files
nest.add('buildDir', [buildDir], create_dir=False)
nest.add('R_dir', [R_dir], create_dir=False)
nest.add('genome_index', [genome_index], create_dir=False)
nest.add('fragFile', [fragFile], create_dir=False)
nest.add('exp_design', [expDesignFile], create_dir=False)


# building directory tree
nest.build(buildDir)

# bash file to run
bashFile = os.path.join(buildDir, 'SIPSimRun.sh')

Experimental design





In [9]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_expDesign.sh'
bashFileTmp



    


















    
Out[9]:








'/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_expDesign.sh'

















In [10]:



    


%%writefile $bashFileTmp
#!/bin/bash
source activate SIPSim
# OPENBLAS threads 
export OMP_NUM_THREADS=1

echo '#-- Experimental design --#'

echo '# Making an isotope incorporation config file'
echo '## 3 replicate gradients for control & treatment'
SIPSim incorp_config_example \
  --percIncorpUnif {percIncorp} \
  --n_reps 3 \
  > incorp.config

echo '# Selecting incorporator taxa'
echo '## This is to make the gradient replicates consistent (qSIP finds mean among replicates)'
SIPSim KDE_select_taxa \
    -p {percTaxa} \
    {fragFile} \
    > incorporators.txt

echo '# Creating a community file (3 replicate control, 3 replicate treatment)'
SIPSim communities \
    --config incorp.config \
    {genome_index} \
    > comm.txt    

echo '# simulating gradient fractions'
SIPSim gradient_fractions \
    --BD_min {min_BD} \
    --BD_max {max_BD} \
    comm.txt \
    > fracs.txt



    


















    






Writing /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_expDesign.sh

















In [11]:



    


!chmod 755 $bashFileTmp



    











In [12]:



    


%%bash -s $workDir $bashFileTmp $buildDir
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate py2_ley0.4
# change to working dir
cd $1
# run job 
nestrun --template-file $2 -d $3 --log-file exp_design.log -j 20



    


















    






2017-07-30 15:16:48,135 * INFO * Template: ./SIPSimRun_expDesign.sh
2017-07-30 15:16:48,140 * INFO * [44847] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/1
2017-07-30 15:16:48,145 * INFO * [44850] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/2
2017-07-30 15:16:48,149 * INFO * [44858] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/3
2017-07-30 15:16:48,156 * INFO * [44866] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/4
2017-07-30 15:16:48,160 * INFO * [44874] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/5
2017-07-30 15:16:48,167 * INFO * [44885] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/6
2017-07-30 15:16:48,171 * INFO * [44890] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/7
2017-07-30 15:16:48,177 * INFO * [44900] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/8
2017-07-30 15:16:48,184 * INFO * [44908] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/9
2017-07-30 15:16:48,189 * INFO * [44916] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/10
2017-07-30 15:16:48,195 * INFO * [44924] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/1
2017-07-30 15:16:48,201 * INFO * [44931] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/2
2017-07-30 15:16:48,207 * INFO * [44940] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/3
2017-07-30 15:16:48,214 * INFO * [44946] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/4
2017-07-30 15:16:48,221 * INFO * [44958] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/5
2017-07-30 15:16:48,226 * INFO * [44964] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/6
2017-07-30 15:16:48,234 * INFO * [44972] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/7
2017-07-30 15:16:48,241 * INFO * [44980] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/8
2017-07-30 15:16:48,246 * INFO * [44986] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/9
2017-07-30 15:16:48,254 * INFO * [44994] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/10
2017-07-30 15:17:01,916 * INFO * [44916] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/10 Finished with 0
2017-07-30 15:17:01,926 * INFO * [45628] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/1
2017-07-30 15:17:01,954 * INFO * [44964] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/6 Finished with 0
2017-07-30 15:17:01,963 * INFO * [45639] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/2
2017-07-30 15:17:01,998 * INFO * [44866] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/4 Finished with 0
2017-07-30 15:17:02,021 * INFO * [45647] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/3
2017-07-30 15:17:02,021 * INFO * [44890] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/7 Finished with 0
2017-07-30 15:17:02,025 * INFO * [45650] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/4
2017-07-30 15:17:02,026 * INFO * [44900] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/8 Finished with 0
2017-07-30 15:17:02,032 * INFO * [45660] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/5
2017-07-30 15:17:02,032 * INFO * [44885] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/6 Finished with 0
2017-07-30 15:17:02,043 * INFO * [45671] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/6
2017-07-30 15:17:02,044 * INFO * [44931] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/2 Finished with 0
2017-07-30 15:17:02,050 * INFO * [45677] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/7
2017-07-30 15:17:02,066 * INFO * [44858] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/3 Finished with 0
2017-07-30 15:17:02,073 * INFO * [45687] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/8
2017-07-30 15:17:02,074 * INFO * [44994] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/10 Finished with 0
2017-07-30 15:17:02,085 * INFO * [45691] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/9
2017-07-30 15:17:02,087 * INFO * [44850] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/2 Finished with 0
2017-07-30 15:17:02,093 * INFO * [45698] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/10
2017-07-30 15:17:02,095 * INFO * [44924] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/1 Finished with 0
2017-07-30 15:17:02,102 * INFO * [45710] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/1
2017-07-30 15:17:02,137 * INFO * [44940] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/3 Finished with 0
2017-07-30 15:17:02,144 * INFO * [45719] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/2
2017-07-30 15:17:02,176 * INFO * [44908] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/9 Finished with 0
2017-07-30 15:17:02,183 * INFO * [45727] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/3
2017-07-30 15:17:02,183 * INFO * [44980] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/8 Finished with 0
2017-07-30 15:17:02,189 * INFO * [45730] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/4
2017-07-30 15:17:02,189 * INFO * [44946] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/4 Finished with 0
2017-07-30 15:17:02,198 * INFO * [45738] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/5
2017-07-30 15:17:02,198 * INFO * [44847] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/1 Finished with 0
2017-07-30 15:17:02,203 * INFO * [45746] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/6
2017-07-30 15:17:02,203 * INFO * [44958] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/5 Finished with 0
2017-07-30 15:17:02,208 * INFO * [45755] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/7
2017-07-30 15:17:02,212 * INFO * [44972] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/7 Finished with 0
2017-07-30 15:17:02,219 * INFO * [45767] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/8
2017-07-30 15:17:02,220 * INFO * [44874] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/5 Finished with 0
2017-07-30 15:17:02,228 * INFO * [45774] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/9
2017-07-30 15:17:02,243 * INFO * [44986] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/9 Finished with 0
2017-07-30 15:17:02,251 * INFO * [45783] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/10
2017-07-30 15:17:12,319 * INFO * [45639] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/2 Finished with 0
2017-07-30 15:17:12,324 * INFO * [46417] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/1
2017-07-30 15:17:12,396 * INFO * [45671] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/6 Finished with 0
2017-07-30 15:17:12,416 * INFO * [46425] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/2
2017-07-30 15:17:12,428 * INFO * [45628] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/1 Finished with 0
2017-07-30 15:17:12,435 * INFO * [46433] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/3
2017-07-30 15:17:12,465 * INFO * [45755] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/7 Finished with 0
2017-07-30 15:17:12,472 * INFO * [46441] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/4
2017-07-30 15:17:12,472 * INFO * [45660] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/5 Finished with 0
2017-07-30 15:17:12,478 * INFO * [46444] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/5
2017-07-30 15:17:12,478 * INFO * [45698] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/10 Finished with 0
2017-07-30 15:17:12,491 * INFO * [46453] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/6
2017-07-30 15:17:12,491 * INFO * [45677] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/7 Finished with 0
2017-07-30 15:17:12,498 * INFO * [46462] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/7
2017-07-30 15:17:12,498 * INFO * [45687] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/8 Finished with 0
2017-07-30 15:17:12,504 * INFO * [46468] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/8
2017-07-30 15:17:12,505 * INFO * [45650] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/4 Finished with 0
2017-07-30 15:17:12,510 * INFO * [46477] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/9
2017-07-30 15:17:12,510 * INFO * [45719] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/2 Finished with 0
2017-07-30 15:17:12,516 * INFO * [46484] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/10
2017-07-30 15:17:12,516 * INFO * [45691] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/9 Finished with 0
2017-07-30 15:17:12,522 * INFO * [46494] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/1
2017-07-30 15:17:12,523 * INFO * [45647] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/3 Finished with 0
2017-07-30 15:17:12,531 * INFO * [46504] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/2
2017-07-30 15:17:12,537 * INFO * [45738] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/5 Finished with 0
2017-07-30 15:17:12,543 * INFO * [46513] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/3
2017-07-30 15:17:12,574 * INFO * [45774] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/9 Finished with 0
2017-07-30 15:17:12,581 * INFO * [46521] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/4
2017-07-30 15:17:12,582 * INFO * [45710] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/1 Finished with 0
2017-07-30 15:17:12,587 * INFO * [46524] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/5
2017-07-30 15:17:12,629 * INFO * [45767] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/8 Finished with 0
2017-07-30 15:17:12,636 * INFO * [46537] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/6
2017-07-30 15:17:12,640 * INFO * [45783] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/10 Finished with 0
2017-07-30 15:17:12,650 * INFO * [46545] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/7
2017-07-30 15:17:12,650 * INFO * [45727] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/3 Finished with 0
2017-07-30 15:17:12,659 * INFO * [46552] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/8
2017-07-30 15:17:12,659 * INFO * [45730] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/4 Finished with 0
2017-07-30 15:17:12,664 * INFO * [46558] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/9
2017-07-30 15:17:12,664 * INFO * [45746] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/6 Finished with 0
2017-07-30 15:17:12,674 * INFO * [46569] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/10
2017-07-30 15:17:22,543 * INFO * [46433] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/3 Finished with 0
2017-07-30 15:17:22,549 * INFO * [47198] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/1
2017-07-30 15:17:22,709 * INFO * [46462] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/7 Finished with 0
2017-07-30 15:17:22,714 * INFO * [47206] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/2
2017-07-30 15:17:22,714 * INFO * [46444] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/5 Finished with 0
2017-07-30 15:17:22,722 * INFO * [47211] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/3
2017-07-30 15:17:22,723 * INFO * [46484] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/10 Finished with 0
2017-07-30 15:17:22,729 * INFO * [47219] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/4
2017-07-30 15:17:22,746 * INFO * [46468] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/8 Finished with 0
2017-07-30 15:17:22,764 * INFO * [47230] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/5
2017-07-30 15:17:22,765 * INFO * [46494] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/1 Finished with 0
2017-07-30 15:17:22,769 * INFO * [47233] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/6
2017-07-30 15:17:22,777 * INFO * [46417] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/1 Finished with 0
2017-07-30 15:17:22,782 * INFO * [47246] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/7
2017-07-30 15:17:22,796 * INFO * [46477] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/9 Finished with 0
2017-07-30 15:17:22,809 * INFO * [47254] Started ./SIPSimRun_expDesign.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/8
2017-07-30 15:17:22,809 * INFO * [46441] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/4 Finished with 0
2017-07-









    






limit_output extension: Maximum message size of 20000 exceeded with 67908 characters
















In [13]:



    


%pushmsg "exp_design complete: $buildDir"

SIPSim pipeline





In [14]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_SIPSim-pipeline.sh'
bashFileTmp



    


















    
Out[14]:








'/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_SIPSim-pipeline.sh'

















In [15]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim
# OPENBLAS threads 
export OMP_NUM_THREADS=1


echo '#-- SIPSim pipeline --#'    
    
echo '# Adding diffusion'    
SIPSim diffusion \
    -n {Monte_rep} \
    --bw {bandwidth} \
    --np {np} \
    {fragFile} \
    > ampFrags_KDE_dif.pkl    

echo '# Adding DBL contamination; abundance-weighted smearing'
SIPSim DBL \
    -n {Monte_rep} \
    --comm comm.txt \
    --commx {DBL_scaling} \
    --np {np} \
    ampFrags_KDE_dif.pkl \
    > ampFrags_KDE_dif_DBL.pkl 

echo '# Adding isotope incorporation to BD distribution'
SIPSim isotope_incorp \
    -n {Monte_rep} \
    --comm comm.txt \
    --taxa incorporators.txt \
    --np {np} \
    ampFrags_KDE_dif_DBL.pkl \
    incorp.config \
    > ampFrags_KDE_dif_DBL_inc.pkl

echo '# Simulating an OTU table'
SIPSim OTU_table \
    --abs {abs} \
    --np {np} \
    ampFrags_KDE_dif_DBL_inc.pkl \
    comm.txt \
    fracs.txt \
    > OTU_abs{abs}.txt
    
echo '# Simulating PCR'
SIPSim OTU_PCR \
    OTU_abs{abs}.txt \
    > OTU_abs{abs}_PCR.txt    
    
echo '# Subsampling from the OTU table (simulating sequencing of the DNA pool)'
SIPSim OTU_subsample \
    --dist {subsample_dist} \
    --dist_params low:{subsample_size},high:{subsample_size} \
    OTU_abs{abs}_PCR.txt \
    > OTU_abs{abs}_PCR_sub.txt
        
echo '# Making a wide-formatted table'
SIPSim OTU_wide_long -w \
    OTU_abs{abs}_PCR_sub.txt \
    > OTU_abs{abs}_PCR_sub_w.txt
    
echo '# Making metadata (phyloseq: sample_data)'
SIPSim OTU_sample_data \
    OTU_abs{abs}_PCR_sub.txt \
    > OTU_abs{abs}_PCR_sub_meta.txt
       

#-- removing large intermediate files --#
rm -f ampFrags_KDE_dif.pkl
rm -f ampFrags_KDE_dif_DBL.pkl
rm -f ampFrags_KDE_dif_DBL_inc.pkl



    


















    






Writing /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_SIPSim-pipeline.sh

















In [16]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file SIPSim_pipeline.log -j 6 --stop-on-error



    











In [ ]:



    


%pushmsg "SIPSim pipeline complete: $buildDir"

Summary of simulated data





In [ ]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_SIPSim-summary.sh'
bashFileTmp



    











In [ ]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim 

echo "# Plotting taxon abundances"

# plotting 'raw' taxon abundances
Rscript {R_dir}OTU_taxonAbund.R \
    OTU_abs{abs}.txt \
    -r {topTaxaToPlot} \
    -o OTU_abs{abs}

# plotting 'sequenced' taxon abundances
Rscript {R_dir}OTU_taxonAbund.R \
    OTU_abs{abs}_PCR_sub.txt \
    -r {topTaxaToPlot} \
    -o OTU_abs{abs}_PCR_sub



    











In [ ]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file SIPSim_summary.log -j 20

HR-SIP





In [ ]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_HRSIP.sh'
bashFileTmp



    











In [ ]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim 

# phyloseq
## making phyloseq object from OTU table
Rscript {R_dir}phyloseq_make.R \
    OTU_abs{abs}_PCR_sub_w.txt \
    -s OTU_abs{abs}_PCR_sub_meta.txt \
    > OTU_abs{abs}_PCR_sub.physeq

## filtering phyloseq object to just 'heavy' fractions
Rscript {R_dir}phyloseq_edit.R \
    OTU_abs{abs}_PCR_sub.physeq \
    --BD_min {heavy_BD_min} \
    --BD_max {heavy_BD_max} \
    > OTU_abs{abs}_PCR_sub_filt.physeq

## making ordination
Rscript {R_dir}phyloseq_ordination.R \
    OTU_abs{abs}_PCR_sub_filt.physeq \
    OTU_abs{abs}_PCR_sub_filt_bray-NMDS.pdf

# DESeq2
Rscript {R_dir}phyloseq_DESeq2.R \
    --log2 {log2} \
    --hypo greater \
    --cont 1,3,5 \
    --treat 2,4,6 \
    OTU_abs{abs}_PCR_sub_filt.physeq \
    > OTU_abs{abs}_PCR_sub_filt_DESeq2



    











In [ ]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file HR-SIP.log -j 20



    











In [ ]:



    


%pushmsg "HR-SIP complete: $buildDir"

MW-HR-SIP





In [60]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_MWHRSIP.sh'
bashFileTmp



    


















    
Out[60]:








'/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_MWHRSIP.sh'

















In [61]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim 

## HR SIP pipeline
Rscript {R_dir}phyloseq_DESeq2.R \
    --log2 {log2} \
    --hypo greater \
    --cont 1,3,5 \
    --treat 2,4,6 \
    --occur_all 0.0,0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5 \
    -w 1.70-1.73,1.72-1.75,1.74-1.77 \
    --all OTU_abs1e9_PCR_sub_MW-all.txt \
    OTU_abs{abs}_PCR_sub.physeq \
    > OTU_abs{abs}_PCR_sub_filt_MW_DESeq2



    


















    






Overwriting /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_MWHRSIP.sh

















In [62]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file MW-HR-SIP.log -j 20



    











In [ ]:



    


%pushmsg "MW-HR-SIP complete: $buildDir"

q-SIP





In [ ]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_qSIP.sh'
bashFileTmp



    











In [ ]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim 
# OPENBLAS threads 
export OMP_NUM_THREADS=1

# qSIP
SIPSim qSIP \
    OTU_abs{abs}.txt \
    OTU_abs{abs}_PCR_sub.txt \
    > OTU_abs{abs}_PCR_sub_qSIP.txt
        

# qSIP: atom excess
SIPSim qSIP_atom_excess \
    --np {np} \
    OTU_abs{abs}_PCR_sub_qSIP.txt \
    {exp_design} \
    > OTU_abs{abs}_PCR_sub_qSIP_atom.txt



    











In [ ]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file qSIP.log -j 6



    











In [ ]:



    


%pushmsg "q-SIP complete: $buildDir"

delta-BD





In [ ]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_dBD.sh'
bashFileTmp



    











In [ ]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim 
# OPENBLAS threads 
export OMP_NUM_THREADS=1

#deltaBD 
SIPSim deltaBD \
    OTU_abs{abs}_PCR_sub.txt \
    {exp_design} \
    > OTU_abs{abs}_PCR_sub_dBD.txt



    











In [ ]:



    


!chmod 755 $bashFileTmp



    











In [ ]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file deltaBD.log -j 20



    











In [ ]:



    


%pushmsg "deltaBD complete: $buildDir"

Making confusion matrices





In [72]:



    


bashFileTmp = os.path.splitext(bashFile)[0] + '_cMtx.sh'
bashFileTmp



    


















    
Out[72]:








'/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_cMtx.sh'

















In [73]:



    


%%writefile $bashFileTmp
#!/bin/bash
# offset job start to prevent conda activate errors
sleep $[ ( $RANDOM % 10 )  + 1 ]s
source activate SIPSim

# HR-SIP
Rscript {R_dir}DESeq2_confuseMtx.R \
    --libs 2,4,6 \
    --padj {padj} \
    BD-shift_stats.txt \
    OTU_abs{abs}_PCR_sub_filt_DESeq2

# HR-SIP multiple 'heavy' BD windows
Rscript {R_dir}DESeq2_confuseMtx.R \
    --libs 2,4,6 \
    --padj {padj} \
    -o DESeq2_multi-cMtx \
    BD-shift_stats.txt \
    OTU_abs{abs}_PCR_sub_filt_MW_DESeq2
    
# qSIP    
Rscript {R_dir}qSIP_confuseMtx.R \
    --libs 2,4,6 \
    BD-shift_stats.txt \
    OTU_abs{abs}_PCR_sub_qSIP_atom.txt

# heavy-SIP    
Rscript {R_dir}heavy_confuseMtx.R \
    --treat 2,4,6 \
    --con 1,3,5 \
    --method 1 \
    BD-shift_stats.txt \
    OTU_abs{abs}_PCR_sub.txt



    


















    






Overwriting /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/SIPSimRun_cMtx.sh

















In [74]:



    


!chmod 755 $bashFileTmp



    











In [75]:



    


%%bash -s $workDir $bashFileTmp $buildDir
source activate py2_ley0.4
cd $1

nestrun --template-file $2 -d $3 --log-file cMtx.log -j 20



    


















    






2017-08-11 11:09:32,092 * INFO * Template: ./SIPSimRun_cMtx.sh
2017-08-11 11:09:32,096 * INFO * [4357] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/1
2017-08-11 11:09:32,100 * INFO * [4359] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/2
2017-08-11 11:09:32,104 * INFO * [4361] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/3
2017-08-11 11:09:32,107 * INFO * [4363] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/4
2017-08-11 11:09:32,110 * INFO * [4365] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/5
2017-08-11 11:09:32,114 * INFO * [4367] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/6
2017-08-11 11:09:32,118 * INFO * [4369] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/7
2017-08-11 11:09:32,122 * INFO * [4373] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/8
2017-08-11 11:09:32,125 * INFO * [4375] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/9
2017-08-11 11:09:32,129 * INFO * [4377] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/10
2017-08-11 11:09:32,133 * INFO * [4379] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/1
2017-08-11 11:09:32,136 * INFO * [4381] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/2
2017-08-11 11:09:32,139 * INFO * [4383] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/3
2017-08-11 11:09:32,143 * INFO * [4385] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/4
2017-08-11 11:09:32,146 * INFO * [4389] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/5
2017-08-11 11:09:32,150 * INFO * [4391] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/6
2017-08-11 11:09:32,154 * INFO * [4393] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/7
2017-08-11 11:09:32,157 * INFO * [4395] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/8
2017-08-11 11:09:32,160 * INFO * [4397] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/9
2017-08-11 11:09:32,164 * INFO * [4399] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/10
2017-08-11 11:09:57,238 * INFO * [4399] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/10 Finished with 0
2017-08-11 11:09:57,250 * INFO * [5942] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/1
2017-08-11 11:09:57,906 * INFO * [4385] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/4 Finished with 0
2017-08-11 11:09:57,914 * INFO * [5978] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/2
2017-08-11 11:09:58,226 * INFO * [4373] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/8 Finished with 0
2017-08-11 11:09:58,234 * INFO * [5982] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/3
2017-08-11 11:09:58,336 * INFO * [4357] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/1 Finished with 0
2017-08-11 11:09:58,344 * INFO * [5986] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/4
2017-08-11 11:09:58,671 * INFO * [4397] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/9 Finished with 0
2017-08-11 11:09:58,679 * INFO * [5990] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/5
2017-08-11 11:09:58,915 * INFO * [4383] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/3 Finished with 0
2017-08-11 11:09:58,922 * INFO * [5994] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/6
2017-08-11 11:09:59,710 * INFO * [4363] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/4 Finished with 0
2017-08-11 11:09:59,718 * INFO * [5998] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/7
2017-08-11 11:10:00,271 * INFO * [4377] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/10 Finished with 0
2017-08-11 11:10:00,278 * INFO * [6009] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/8
2017-08-11 11:10:00,467 * INFO * [4375] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/9 Finished with 0
2017-08-11 11:10:00,476 * INFO * [6013] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/9
2017-08-11 11:10:00,913 * INFO * [4391] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/6 Finished with 0
2017-08-11 11:10:00,922 * INFO * [6020] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/10
2017-08-11 11:10:01,109 * INFO * [4367] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/6 Finished with 0
2017-08-11 11:10:01,115 * INFO * [6032] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/1
2017-08-11 11:10:01,168 * INFO * [4365] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/5 Finished with 0
2017-08-11 11:10:01,175 * INFO * [6036] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/2
2017-08-11 11:10:01,913 * INFO * [4379] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/1 Finished with 0
2017-08-11 11:10:01,919 * INFO * [6050] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/3
2017-08-11 11:10:04,105 * INFO * [4359] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/2 Finished with 0
2017-08-11 11:10:04,114 * INFO * [6144] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/4
2017-08-11 11:10:04,114 * INFO * [4393] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/7 Finished with 0
2017-08-11 11:10:04,120 * INFO * [6146] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/5
2017-08-11 11:10:04,328 * INFO * [4369] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/7 Finished with 0
2017-08-11 11:10:04,337 * INFO * [6160] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/6
2017-08-11 11:10:04,528 * INFO * [4389] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/5 Finished with 0
2017-08-11 11:10:04,533 * INFO * [6173] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/7
2017-08-11 11:10:05,289 * INFO * [4395] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/8 Finished with 0
2017-08-11 11:10:05,298 * INFO * [6215] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/8
2017-08-11 11:10:05,379 * INFO * [4381] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/1000/2 Finished with 0
2017-08-11 11:10:05,386 * INFO * [6235] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/9
2017-08-11 11:10:05,491 * INFO * [4361] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100/3 Finished with 0
2017-08-11 11:10:05,499 * INFO * [6239] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/10
2017-08-11 11:10:23,994 * INFO * [5978] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/2 Finished with 0
2017-08-11 11:10:23,998 * INFO * [7161] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/1
2017-08-11 11:10:25,351 * INFO * [5942] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/1 Finished with 0
2017-08-11 11:10:25,358 * INFO * [7196] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/2
2017-08-11 11:10:25,736 * INFO * [5982] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/3 Finished with 0
2017-08-11 11:10:25,743 * INFO * [7225] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/3
2017-08-11 11:10:26,413 * INFO * [6036] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/2 Finished with 0
2017-08-11 11:10:26,423 * INFO * [7263] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/4
2017-08-11 11:10:27,643 * INFO * [5990] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/5 Finished with 0
2017-08-11 11:10:27,649 * INFO * [7291] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/5
2017-08-11 11:10:28,091 * INFO * [6050] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/3 Finished with 0
2017-08-11 11:10:28,098 * INFO * [7295] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/6
2017-08-11 11:10:28,189 * INFO * [5986] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/4 Finished with 0
2017-08-11 11:10:28,199 * INFO * [7299] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/7
2017-08-11 11:10:30,017 * INFO * [6013] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/9 Finished with 0
2017-08-11 11:10:30,024 * INFO * [7321] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/8
2017-08-11 11:10:30,357 * INFO * [5998] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/7 Finished with 0
2017-08-11 11:10:30,362 * INFO * [7325] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/9
2017-08-11 11:10:31,225 * INFO * [6009] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/8 Finished with 0
2017-08-11 11:10:31,233 * INFO * [7338] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/10
2017-08-11 11:10:32,272 * INFO * [5994] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/6 Finished with 0
2017-08-11 11:10:32,277 * INFO * [7357] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/1
2017-08-11 11:10:32,772 * INFO * [6173] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/7 Finished with 0
2017-08-11 11:10:32,779 * INFO * [7384] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/2
2017-08-11 11:10:33,213 * INFO * [6032] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/1 Finished with 0
2017-08-11 11:10:33,227 * INFO * [7406] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/3
2017-08-11 11:10:33,630 * INFO * [6239] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/10 Finished with 0
2017-08-11 11:10:33,639 * INFO * [7431] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/4
2017-08-11 11:10:33,763 * INFO * [6144] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/4 Finished with 0
2017-08-11 11:10:33,771 * INFO * [7436] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/5
2017-08-11 11:10:34,161 * INFO * [6020] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/10000/10 Finished with 0
2017-08-11 11:10:34,169 * INFO * [7454] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/6
2017-08-11 11:10:34,302 * INFO * [6146] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/5 Finished with 0
2017-08-11 11:10:34,310 * INFO * [7467] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/7
2017-08-11 11:10:37,334 * INFO * [6235] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/9 Finished with 0
2017-08-11 11:10:37,346 * INFO * [7674] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/8
2017-08-11 11:10:38,716 * INFO * [6215] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/8 Finished with 0
2017-08-11 11:10:38,725 * INFO * [7758] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/9
2017-08-11 11:10:39,130 * INFO * [6160] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/0/100000/6 Finished with 0
2017-08-11 11:10:39,137 * INFO * [7770] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/10
2017-08-11 11:10:49,374 * INFO * [7161] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/1 Finished with 0
2017-08-11 11:10:49,386 * INFO * [8286] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/1
2017-08-11 11:10:54,825 * INFO * [7263] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/4 Finished with 0
2017-08-11 11:10:54,833 * INFO * [8460] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/2
2017-08-11 11:10:55,936 * INFO * [7295] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/6 Finished with 0
2017-08-11 11:10:55,945 * INFO * [8488] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/3
2017-08-11 11:10:56,556 * INFO * [7291] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/5 Finished with 0
2017-08-11 11:10:56,565 * INFO * [8500] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/4
2017-08-11 11:10:56,784 * INFO * [7338] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/10 Finished with 0
2017-08-11 11:10:56,793 * INFO * [8504] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/5
2017-08-11 11:10:57,101 * INFO * [7325] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/9 Finished with 0
2017-08-11 11:10:57,113 * INFO * [8508] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/6
2017-08-11 11:10:57,689 * INFO * [7321] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/8 Finished with 0
2017-08-11 11:10:57,696 * INFO * [8527] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/7
2017-08-11 11:10:57,957 * INFO * [7225] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/3 Finished with 0
2017-08-11 11:10:57,965 * INFO * [8531] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/8
2017-08-11 11:10:59,191 * INFO * [7406] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/1000/3 Finished with 0
2017-08-11 11:10:59,202 * INFO * [8574] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/10000/9
2017-08-11 11:10:59,600 * INFO * [7196] /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/15/100/2 Finished with 0
2017-08-11 11:10:59,610 * INFO * [8591] Started ./SIPSimRun_cMtx.sh in /ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome11









    






limit_output extension: Maximum message size of 20000 exceeded with 66721 characters

Aggregating confusion matrices





In [76]:



    


def agg_cMtx(prefix):
    # all data
    x = prefix + '-cMtx_data.txt'
    !nestagg delim \
       -d $buildDir \
       -k percIncorp,subsample_size,rep \
       -o $x \
       --tab \
       $x

    # overall
    x = prefix + '-cMtx_overall.txt'
    !nestagg delim \
        -d $buildDir \
        -k percIncorp,subsample_size,rep \
        -o $x \
        --tab \
        $x

    # by class
    x = prefix + '-cMtx_byClass.txt'
    !nestagg delim \
        -d $buildDir \
        -k percIncorp,subsample_size,rep \
        -o $x \
        --tab \
        $x
        
agg_cMtx('DESeq2')
agg_cMtx('DESeq2_multi')
agg_cMtx('qSIP') 
agg_cMtx('heavy')



    











In [77]:



    


%pushmsg "atomIncorp_sampleDepth complete!"

--End of simulation--

Results





In [78]:



    


F = os.path.join(buildDir, '*-cMtx_byClass.txt')
files = glob.glob(F)
files



    


















    
Out[78]:








['/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/DESeq2-cMtx_byClass.txt',
 '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/DESeq2_multi-cMtx_byClass.txt',
 '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/qSIP-cMtx_byClass.txt',
 '/ebio/abt3_projects/methanogen_host_evo/SIPSim_pt2/data/bac_genome1147/atomIncorp_sampDepth/heavy-cMtx_byClass.txt']

















In [ ]:

Content source: nick-youngblut/SIPSim

Similar notebooks:

notebook.community | gallery | about