Some spot checkes of run time of different implementations of unweighted unifrac. Note that because this is comparing different software packages, the whole notebook can't be run in a single environment.

Comparing UniFrac in optimized mode and in pairwise mode. This is working from @wasade's fast_unifrac branch (https://github.com/wasade/scikit-bio/tree/fast_unifrac) commit dd1a405 on 9 Nov 2015.



In [61]:

    
from biom import load_table
from skbio import TreeNode
from skbio.diversity.beta import beta_diversity
from skbio.diversity.beta import unweighted_unifrac
import numpy as np

t = load_table('/Users/caporaso/code/q2d2/example-data/keyboard/q191/otu-table.tsv')
d = t.matrix_data.toarray().T
#tree = TreeNode.read('/Users/caporaso/temp/forensic-test/or-otus/rep_set.tre')
tree = TreeNode.read('/Users/caporaso/code/q2d2/example-data/keyboard/q191/rep-set.tre')



In [66]:

    
%%timeit
beta_diversity('unweighted_unifrac', d, ids=t.ids('sample'), otu_ids=t.ids('observation'), tree=tree)









    



10 loops, best of 3: 154 ms per loop



In [69]:

    
beta_diversity('unweighted_unifrac', d, ids=t.ids('sample'), otu_ids=t.ids('observation'), tree=tree)









    Out[69]:



In [70]:

    
%%timeit
beta_diversity(unweighted_unifrac, d, ids=t.ids('sample'), otu_ids=t.ids('observation'), tree=tree)









    



1 loops, best of 3: 42.4 s per loop



In [71]:

    
42.5/.154









    Out[71]:





275.974025974026

Using master (i.e., Python implementation of UniFrac) for comparison. This is working from master, commit ab72f3c on 9 Nov 2015.



In [1]:

    
from biom import load_table
from skbio import TreeNode
from skbio.diversity.beta import pw_distances
from skbio.diversity.beta import unweighted_unifrac
import numpy as np

t = load_table('/Users/caporaso/code/q2d2/example-data/keyboard/q191/otu-table.tsv')
d = t.matrix_data.toarray().T
#tree = TreeNode.read('/Users/caporaso/temp/forensic-test/or-otus/rep_set.tre')
tree = TreeNode.read('/Users/caporaso/code/q2d2/example-data/keyboard/q191/rep-set.tre')



In [3]:

    
%%timeit
pw_distances('unweighted_unifrac', d, ids=t.ids('sample'), otu_ids=t.ids('observation'), tree=tree)









    



1 loops, best of 3: 16.4 s per loop

Now with QIIME 1.9.1



In [3]:

    
%%timeit
!beta_diversity.py -i /Users/caporaso/code/q2d2/example-data/keyboard/q191/otu-table.tsv -t /Users/caporaso/code/q2d2/example-data/keyboard/q191/rep-set.tre -o /Users/caporaso/temp -m unweighted_unifrac









    



1 loops, best of 3: 1.08 s per loop



In [5]:

    
%%timeit
!parallel_beta_diversity.py -O 7 -i /Users/caporaso/code/q2d2/example-data/keyboard/q191/otu-table.tsv -t /Users/caporaso/code/q2d2/example-data/keyboard/q191/rep-set.tre -o /Users/caporaso/temp -m unweighted_unifrac









    



1 loops, best of 3: 4.79 s per loop



In [6]:

    
!print_qiime_config.py









    



System information
==================
         Platform:	darwin
   Python version:	2.7.10 |Continuum Analytics, Inc.| (default, Oct 19 2015, 18:31:17)  [GCC 4.2.1 (Apple Inc. build 5577)]
Python executable:	/Users/caporaso/.miniconda/envs/qiime-191/bin/python

QIIME default reference information
===================================
For details on what files are used as QIIME's default references, see here:
 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3

Dependency versions
===================
          QIIME library version:	1.9.1
           QIIME script version:	1.9.1
qiime-default-reference version:	0.1.3
                  NumPy version:	1.10.1
                  SciPy version:	0.16.0
                 pandas version:	0.16.2
             matplotlib version:	1.4.3
            biom-format version:	2.1.5
                   h5py version:	2.5.0 (HDF5 version: 1.8.15)
                   qcli version:	0.1.1
                   pyqi version:	0.3.2
             scikit-bio version:	0.2.3
                 PyNAST version:	1.2.2
                Emperor version:	0.9.51
                burrito version:	0.9.1
       burrito-fillings version:	0.1.1
              sortmerna version:	SortMeRNA version 2.0, 29/11/2014
              sumaclust version:	Not installed.
                  swarm version:	Swarm 1.2.19 [Jul 15 2015 16:29:27]
                          gdata:	Installed.

QIIME config values
===================
For definitions of these settings and to learn how to configure QIIME, see here:
 http://qiime.org/install/qiime_config.html
 http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir:	None
      pick_otus_reference_seqs_fp:	/Users/caporaso/.miniconda/envs/qiime-191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                         sc_queue:	all.q
      topiaryexplorer_project_dir:	None
     pynast_template_alignment_fp:	/Users/caporaso/.miniconda/envs/qiime-191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta
                  cluster_jobs_fp:	start_parallel_jobs.py
pynast_template_alignment_blastdb:	None
assign_taxonomy_reference_seqs_fp:	/Users/caporaso/.miniconda/envs/qiime-191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                     torque_queue:	friendlyq
                    jobs_to_start:	1
                       slurm_time:	None
            denoiser_min_per_core:	50
assign_taxonomy_id_to_taxonomy_fp:	/Users/caporaso/.miniconda/envs/qiime-191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt
                         temp_dir:	/Users/caporaso/temp/
                     slurm_memory:	None
                      slurm_queue:	None
                      blastall_fp:	blastall
                 seconds_to_sleep:	1



In [ ]: