ipyrad-analysis toolkit: treemix

View as notebook

The program TreeMix by Pickrell & Pritchard (2012) is used to infer population splits and admixture from allele frequency data. From the TreeMix documentation: "In the underlying model, the modern-day populations in a species are related to a common ancestor via a graph of ancestral populations. We use the allele frequencies in the modern populations to infer the structure of this graph."

Required software

A minor detail of the treemix conda installation is that it installs a bunch of junk alongside it, including R and openblas, into your conda environment. It's a ton of bloat that in my experience has a high chance of causing incompatibilities with other packages in your conda installation eventually. For this reason I recommend installing it into a separate conda environment.

The installation instructions below can be used to setup a new environment called treemix that you can use for this analyis. This will avoid the installation from conflicting with any of your other software. When you are done you can switch back to your default environment.


In [ ]:
# conda init
# conda create -n treemix
# conda activate treemix
# conda install treemix -c bioconda -c conda-forge
# conda install ipyrad -c bioconda -c conda-forge
# conda install jupyter -c conda-forge
# conda install toytree -c eaton-lab -c conda-forge
# jupyter-notebook

In [2]:
import ipyrad.analysis as ipa
import toytree
import toyplot

In [3]:
print('ipyrad', ipa.__version__)
print('toytree', toytree.__version__)
! treemix --version | grep 'TreeMix v. '


ipyrad 0.9.15
toytree 0.2.3
TreeMix v. 1.13

Required input data files

Your input data should be a .snps.hdf database file produced by ipyrad. If you do not have this you can generate it from any VCF file following the vcf2hdf5 tool tutorial. The database file contains the genotype calls information as well as linkage information that is used for subsampling unlinked SNPs and bootstrap resampling.


In [4]:
# the path to your HDF5 formatted snps file
data = "/home/deren/Downloads/ref_pop2.snps.hdf5"

Short Tutorial:

If you entered population information during data assembly then you may have already produced a .treemix.gz output file that can be used as input to the treemix command line program. Alternatively, you can run treemix using the ipyrad tool here which offers some additional flexibility for filtering SNP data, and for running treemix programatically over many parameter settings.

The key features offered by ipa.treemix include:

  1. Filter unlinked SNPs (1 per locus) many times for replicate analyses.
  2. Filter by sample or populations coverage.
  3. Plotting functions.
  4. Easy to write for-loops

In [5]:
# group individuals into populations
imap = {
    "virg": ["TXWV2", "LALC2", "SCCU3", "FLSF33", "FLBA140"],
    "mini": ["FLSF47", "FLMO62", "FLSA185", "FLCK216"],
    "gemi": ["FLCK18", "FLSF54", "FLWO6", "FLAB109"],
    "bran": ["BJSL25", "BJSB3", "BJVL19"],
    "fusi": ["MXED8", "MXGT4", "TXGR3", "TXMD3"],
    "sagr": ["CUVN10", "CUCA4", "CUSV6", "CUMM5"],
    "oleo": ["CRL0030", "HNDA09", "BZBB1", "MXSA3017"],
}

# minimum % of samples that must be present in each SNP from each group
minmap = {i: 0.5 for i in imap}

In [6]:
# init a treemix analysis object with some param arguments
tmx = ipa.treemix(
    data=data, 
    imap=imap,
    minmap=minmap, 
    seed=123456,
    root="bran,fusi",
    m=2,
)


Samples: 28
Sites before filtering: 349914
Filtered (indels): 0
Filtered (bi-allel): 13077
Filtered (mincov): 0
Filtered (minmap): 109383
Filtered (combined): 117718
Sites after filtering: 232196
Sites containing missing values: 221834 (95.54%)
Missing values in SNP matrix: 822320 (12.65%)
subsampled 29923 unlinked SNPs

In [7]:
# print the command string that will be called and run it
print(tmx.command)
tmx.run()


treemix -i /home/deren/Documents/ipyrad/newdocs/API-analysis/analysis-treemix/test.treemix.in.gz -o /home/deren/Documents/ipyrad/newdocs/API-analysis/analysis-treemix/test -m 2 -seed 123456 -root bran,fusi

In [8]:
# draw the resulting tree
tmx.draw_tree();


minigemivirgoleosagrfusibran0.000.050.10

In [9]:
# draw the covariance matrix
tmx.draw_cov();


0.0392740.008558-0.010861-0.007422-0.009953-0.010672-0.008925bran0.0085580.017783-0.006984-0.005362-0.003162-0.005733-0.005100fusi-0.010861-0.0069840.0175430.009980-0.003749-0.002483-0.003446sagr-0.007422-0.0053620.0099800.030868-0.009673-0.009907-0.008485oleo-0.009953-0.003162-0.003749-0.0096730.0182450.0049720.003321virg-0.010672-0.005733-0.002483-0.0099070.0049720.0167360.007087gemi-0.008925-0.005100-0.003446-0.0084850.0033210.0070870.015547minibranfusisagroleovirggemimini

1. Finding the best value for m

As with structure plots there is no True best value, but you can use model selection methods to decide whether one is a statistically better fit to your data than another. Adding additional admixture edges will always improve the likelihood score, but with diminishing returns as you add additional edges that explain little variation in the data. You can look at the log likelihood score of each model fit by running a for-loop like below. You may want to run this within another for-loop that iterates over different subsampled SNPs.


In [10]:
# init a treemix analysis object with some param arguments
tmx = ipa.treemix(
    data=data, 
    imap=imap,
    minmap=minmap, 
    seed=1234,
    root="bran,fusi",
)


Samples: 28
Sites before filtering: 349914
Filtered (indels): 0
Filtered (bi-allel): 13077
Filtered (mincov): 0
Filtered (minmap): 109383
Filtered (combined): 117718
Sites after filtering: 232196
Sites containing missing values: 221834 (95.54%)
Missing values in SNP matrix: 822320 (12.65%)
subsampled 29923 unlinked SNPs

In [11]:
tests = {}
nadmix = [0, 1, 2, 3, 4, 5]

# iterate over n admixture edges and store results in a dictionary
for adm in nadmix:
    tmx.params.m = adm
    tmx.run()
    tests[adm] = tmx.results.llik

In [12]:
# plot the likelihood for different values of m
toyplot.plot(
    nadmix,
    [tests[i] for i in nadmix],
    width=350, 
    height=275,
    stroke_width=3,
    xlabel="n admixture edges",
    ylabel="ln(likelihood)",
);


012345n admixture edges-1000100200ln(likelihood)

2. Iterate over different subsamples of SNPs

The treemix tool randomly subsamples 1 SNP per locus to reduce the effect of linkage on the results. However, depending on the size of your data set, and the strength of the signal, subsampling may yield slightly different results in different iterations. You can check over different subsampled iterations by re-initing the treemix tool with a different (or no) random seed. Below I plot the results of 9 iterations for m=2. I also use the global_=True option here which performs a more thorough (but slower) search.


In [13]:
# a gridded canvas to plot trees on 
canvas = toyplot.Canvas(width=600, height=700)

# iterate over multiple set of SNPs
for i in range(9):
    
    # init a treemix analysis object with a random (no) seed
    tmx = ipa.treemix(
        data=data, 
        imap=imap,
        minmap=minmap,
        root="bran,fusi",
        global_=True,
        m=2,
        quiet=True
    )
    
    # run model fit
    tmx.run()
    
    # select a plot grid axis and add tree to axes
    axes = canvas.cartesian(grid=(3, 3, i))
    tmx.draw_tree(axes)


gemiminivirgsagroleofusibran0.000.050.11virggemiminioleosagrbranfusi0.000.040.09gemiminivirgoleosagrfusibran0.000.060.13minigemivirgsagroleofusibran0.000.060.11minigemivirgoleosagrbranfusi0.000.050.10virggemiminioleosagrbranfusi0.000.050.10minigemivirgsagroleofusibran0.000.050.10minigemivirgsagroleofusibran0.000.060.12gemiminivirgsagroleofusibran0.000.050.09

In [14]:
# a gridded canvas to plot trees on 
canvas = toyplot.Canvas(width=600, height=700)

# iterate over multiple set of SNPs
for i in range(9):
    
    # init a treemix analysis object with a random (no) seed
    tmx = ipa.treemix(
        data=data, 
        imap=imap,
        minmap=minmap,
        root="bran,fusi",
        global_=True,
        m=3,
        quiet=True
    )
    
    # run model fit
    tmx.run()
    
    # create a grid axis and add tree to axes
    axes = canvas.cartesian(grid=(3, 3, i))
    tmx.draw_tree(axes)


gemiminivirgoleosagrfusibran0.000.050.09minigemivirgoleosagrfusibran0.000.050.11gemiminivirgoleosagrbranfusi0.000.050.10minigemivirgsagroleobranfusi0.000.040.08gemiminivirgoleosagrbranfusi0.000.040.08minigemivirgoleosagrfusibran0.000.060.11gemiminivirgsagroleofusibran0.000.050.10gemivirgminisagroleobranfusi0.000.050.10gemiminivirgoleosagrfusibran0.000.060.13

3. Save the plot to pdf


In [15]:
import toyplot.pdf
toyplot.pdf.render(canvas, "treemix-m3.pdf")