This notebook demonstrates how to access and manipulate data in the Allen Mouse Brain Connectivity Atlas. The MouseConnectivityCache AllenSDK class provides methods for downloading metadata about experiments, including their viral injection site and the mouse's transgenic line. You can request information either as a Pandas DataFrame or a simple list of dictionaries.
An important feature of the MouseConnectivityCache is how it stores and retrieves data for you. By default, it will create (or read) a manifest file that keeps track of where various connectivity atlas data are stored. If you request something that has not already been downloaded, it will download it and store it in a well known location.
In [41]:
from allensdk.core.mouse_connectivity_cache import MouseConnectivityCache
# The manifest file is a simple JSON file that keeps track of all of
# the data that has already been downloaded onto the hard drives.
# If you supply a relative path, it is assumed to be relative to your
# current working directory.
mcc = MouseConnectivityCache(manifest_file='connectivity/manifest.json')
# open up a list of all of the experiments
all_experiments = mcc.get_experiments(dataframe=True)
print "%d total experiments" % len(all_experiments)
# take a look at what we know about an experiment with a primary motor injection
all_experiments.loc[122642490]
Out[41]:
MouseConnectivityCache has a method for retrieving the adult mouse structure ontology as an Ontology class instance. This is a light wrapper around a Pandas DataFrame of structures that provides some helper methods for figuring out which structures are parents/children of each other. It's also handy for finding the ID of a structure by its acronym.
In [42]:
# grab the Ontology instance
ontology = mcc.get_ontology()
# get some info on the isocortex
isocortex = ontology['Isocortex']
isocortex
Out[42]:
On the connectivity atlas web site, you'll see that we show most of our data at a fairly coarse structure level. We did this by creating a "structure set" of ~300 structures. If you want to filter your ontology structures down to that list, you can do this:
In [43]:
from allensdk.api.queries.ontologies_api import OntologiesApi
# get the adult mouse structures summary structures
summary_structures = OntologiesApi().get_structures(structure_set_names='Mouse Connectivity - Summary')
summary_structure_ids = [ s['id'] for s in summary_structures ]
ontology[summary_structure_ids]
Out[43]:
This is how you can filter experiments by transgenic line:
In [44]:
# fetch the experiments that have injections in the isocortex of cre-positive mice
cre_cortical_experiments = mcc.get_experiments(cre=True,
injection_structure_ids=isocortex['id'])
print "%d Rbp4 cortical experiments" % len(cre_cortical_experiments)
# same as before, but restrict the cre line
rbp4_cortical_experiments = mcc.get_experiments(cre=[ 'Rbp4-Cre_KL100' ],
injection_structure_ids=isocortex['id'])
print "%d Rbp4 cortical experiments" % len(rbp4_cortical_experiments)
The ProjectionStructureUnionizes API data tells you how much signal there was in a given structure and experiment. It contains the density of projecting signal, volume of projecting signal, and other information. MouseConnectivityCache provides methods for querying and storing this data.
In [45]:
# find wild-type injections into primary visual area
visp = ontology['VISp']
visp_experiments = mcc.get_experiments(cre=False,
injection_structure_ids=visp['id'])
print "%d VISp experiments" % len(visp_experiments)
structure_unionizes = mcc.get_structure_unionizes([ e['id'] for e in visp_experiments ],
is_injection=False,
structure_ids=isocortex['id'])
print "%d VISp non-injection, cortical structure unionizes" % len(structure_unionizes)
In [46]:
structure_unionizes
Out[46]:
This is a rather large table, even for a relatively small number of experiments. You can filter it down to a smaller list of structures like this.
In [47]:
dense_unionizes = structure_unionizes[ structure_unionizes.projection_density > .5 ]
large_unionizes = dense_unionizes[ dense_unionizes.volume > .5 ]
large_structures = ontology[large_unionizes.structure_id]
print "%d large, dense, cortical, non-injection unionizes, %d structures" % ( len(large_unionizes), len(large_structures) )
print large_structures.name
large_unionizes
Out[47]:
The MouseConnectivityCache class provides a helper method for converting ProjectionStructureUnionize records for a set of experiments and structures into a matrix. This code snippet demonstrates how to make a matrix of projection density values in auditory sub-structures for cre-negative VISp experiments.
In [48]:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
visp_experiment_ids = [ e['id'] for e in visp_experiments ]
ctx_children = ontology.get_child_ids( ontology['Isocortex'].id )
pm = mcc.get_projection_matrix(experiment_ids = visp_experiment_ids,
projection_structure_ids = ctx_children,
hemisphere_ids= [2], # right hemisphere, ipsilateral
parameter = 'projection_density')
row_labels = pm['rows'] # these are just experiment ids
column_labels = [ c['label'] for c in pm['columns'] ]
matrix = pm['matrix']
fig, ax = plt.subplots(figsize=(15,15))
heatmap = ax.pcolor(matrix, cmap=plt.cm.afmhot)
# put the major ticks at the middle of each cell
ax.set_xticks(np.arange(matrix.shape[1])+0.5, minor=False)
ax.set_yticks(np.arange(matrix.shape[0])+0.5, minor=False)
ax.set_xlim([0, matrix.shape[1]])
ax.set_ylim([0, matrix.shape[0]])
# want a more natural, table-like display
ax.invert_yaxis()
ax.xaxis.tick_top()
ax.set_xticklabels(column_labels, minor=False)
ax.set_yticklabels(row_labels, minor=False)
plt.show()
The MouseConnectivityCache class also helps you download and open every experiment's projection grid data volume. By default it will download 25um volumes, but uou could also download data at other resolutions if you prefer (10um, 50um, 100um).
This demonstrates how you can load the projection density for a particular experiment. It also shows how to download the template volume to which all grid data is registered. Voxels in that template have been structurally annotated by neuroanatomists and stored in a separate annotation volume image.
In [49]:
experiment_id = 181599674
# projection density: number of projecting pixels / voxel volume
pd, pd_info = mcc.get_projection_density(experiment_id)
# injection density: number of projecting pixels in injection site / voxel volume
ind, ind_info = mcc.get_injection_density(experiment_id)
# injection fraction: number of pixels in injection site / voxel volume
inf, inf_info = mcc.get_injection_fraction(experiment_id)
# data mask:
# binary mask indicating which voxels contain valid data
dm, dm_info = mcc.get_data_mask(experiment_id)
template, template_info = mcc.get_template_volume()
annot, annot_info = mcc.get_annotation_volume()
print pd_info
print pd.shape, template.shape, annot.shape
Once you have these loaded, you can use matplotlib see what they look like.
In [50]:
# compute the maximum intensity projection of the projection density
pd_mip = pd.max(axis=0)
ind_mip = ind.max(axis=0)
inf_mip = inf.max(axis=0)
dm_mip = dm.min(axis=0)
# show that slice of all volumes side-by-side
f, ((ax1, ax2, ax3), (ax4, ax5, ax6)) = plt.subplots(2, 3, figsize=(15, 6))
ax1.imshow(pd_mip, cmap='hot', aspect='equal')
ax1.set_title("projection density MaxIP")
ax2.imshow(ind_mip, cmap='hot', aspect='equal')
ax2.set_title("injection density MaxIP")
ax3.imshow(inf_mip, cmap='hot', aspect='equal')
ax3.set_title("injection fraction MaxIP")
# pick a slice to show
slice_idx = 264
ax4.imshow(template[slice_idx,:,:], cmap='gray', aspect='equal', vmin=template.min(), vmax=template.max())
ax4.set_title("registration template")
ax5.imshow(annot[slice_idx,:,:], cmap='gray', aspect='equal', vmin=0, vmax=2000)
ax5.set_title("annotation volume")
ax6.imshow(dm_mip, cmap='hot', aspect='equal', vmin=0, vmax=1)
ax6.set_title('data mask MinIP')
Out[50]:
The MouseConnectivityCache also provides you with a way to compute and store structure masks. A structure mask is a numpy array that tells you if a voxel belongs to a particular structure.
How is this different from the annotation volume? The annotation volume assigns each voxel a single label, usually one that is at the bottom of the structure ontology. You won't find a single voxel labeled 'Isocortex' in the annotation volume, but there are many voxels that are labeled as descendants of the Isocortex. MouseConnectivityCache can compute for you a mask of voxels that are labeled as any of the descendants of a structure.
Careful. Computing masks is a time-consuming task!
In [51]:
isocortex_mask, _ = mcc.get_structure_mask(isocortex['id'])
# pull out the values of all voxels in the isocortex mask
isocortex_pd = pd[isocortex_mask > 0]
# print out the average projection density of voxels in the isocortex
print isocortex_pd.mean()