Previous: pipeline_vis ::: Next Synchronization
preprocess
In [ ]:
%pylab inline
pylab.rcParams['figure.figsize'] = (6, 6)
The preprocess schema extracts, aligns, and synchronizes multiphoton trace data from both galvo and AOD systems.
In [ ]:
import datajoint as dj
from pipeline.preprocess import *
Here are the main elements of the preprocess schema:
In [ ]:
(dj.ERD.from_sequence([Prepare, Sync, ExtractRaw, Spikes])-1).draw()
Prepare and ExtractRaw combine the information from the galvo-based two-photon setups and the AOD-based setup. They perform all the operations that are specific to each type of imaging so that subsequent processing is no longer distinguished.
Many tables have part tables. A part table is so tightly dependent to its master table that it is never modified separately from its master. They are implemented as nested classes. Only one level of nesting is allowed. In other words, parts cannot have their own parts. On the ERD, parts are shown in a smaller font with no color code.
For example, preprocess.Prepare has many subtables that perform various jobs.
In [ ]:
dj.ERD(Prepare).add_parts().draw()
Similarly, preprocess.ExtractRaw has several part tables.
In [ ]:
dj.ERD(ExtractRaw).add_parts().draw()
In [ ]:
Prepare.GalvoAverageFrame().heading
If you know the exact identity of the scan of interest, use it as restriction:
In [ ]:
frames = Prepare.GalvoAverageFrame() & dict(animal_id=8623, slice=3, scan_idx=6)
frames
In [ ]:
# fetch data
g, r = frames.fetch.order_by('channel')['frame']
# plot data
def normalize(img):
m = img.min()
return (img - m)/(img.max() - m)
plt.imshow(np.stack(
[normalize(r), normalize(g), np.zeros_like(r)], 2))
grid(False)
Alternatively, get a whole set of scans matching some condition.
In [ ]:
experiment.Fluorophore()
In [ ]:
# select all slices imaged with GCaMP6f
slices = Prepare()*Slice() & (
experiment.Session.Fluorophore() & 'fluorophore="GCaMP6f"') & Prepare.GalvoAverageFrame()
slices
To address each entity in the set, get its primary key values with the fetch.keys() iterator. Then each key can be used in a restriction.
In [ ]:
keys = list(slices.fetch.keys())
key = keys[8] # pick an arbitrary frame
key
In [ ]:
frames = Prepare.GalvoAverageFrame() & key
g, r = frames.fetch.order_by('channel')['frame']
plt.imshow(np.stack(
[normalize(r), normalize(g), np.zeros_like(r)], 2))
grid(False)
Raw calcium traces are hosted in preprocess.ComputeTraces.Trace whereas inferred spikes are hosted in preprocess.Spikes.RateTrace.
In [ ]:
calcium = ComputeTraces.Trace()
spikes = Spikes.RateTrace()
traces = calcium*spikes # combined raw and spike rate traces
traces.heading
In [ ]:
Method.Galvo() # trace extraction methods for galvo imaging
In [ ]:
SpikeMethod()
In [ ]:
traces &= dict(extract_method=2, spike_method=3)
traces
Let's plot a random trace:
In [ ]:
key = np.random.choice(list(traces.fetch.keys())) # pick a random trace
key
In [ ]:
# fetch synchronized times
times, nslices = (Sync()*Prepare.Galvo() & key).fetch1['frame_times', 'nslices']
times = times[0,::nslices] - times[0,0] # start at 0
calcium, spikes = (traces & key).fetch1['trace', 'rate_trace']
In [ ]:
import seaborn as sns
with sns.axes_style('whitegrid'):
fig, ax = plt.subplots(2,1, sharex=True)
ax[0].plot(times, calcium.squeeze(), label='Ca Trace')
ax[1].plot(times, spikes.squeeze(), label='Spike Rate')
ax[0].set_ylabel('Fluorescence')
ax[1].set_ylabel('inferred spike rate')
ax[1].set_xlabel('time [s]')
sns.despine(fig)
fig.tight_layout()
Next: Synchronization