Previous: Big Picture ::: Next: pipeline_vis
experiment
In [ ]:
%pylab inline
pylab.rcParams['figure.figsize'] = (5, 5)
The data pipeline consists of modules also called "schemas" because each module represents a schema (database) on the database server.
The first schemas you need to know are experiment, psy, and preprocessing. They contain information entered during experiments and immediate preprocessing of the data.
In [ ]:
import datajoint as dj
from pipeline.experiment import *
During the two-photon experiments, users enter information in the pipeline_experiment database in tables Session, Session.Fluorophore, and Scan. If a scan should not be processed, it is also entered in ScanIgnored.
In [ ]:
(dj.ERD(Session)+2).draw()
The manual tables have many references to lookup tables that define the allowed choices.
In [ ]:
(dj.ERD(Session)-1+1-dj.ERD(Scan)).draw()
In [ ]:
(dj.ERD(Scan)-1).draw()
Each node represents a table in the database and a class in the module. The lines between them represents dependencies.
You may preview the contents any table using its __repr__ method:
In [ ]:
Session()
The first few column names in italics represent the primary key of the relation, i.e. the identifying information for each row.
We use the terms table and relation interchangeably. A table row may also be called a tuple.
You may restrict a relaiton to a subset of its rows using the & operator:
In [ ]:
Session() & 'session_date>"2016-05-01"' & 'username="Jake"'
You may restrict one relation by another:
In [ ]:
recent = Session() & 'session_date>"2016-05-01"'
(Scan() & 'laser_power>50') & recent
The negative restriction uses the - operator and denotes "all except":
In [ ]:
Scan() - recent & 'laser_power>=60'
You may join two tables into one using the * operator.
In [ ]:
scans = Scan()*Session() & (Session.Fluorophore() & dict(fluorophore="GCaMP6f"))
scans
Joins and restrictions with other relations are meaningful even when they are not directly linked in the ERD. As long as there is a path from table A to B along solid lines, the join A * B or the restriction A & B is meaningful.
Solid lines represent prime dependencies, which propagate identifying information down the hiearachy.
You can see the descriptions of all attributes using the heading property:
In [ ]:
scans.heading
When the desired query is formed, the data may be retrived using the fetch method:
In [ ]:
data = scans.fetch()
data[4]
Or as a list of dicts,
In [ ]:
data = scans.fetch.as_dict()
data[4]
You can fetch just the desired attributes using the fetch['attr1', ..., 'attrn'] notation:
In [ ]:
d, i= scans.fetch['session_date', 'scan_idx']
d
If curious, you may see the underlying SQL queries using the make_sql method:
In [ ]:
scans.make_sql()
Next: pipeline_vis