Using R functions in Python

This notebook demonstrates the usage of nat, elmr or catnat in Python. Pymaid offers a few wrappers to use specific functions right away but more importantly it converts between R and Python data so that you can use pretty much any function without much effort.

For example, you could:

  1. load a neuron into Python
  2. convert it to R
  3. use R to prune it and
  4. convert back to Python
  5. plot

In [2]:
import pymaid
from pymaid import rmaid

# Omit http user and password if not required
remote_instance = pymaid.CatmaidInstance('server_url', 'api_token', 'http_user', 'http_password')

print('Tested with pymaid version {0}'.format(pymaid.__version__))


INFO  : Global CATMAID instance set. (pymaid.fetch)
Tested with pymaid version 0.72.3

pymaid.rmaid provides a function, data2py, to convert general R data to Python data. More importantly, there are two functions, neuron2py and neuron2r, that allow you to convert back and forth between R and Python objects.


In [5]:
# Load a neuron using PyMaid
neuron_py = pymaid.get_neuron(16, remote_instance = remote_instance )


Fetching neurons: 100%|██████████| 1/1 [00:04<00:00,  1.00s/it]

In [7]:
neuron_py


Out[7]:
type              <class 'pymaid.core.CatmaidNeuron'>
neuron_name                  PN glomerulus VA6 017 DB
skeleton_id                                        16
n_nodes                                         12732
n_connectors                                     1937
n_branch_nodes                                    774
n_end_nodes                                       823
n_open_ends                                       280
cable_length                                  2865.99
review_status                                      NA
soma                                          2941309
dtype: object

In [8]:
#Convert to R
neuron_r = rmaid.neuron2r( neuron_py )
neuron_r


Out[8]:
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12e0f02c8 / R:0x127d982a0>
[ListV..., RNULL..., IntVe..., ..., Float..., DataF..., IntVe...]
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12e0f02c8 / R:0x127d982a0>
[ListV..., RNULL..., IntVe..., ..., Float..., DataF..., IntVe...]
  url: <class 'rpy2.rinterface.RNULLType'>
  rpy2.rinterface.NULL
  StartPoint: <class 'rpy2.robjects.vectors.IntVector'>
  R object with classes: ('integer',) mapped to:
<IntVector - Python:0x128e11cc8 / R:0x12d9a45d8>
[      26]
  ...
  tags: <class 'rpy2.robjects.vectors.FloatVector'>
  R object with classes: ('numeric',) mapped to:
<FloatVector - Python:0x12ed4bd88 / R:0x123a11228>
[1.000000]
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12e0f02c8 / R:0x127d982a0>
[ListV..., RNULL..., IntVe..., ..., Float..., DataF..., IntVe...]
  StartPoint: <class 'rpy2.robjects.vectors.IntVector'>
  R object with classes: ('integer',) mapped to:
<IntVector - Python:0x125a26d48 / R:0x12d5e0c00>
[     493,      496,      543, ...,    12726,    12729,    12732]

Let's try it the other way around. Rmaid offers a wrapper to initialise R's catmaid library:


In [9]:
# Here we use credentials stored in the alread existing <remote_instance> but we could also just pass those by hand
# see help(rmaid.init_rcatmaid)
rcatmaid = rmaid.init_rcatmaid( remote_instance = remote_instance )


INFO  : Rcatmaid successfully initiated. (pymaid.rmaid)

rcatmaid behaves pretty much like a normal Python module such as pymaid:


In [10]:
#Check which functions are available
dir(rcatmaid)


Out[10]:
['*_catmaidneuron',
 '+_catmaidneuron',
 '___NAMESPACE___',
 '___S3MethodsTable___',
 '__doc__',
 '__loader__',
 '__name__',
 '__package__',
 '__rdata__',
 '__rname__',
 '__spec__',
 '__version__',
 '_env',
 '_exported_names',
 '_packageName',
 '_package_statevars',
 '_rpy2r',
 '_symbol_check_after',
 '_symbol_r2python',
 '_translation',
 'as_catmaidmesh',
 'as_catmaidmesh_catmaidmesh',
 'as_catmaidmesh_default',
 'as_catmaidmesh_hxsurf',
 'as_catmaidmesh_mesh3d',
 'as_mesh3d_catmaidmesh',
 'authname',
 'authpassword',
 'catmaid_add_volume',
 'catmaid_aids',
 'catmaid_cache_connection',
 'catmaid_cached_connection',
 'catmaid_connection',
 'catmaid_connection_fingerprint',
 'catmaid_connection_getenv',
 'catmaid_connection_setenv',
 'catmaid_connection_unsetenv',
 'catmaid_entities_from_models',
 'catmaid_error_check',
 'catmaid_fetch',
 'catmaid_get_annotationlist',
 'catmaid_get_annotations_for_skeletons',
 'catmaid_get_compact_skeleton',
 'catmaid_get_connector_table',
 'catmaid_get_connectors',
 'catmaid_get_connectors_between',
 'catmaid_get_contributor_stats',
 'catmaid_get_label_stats',
 'catmaid_get_neuronnames',
 'catmaid_get_review_status',
 'catmaid_get_treenode_table',
 'catmaid_get_user_list',
 'catmaid_get_volume',
 'catmaid_get_volumelist',
 'catmaid_last_connection',
 'catmaid_login',
 'catmaid_parse_json',
 'catmaid_parse_url',
 'catmaid_query_by_annotation',
 'catmaid_query_by_name',
 'catmaid_query_connected',
 'catmaid_remove_annotations_for_skeletons',
 'catmaid_rename_neuron',
 'catmaid_set_annotations_for_skeletons',
 'catmaid_skids',
 'catmaid_user_history',
 'catmaid_version',
 'connectors',
 'connectors_catmaidneuron',
 'connectors_neuron',
 'connectors_neuronlist',
 'copy_tags_connectors',
 'get_neuronid_for_skid',
 'getenvoroption',
 'list2df',
 'normalise_colours',
 'nsoma',
 'plot3d_catmaidneuron',
 'plot3d_somarad',
 'process_one_user_history',
 'query_by_neuron_or_annotation',
 'read_catmaid_selection',
 'read_neuron_catmaid',
 'read_neurons_catmaid',
 'server',
 'somapos_catmaidneuron',
 'summary_catmaidneuron',
 'token',
 'write_catmaid_selection',
 'xform_catmaidneuron']

Note that '.' (dots) from R have been replaced with '_' (underscore). For example read.neurons.catmaid is in Python read_neurons_catmaid:


In [11]:
neuron_rcatmaid = rcatmaid.read_neuron_catmaid(16)
neuron_rcatmaid


Out[11]:
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12f008e48 / R:0x12db31148>
[IntVe..., IntVe..., IntVe..., ..., ListV..., StrVe..., ListV...]
  NumPoints: <class 'rpy2.robjects.vectors.IntVector'>
  R object with classes: ('integer',) mapped to:
<IntVector - Python:0x12edadd08 / R:0x132956b38>
[   12732]
  StartPoint: <class 'rpy2.robjects.vectors.IntVector'>
  R object with classes: ('integer',) mapped to:
<IntVector - Python:0x12edad188 / R:0x132956b68>
[      26]
  BranchPoints: <class 'rpy2.robjects.vectors.IntVector'>
  R object with classes: ('integer',) mapped to:
<IntVector - Python:0x125a2a348 / R:0x12a31aa00>
[     493,      496,      543, ...,    12726,    12729,    12732]
  ...
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12f008e48 / R:0x12db31148>
[IntVe..., IntVe..., IntVe..., ..., ListV..., StrVe..., ListV...]
  StartPoint: <class 'rpy2.robjects.vectors.StrVector'>
  R object with classes: ('character',) mapped to:
<StrVector - Python:0x12ed4b548 / R:0x12c90d938>
['https://neuropil.janelia.org/tracing/fafb/v14//1...]
R object with classes: ('catmaidneuron', 'neuron') mapped to:
<ListVector - Python:0x12f008e48 / R:0x12db31148>
[IntVe..., IntVe..., IntVe..., ..., ListV..., StrVe..., ListV...]

In [12]:
#Convert to Python
neuron_conv = rmaid.neuron2py(neuron_rcatmaid)
neuron_conv


WARNING : Please provide a remote instance if you want to add neuron name. (pymaid.rmaid)
Making neurons: 100%|██████████| 1/1 [00:00<00:00,  4.22it/s]
Out[12]:
<class 'pymaid.core.CatmaidNeuronList'> of 1 neurons 
   neuron_name  skeleton_id  n_nodes  n_connectors  n_branch_nodes  \
0          NA           16    12732          1937             774   

   n_end_nodes  open_ends  cable_length review_status  soma  
0          823        280   2865.993245            NA  True  

Note how neuron_name is NA? Some data that is stored in Python neurons does not exists in R and vice versa. Nodes, skeleton_id, connectors and tags, however, are always preserved.

Loading other libraries

Above example uses rcatmaid for which we have a wrapper in rmaid. But even loading other libraries is a piece of cake!


In [1]:
from rpy2.robjects.packages import importr

In [3]:
nat = importr('nat')
# Look at the last 20 functions
dir(nat)[-20:]


Out[3]:
['xformpoints_default',
 'xformpoints_reglist',
 'xprodm',
 'xyzmatrix',
 'xyzmatrix<-',
 'xyzmatrix<-_default',
 'xyzmatrix<-_dotprops',
 'xyzmatrix<-_hxsurf',
 'xyzmatrix<-_igraph',
 'xyzmatrix<-_neuron',
 'xyzmatrix<-_neuronlist',
 'xyzmatrix<-_shape3d',
 'xyzmatrix_default',
 'xyzmatrix_dotprops',
 'xyzmatrix_hxsurf',
 'xyzmatrix_igraph',
 'xyzmatrix_mesh3d',
 'xyzmatrix_neuron',
 'xyzmatrix_neuronlist',
 'xyzpos']

Let us use some basic nat function like prune_strahler


In [15]:
pruned = nat.prune_strahler( neuron_r  )

In [16]:
pruned_py = rmaid.neuron2py( pruned )
pruned_py


WARNING : Neuron has only nodes (no name, skid, connectors or tags). (pymaid.rmaid)
Making neurons: 100%|██████████| 1/1 [00:00<00:00, 12.73it/s]
Out[16]:
<class 'pymaid.core.CatmaidNeuronList'> of 1 neurons 
   neuron_name skeleton_id  n_nodes  n_connectors  n_branch_nodes  n_end_nodes  \
0          NA          NA     6137             0              59           60   

   open_ends  cable_length review_status   soma  
0         60   1211.530666            NA  False  

Now we will plot the pruned vs non-pruned neuron.

Please note that the cell with the 3D plot will be empty when you first open this notebook. You've got to run the code!


In [17]:
import plotly.offline as pyoff

#Need to initialize plotly for interactive rendering with Jupyter notebooks
from plotly.offline import init_notebook_mode
init_notebook_mode(connected=True)



In [20]:
fig = pymaid.plot3d( [neuron_py,pruned_py], backend='plotly', width=1000 )
pyoff.iplot(fig)


INFO  : Preparing neurons for plotting... (pymaid.plotting)
INFO  : Generating traces... (pymaid.plotting)
INFO  : Tracing done. (pymaid.plotting)
INFO  : Done. Plotted 18869 nodes and 0 connectors (pymaid.plotting)
INFO  : Use plotly.offline.plot(fig, filename="3d_plot.html") to plot. Optimised for Google Chrome. (pymaid.plotting)

In [ ]: