In Workflow notebook you learnt about Workflows that specify processing by an execution graph and offer efficient recomputing. However, sometimes you might want to use Interfaces that gives better control of the execution of each step and can be easily combine with any Python code. Unfortunately, Interfaces do not offer any caching and you always dully recompute your task.
Solution to this problem can be a caching mechanism supported by Nipype. Nipype caching relies on the Memory class and creates an execution context that is bound to a disk cache.
When you instantiate the class you should provide base_dir and additional subdirectory called nipype_mem will be automatically created.
In [ ]:
from nipype.caching import Memory
mem = Memory(base_dir='/output/workingdir')
If we want to ask for caching for the BET interface, we can use cache method that takes interfaces classes as an argument.
In [ ]:
from nipype.interfaces import fsl
bet_mem = mem.cache(fsl.BET)
Now, bet_mem can be applied as a function with inputs of the BET interface as the function arguments. Those inputs are given as keyword arguments, bearing the same name as the name in the inputs specs of the interface.
In [ ]:
bet_mem(in_file="/data/ds000114/sub-02/ses-test/anat/sub-02_ses-test_T1w.nii.gz",
out_file="/output/sub-02_T1w_brain.nii.gz",
mask=True)
Out[ ]:
As you can seen bet command was run as expected. We can now check the content of caching file:
In [ ]:
! ls -l /output/workingdir/nipype_mem
A special subdirectory for our interface has been created. Let's try to run this command again:
In [ ]:
bet_mem(in_file="/data/ds000114/sub-02/ses-test/anat/sub-02_ses-test_T1w.nii.gz",
out_file="/output/sub-02_T1w_brain.nii.gz",
mask=True)
Out[ ]:
Now, the bet command was not run, but precomputed outputs were collected!
If you created cached results that you're not going reuse, you can use Memory.clear_runs_since() to flush the cache. Note, that if you use the method without any argument it will remove results used before current date, so will keep the results we've just calculated, let's check:
In [ ]:
mem.clear_runs_since()
bet_mem(in_file="/data/ds000114/sub-02/ses-test/anat/sub-02_ses-test_T1w.nii.gz",
out_file="/output/sub-02_T1w_brain.nii.gz",
mask=True)
Out[ ]:
As you can see, Nipype again collected the old results. If we want to remove everything, we have to put some future date:
In [ ]:
mem.clear_runs_since(year=2020, month=1, day=1)
You can also check Memory.clear_runs_since().