Execution Configuration Options

Nipype gives you many liberties on how to create workflows, but the execution of them uses a lot of default parameters. But you have of course all the freedom to change them as you like.

Nipype looks for the configuration options in the local folder under the name nipype.cfg and in ~/.nipype/nipype.cfg (in this order). It can be divided into Logging and Execution options. A few of the possible options are the following:

Logging

workflow_level: How detailed the logs regarding workflow should be
log_to_file: Indicates whether logging should also send the output to a file

Execution

stop_on_first_crash: Should the workflow stop upon first node crashing or try to execute as many nodes as possible?
remove_unnecessary_outputs: This will remove any interface outputs not needed by the workflow. If the required outputs from a node changes, rerunning the workflow will rerun the node. Outputs of leaf nodes (nodes whose outputs are not connected to any other nodes) will never be deleted independent of this parameter.
use_relative_paths: Should the paths stored in results (and used to look for inputs) be relative or absolute. Relative paths allow moving the whole working directory around but may cause problems with symlinks.
job_finished_timeout: When batch jobs are submitted through, SGE/PBS/Condor they could be killed externally. Nipype checks to see if a results file exists to determine if the node has completed. This timeout determines for how long this check is done after a job finish is detected. (float in seconds; default value: 5)
poll_sleep_duration: This controls how long the job submission loop will sleep between submitting all pending jobs and checking for job completion. To be nice to cluster schedulers the default is set to 2

For the full list, see Configuration File.

Global, workflow & node level

The configuration options can be changed globally (i.e. for all workflows), for just a workflow, or for just a node. The implementations look as follows (note that you should first create directories if you want to change crashdump_dir and log_directory):

At the global level:



In [4]:

    
from nipype import config, logging
import os
os.makedirs('/output/log_folder', exist_ok=True)
os.makedirs('/output/crash_folder', exist_ok=True)

config_dict={'execution': {'remove_unnecessary_outputs': 'true',
                  
                           'keep_inputs': 'false',
                           'poll_sleep_duration': '60',
                           'stop_on_first_rerun': 'false',
                           'hash_method': 'timestamp',
                           'local_hash_check': 'true',
                           'create_report': 'true',
                           'crashdump_dir': '/output/crash_folder',
                           'use_relative_paths': 'false',
                           'job_finished_timeout': '5'},
             'logging': {'workflow_level': 'INFO',
                         'filemanip_level': 'INFO',
                         'interface_level': 'INFO',
                         'log_directory': '/output/log_folder',
                         'log_to_file': 'true'}}
config.update_config(config_dict)
logging.update_logging(config)

At the workflow level:



In [5]:

    
from nipype import Workflow
wf = Workflow(name="config_test")

# Change execution parameters
wf.config['execution']['stop_on_first_crash'] = 'true'

# Change logging parameters
wf.config['logging'] = {'workflow_level' : 'DEBUG',
                        'filemanip_level' : 'DEBUG',
                        'interface_level' : 'DEBUG',
                        'log_to_file' : 'True',
                        'log_directory' : '/output/log_folder'}

At the node level:



In [6]:

    
from nipype import Node
from nipype.interfaces.fsl import BET

bet = Node(BET(), name="config_test")

bet.config = {'execution': {'keep_unnecessary_outputs': 'false'}}



In [ ]: