Purpose

The purpose of this pipeline is to complete all the pre-processing steps needed to turn diffusion-weighted images into FA images that will be used to build a template diffusion tensor atlas for fiber tracking.

Inputs

The input to this pipeline is a list of subject IDs that is used to generate lists of the corresponding DWIs processed with automated quality control, T2s, and brain label images that are treated as brain masks.

Pipeline Steps for CreateDWIWorkflow

1. A rigid transform from the b0 of the DWI to the T2 is first derived with BRAINSFit. This rigid transform is then used to resample the DWI in place into the physical space of the T2 (with gtractResampleDWIInPlace) while preserving the voxel lattice of the DWI.

2. The b0 from the DWI resampled in place is extracted with extractNrrdVectorIndex. A BSpline transform from the T2 to the b0 of the DWI resampled in place is then derived with BRAINSFit and used to resample the brain mask into the the space of the DWI resampled in place with BRAINSResample.

3. A masked tensor image is estimated with dtiprocess using the DWI resampled in place and resampled brain mask. dtiprocess is used again to compute FA, MD, RD, Frobenius norm, lambda1 (AD), lambda2, and lambda3 images with the masked tensor image.

-- coding: utf-8 --

3.0

Purpose

=======

The purpose of this pipeline is to complete all the pre-processing steps needed to turn diffusion-weighted images into FA images that will be used to build a template diffusion tensor atlas for fiber tracking.

Inputs

======

The input to this pipeline is a list of subject IDs that is used to generate lists of the corresponding DWIs processed with automated quality control, T2s, and brain label images that are treated as brain masks.

Pipeline Steps for CreateDWIWorkflow

====================================

1. A rigid transform from the b0 of the DWI to the T2 is first derived with BRAINSFit. This rigid transform is then used to resample the DWI in place into the physical space of the T2 (with gtractResampleDWIInPlace) while preserving the voxel lattice of the DWI.

1. The b0 from the DWI resampled in place is extracted with extractNrrdVectorIndex. A BSpline transform from the T2 to the b0 of the DWI resampled in place is then derived with BRAINSFit and used to resample the brain mask into the the space of the DWI resampled in place with BRAINSResample.

1. A masked tensor image is estimated with dtiprocess using the DWI resampled in place and resampled brain mask. dtiprocess is used again to compute FA, MD, RD, Frobenius norm, lambda1 (AD), lambda2, and lambda3 images with the masked tensor image.

import os import glob import sys

\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

#

Prepend the shell environment search paths

PROGRAM_PATHS = '/raid0/homes/johnsonhj/src/BSA-clang31/bin' PROGRAM_PATHS = PROGRAM_PATHS.split(':') PROGRAM_PATHS.extend(os.environ['PATH'].split(':')) os.environ['PATH'] = ':'.join(PROGRAM_PATHS)

CUSTOM_ENVIRONMENT=dict()

Platform specific information

Prepend the python search paths

PYTHON_AUX_PATHS = '/raid0/homes/johnsonhj/src/BRAINSStandAlone/AutoWorkup:/raid0/homes/johnsonhj/src/BSA-clang31/SimpleITK-build/libXXX:/raid0/homes/johnsonhj/src/BSA-clang31/NIPYPE' PYTHON_AUX_PATHS = PYTHON_AUX_PATHS.split(':') PYTHON_AUX_PATHS.extend(sys.path) sys.path = PYTHON_AUX_PATHS

import SimpleITK as sitk import nipype from nipype.interfaces.base import CommandLine, CommandLineInputSpec, TraitedSpec, File, Directory from nipype.interfaces.base import traits, isdefined, BaseInterface from nipype.interfaces.utility import Merge, Split, Function, Rename, IdentityInterface import nipype.interfaces.io as nio # Data i/oS import nipype.pipeline.engine as pe # pypeline engine from nipype.interfaces.freesurfer import ReconAll from SEMTools import *

def get_global_sge_script(pythonPathsList, binPathsList, customEnvironment={}): """This is a wrapper script for running commands on an SGE cluster so that all the python modules and commands are pathed properly"""

custEnvString = ""
for key, value in customEnvironment.items():
    custEnvString += "export " + key + "=" + value + "\n"

PYTHONPATH = ":".join(pythonPathsList)
BASE_BUILDS = ":".join(binPathsList)
GLOBAL_SGE_SCRIPT = """#!/bin/bash

echo "STARTED at: $(date +'%F-%T')" echo "Ran on: $(hostname)" export PATH={BINPATH} export PYTHONPATH={PYTHONPATH}

echo "========= CUSTOM ENVIORNMENT SETTINGS ==========" echo "export PYTHONPATH={PYTHONPATH}" echo "export PATH={BINPATH}" echo "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"

echo "With custom environment:" echo {CUSTENV} {CUSTENV}

NOTE: nipype inserts the actual commands that need running below this section.

""".format(PYTHONPATH=PYTHONPATH, BINPATH=BASE_BUILDS, CUSTENV=custEnvString) return GLOBAL_SGE_SCRIPT

Create the shell wrapper script for ensuring that all jobs running on remote hosts from SGE

have the same environment as the job submission host.

JOB_SCRIPT = get_global_sge_script(sys.path, PROGRAM_PATHS, CUSTOM_ENVIRONMENT) SGE_JOB_SCRIPT=JOB_SCRIPT

def GetDWIReferenceImagesFromSessionID(SESSION_TUPLE,BASE_STRUCT,BASE_DWI): """A function to extract file names from base parameters""" import os import glob PROJ_ID=SESSION_TUPLE[0] SUBJ_ID=SESSION_TUPLE[1] SESSION_ID=SESSION_TUPLE[2] FixImageList=glob.glob("{BASE_STRUCT}/{PROJ_ID}/{SUBJ_ID}/{SESSION_ID}/TissueClassify/t2_average_BRAINSABC.nii.gz".format(BASE_STRUCT=BASE_STRUCT,PROJ_ID=PROJ_ID,SUBJ_ID=SUBJ_ID,SESSION_ID=SESSION_ID)) FixMaskImageList=glob.glob("{BASE_STRUCT}/{PROJ_ID}/{SUBJ_ID}/{SESSION_ID}/TissueClassify/fixed_brainlabels_seg.nii.gz".format(BASE_STRUCT=BASE_STRUCT,PROJ_ID=PROJ_ID,SUBJ_ID=SUBJ_ID,SESSION_ID=SESSION_ID)) MovingDWIList=glob.glob("{BASE_DWI}/{PROJ_ID}/{SUBJ_ID}/{SESSION_ID}/*_concat_QCed.nrrd".format(BASE_DWI=BASE_DWI,PROJ_ID=PROJ_ID,SUBJ_ID=SUBJ_ID,SESSION_ID=SESSION_ID))

## Should check that each list has 1 element

print "^"*80
print SESSION_TUPLE
print BASE_STRUCT
print BASE_DWI
print "^"*80
print FixImageList
print FixMaskImageList
print MovingDWIList
print "^"*80
FixImage=FixImageList[0]
FixMaskImage=FixMaskImageList[0]
MovingDWI=MovingDWIList[0]


print "="*80
print FixImage
print FixMaskImage
print MovingDWI
print "="*80

return FixImage,FixMaskImage,MovingDWI

def CreateDWIWorkFlow(proj_name,subj_name,sessionname):
WFname="DWIPrototype"+str(proj_name)+str(subjname)+""+str(session_name) DWIWorkflow = pe.Workflow(name=WFname)

#DWIWorkflow.base_dir = os.path.join(CACHE_BASE,session_name)

inputsSpec = pe.Node(interface=IdentityInterface(fields=['T2Volume', 'DWIVolume','BrainMask'
                                                ]), name='inputspec')

outputsSpec = pe.Node(interface=IdentityInterface(fields=['FAImage'
                                                ]), name='outputspec')

BFitB0_T2 = pe.Node(interface=BRAINSFit(), name="B0ToT2_Rigid")
#BF_cpu_sge_options_dictionary = {'qsub_args': '-S /bin/bash -pe smp1 2-12 -l h_vmem=14G,mem_free=4G -o /dev/null -e /dev/null ' + CLUSTER_QUEUE, 'overwrite': True}

#BFitB0_T2.plugin_args = BF_cpu_sge_options_dictionary
BFitB0_T2.inputs.costMetric = "MMI"
BFitB0_T2.inputs.numberOfSamples = 100000
BFitB0_T2.inputs.numberOfIterations = [1500]
BFitB0_T2.inputs.numberOfHistogramBins = 50
BFitB0_T2.inputs.maximumStepLength = 0.2
BFitB0_T2.inputs.minimumStepLength = [0.00005]
BFitB0_T2.inputs.useRigid = True
#BFitB0_T2.inputs.useAffine = True  # Using initial transform from BRAINSABC
BFitB0_T2.inputs.maskInferiorCutOffFromCenter = 65
BFitB0_T2.inputs.maskProcessingMode = "ROIAUTO"
BFitB0_T2.inputs.ROIAutoDilateSize = 13
BFitB0_T2.inputs.backgroundFillValue = 0.0
BFitB0_T2.inputs.initializeTransformMode = 'useCenterOfHeadAlign'

BFitB0_T2.inputs.outputTransform = "B0ToT2_RigidTransform.h5"
BFitB0_T2.inputs.outputVolume = "B0_in_T2Space_Output.nii.gz"

DWIWorkflow.connect(inputsSpec, 'T2Volume', BFitB0_T2, 'fixedVolume')
DWIWorkflow.connect(inputsSpec, 'DWIVolume', BFitB0_T2, 'movingVolume')
DWIRIP = pe.Node(interface=gtractResampleDWIInPlace(), name="DWIRIP_B0ToT2")
DWIRIP.inputs.outputVolume = 'DTIPRepOutput_RIP.nrrd'
#DWIRIP.inputs.imageOutputSize = [164,164,100]

DWIWorkflow.connect(BFitB0_T2,'outputTransform',DWIRIP,'inputTransform')
DWIWorkflow.connect(inputsSpec, 'DWIVolume',DWIRIP,'inputVolume')
BSPLINE_T2_TO_RIPB0 = pe.Node(interface=BRAINSFit(), name="BSPLINE_T2_TO_RIPB0")
#BSPLINE_T2_TO_RIPB0.plugin_args = BF_cpu_sge_options_dictionary
BSPLINE_T2_TO_RIPB0.inputs.costMetric = "MMI"
BSPLINE_T2_TO_RIPB0.inputs.numberOfSamples = 100000
BSPLINE_T2_TO_RIPB0.inputs.numberOfIterations = [1500]
BSPLINE_T2_TO_RIPB0.inputs.numberOfHistogramBins = 50
BSPLINE_T2_TO_RIPB0.inputs.maximumStepLength = 0.2
BSPLINE_T2_TO_RIPB0.inputs.minimumStepLength = [0.00025,0.00025,0.00025,0.00025,0.00025]
BSPLINE_T2_TO_RIPB0.inputs.useRigid = True
BSPLINE_T2_TO_RIPB0.inputs.useScaleVersor3D = True
BSPLINE_T2_TO_RIPB0.inputs.useScaleSkewVersor3D = True
BSPLINE_T2_TO_RIPB0.inputs.useAffine = True  # Using initial transform from BRAINSABC

BSPLINE_T2_TO_RIPB0.inputs.useBSpline = True
#BSPLINE_T2_TO_RIPB0.inputs.useROIBSpline = True

##  This needs to be debugged, it should work. BSPLINE_T2_TO_RIPB0.inputs.useROIBSpline = True
BSPLINE_T2_TO_RIPB0.inputs.useExplicitPDFDerivativesMode = "AUTO"
BSPLINE_T2_TO_RIPB0.inputs.useCachingOfBSplineWeightsMode = "ON"
BSPLINE_T2_TO_RIPB0.inputs.maxBSplineDisplacement = 24
BSPLINE_T2_TO_RIPB0.inputs.splineGridSize = [ 14, 10, 12 ]

BSPLINE_T2_TO_RIPB0.inputs.maskInferiorCutOffFromCenter = 65
BSPLINE_T2_TO_RIPB0.inputs.maskProcessingMode = "ROIAUTO"
BSPLINE_T2_TO_RIPB0.inputs.ROIAutoDilateSize = 13
BSPLINE_T2_TO_RIPB0.inputs.backgroundFillValue = 0.0
BSPLINE_T2_TO_RIPB0.inputs.initializeTransformMode = 'useCenterOfHeadAlign'


BSPLINE_T2_TO_RIPB0.inputs.bsplineTransform = "T2ToRIPB0_BSplineTransform.h5"
BSPLINE_T2_TO_RIPB0.inputs.outputVolume = "T2ToRIPB0_Output.nii.gz"

DWIWorkflow.connect(DWIRIP, 'outputVolume', BSPLINE_T2_TO_RIPB0, 'fixedVolume')
DWIWorkflow.connect(inputsSpec, 'T2Volume', BSPLINE_T2_TO_RIPB0, 'movingVolume')
EXTRACT_B0 = pe.Node(interface=extractNrrdVectorIndex(),name="EXTRACT_B0")
EXTRACT_B0.inputs.vectorIndex = 0
EXTRACT_B0.inputs.outputVolume = 'B0_Image.nrrd'
DWIWorkflow.connect(DWIRIP,'outputVolume',EXTRACT_B0,'inputVolume')
RESAMPLE_BRAINMASK = pe.Node(interface=BRAINSResample(), name="RESAMPLE_BRAINMASK")
RESAMPLE_BRAINMASK.inputs.interpolationMode = 'NearestNeighbor' # This needs to be debugged'Binary'
RESAMPLE_BRAINMASK.inputs.outputVolume = 'DeformedBrainMaskDWIRIP.nrrd'
RESAMPLE_BRAINMASK.inputs.pixelType = 'uchar'

DWIWorkflow.connect(BSPLINE_T2_TO_RIPB0,'bsplineTransform',RESAMPLE_BRAINMASK,'warpTransform')
DWIWorkflow.connect(inputsSpec, 'BrainMask',RESAMPLE_BRAINMASK,'inputVolume')
DWIWorkflow.connect(EXTRACT_B0,'outputVolume',RESAMPLE_BRAINMASK,'referenceVolume')
DTIEstim = pe.Node(interface=dtiestim(), name="DTIEstim_Process")
DTIEstim.inputs.method = "wls"
DTIEstim.inputs.tensor_output = 'DTI_Output.nrrd'
DWIWorkflow.connect(DWIRIP, 'outputVolume', DTIEstim, 'dwi_image')
DWIWorkflow.connect(RESAMPLE_BRAINMASK, 'outputVolume', DTIEstim, 'brain_mask')
DTIProcess = pe.Node(interface=dtiprocess(), name="DTIProcess")
DTIProcess.inputs.fa_output = "FA.nrrd"
DTIProcess.inputs.md_output = "MD.nrrd"
DTIProcess.inputs.RD_output = "RD.nrrd"
DTIProcess.inputs.frobenius_norm_output = "frobenius_norm_output.nrrd"
DTIProcess.inputs.lambda1_output = "lambda1_output.nrrd"
DTIProcess.inputs.lambda2_output = "lambda2_output.nrrd"
DTIProcess.inputs.lambda3_output = "lambda3_output.nrrd"
DTIProcess.inputs.scalar_float = True

DWIWorkflow.connect(DTIEstim,'tensor_output',DTIProcess,'dti_image')

DWIWorkflow.connect(DTIProcess,'fa_output',outputsSpec,'FAImage')

return DWIWorkflow

This cell is to provide information specific to the subjects that need to be processed in our lab.

SESSIONS_TO_PROCESS=[('PHD_024','0003','42245'),('PHD_024','0005','33071'),('PHD_024','0029','84091'), ('PHD_024','0091','60387'),('PHD_024','0091','78867'),('PHD_024','0093','50120'), ('PHD_024','0093','60307'),('PHD_024','0093','88775'),('PHD_024','0122','42742'), ('PHD_024','0122','63892'),('PHD_024','0131','76658'),('PHD_024','0131','90863'), ('PHD_024','0132','38235'),('PHD_024','0132','43991'),('PHD_024','0132','74443'), ('PHD_024','0133','63793'),('PHD_024','0133','81826'),('PHD_024','0137','11834'), ('PHD_024','0138','84460'),('PHD_024','0140','31352')] BASE_STRUCT = '/paulsen/Experiments/20130202_PREDICTHD_Results' BASE_DWI ='/paulsen/Experiments/20130211_DWICONCAT'

THE FOLLOWING IS WHERE THE DESIRED BEHAVIOR IS NOT OCCURING PROPERLY

=====================================================================

THE FOLLOWING IS WHERE THE DESIRED BEHAVIOR IS NOT OCCURING PROPERLY

=====================================================================

THE FOLLOWING IS WHERE THE DESIRED BEHAVIOR IS NOT OCCURING PROPERLY

=====================================================================

THE FOLLOWING IS WHERE THE DESIRED BEHAVIOR IS NOT OCCURING PROPERLY

=====================================================================

The desired behavior would be to use a MapNode here, but I can not get the linkage correct.

MasterWFname="ManySubjectDWIPrototype" MasterDWIWorkflow = pe.Workflow(name=MasterWFname)

BASE_DIR = os.path.join('/hjohnson/HDNI/20130214_DWIPROCESSING_NIPYPE/JOY_DWI_CACHE/',"MasterWFname") MasterDWIWorkflow.base_dir = BASE_DIR

MasterDWIWorkflow.config['execution'] = { 'plugin': 'Linear',

    #'stop_on_first_crash':'true',
    #'stop_on_first_rerun': 'true',
    'stop_on_first_crash': 'false',
    'stop_on_first_rerun': 'false',  # This stops at first attempt to rerun, before running, and before deleting previous results.
    'hash_method': 'timestamp',
    'single_thread_matlab': 'true',  # Multi-core 2011a  multi-core for matrix multiplication.
    'remove_unnecessary_outputs': 'false',
    'use_relative_paths': 'false',  # relative paths should be on, require hash update when changed.
    'remove_node_directories': 'false',  # Experimental
    'local_hash_check': 'true',
    'job_finished_timeout': 45
}

MasterDWIWorkflow.config['logging'] = { 'workflow_level': 'DEBUG', 'filemanip_level': 'DEBUG', 'interface_level': 'DEBUG', 'log_directory': BASE_DIR }

if 0:

## I can't figure out how to make this map node work, so resorting to a stupid for loop
GetFileNamesNode = pe.MapNode(interface=Function(function=GetDWIReferenceImagesFromSessionID,
                        input_names=['SESSION_TUPLE','BASE_STRUCT','BASE_DWI'],
                        output_names=['FixImage','FixMaskImage','MovingDWI']),
                        iterfield = ['SESSION_TUPLE'],
                        run_without_submitting=True, name="99_GetDWIReferenceImagesFromSessionID")

GetFileNamesNode.inputs.SESSION_TUPLE=SESSIONS_TO_PROCESS
GetFileNamesNode.inputs.BASE_STRUCT = BASE_STRUCT
GetFileNamesNode.inputs.BASE_DWI = BASE_DWI

ID_interface =  pe.Node(interface=IdentityInterface(fields=['T2Volume', 'DWIVolume','BrainMask'],
                    iterfields=['T2Volume', 'DWIVolume','BrainMask']),
                    name='ID_interface')

MasterDWIWorkflow.connect(GetFileNamesNode,'FixImage',ID_interface,'T2Volume')
MasterDWIWorkflow.connect(GetFileNamesNode,'FixMaskImage',ID_interface,'BrainMask')
MasterDWIWorkflow.connect(GetFileNamesNode,'MovingDWI',ID_interface,'DWIVolume')

MasterDWIWorkflow.connect(ID_interface,'T2Volume',DWIWorkflow,'inputspec.T2Volume')
MasterDWIWorkflow.connect(ID_interface,'BrainMask',DWIWorkflow,'inputspec.BrainMask')
MasterDWIWorkflow.connect(ID_interface,'DWIVolume',DWIWorkflow,'inputspec.DWIVolume')

else: all_workflows = dict() GetFileNamesNode =dict()

length_of_sessions = len(SESSIONS_TO_PROCESS)
MergeFAsNode = pe.Node(interface=Merge( length_of_sessions ),
                       run_without_submitting=True,
                       name="MergeFAsNode")
index = 1
for singleSession in SESSIONS_TO_PROCESS:
    localProjectID=singleSession[0]
    localSubjectID=singleSession[1]
    localSessionID=singleSession[2]
    all_workflows[localSessionID] = CreateDWIWorkFlow(localProjectID,localSubjectID,localSessionID)
    GetFileNamesNodeName='99_GetDWIReferenceImagesFromSessionID_'+str(localSessionID)
    GetFileNamesNode[localSessionID] = pe.Node(interface=Function(function=GetDWIReferenceImagesFromSessionID,
                        input_names=['SESSION_TUPLE','BASE_STRUCT','BASE_DWI'],
                        output_names=['FixImage','FixMaskImage','MovingDWI']),
                        run_without_submitting=True, name=GetFileNamesNodeName)
    GetFileNamesNode[localSessionID].inputs.SESSION_TUPLE = singleSession
    GetFileNamesNode[localSessionID].inputs.BASE_STRUCT = BASE_STRUCT
    GetFileNamesNode[localSessionID].inputs.BASE_DWI = BASE_DWI

    MasterDWIWorkflow.connect(GetFileNamesNode[localSessionID],'FixImage',all_workflows[localSessionID],'inputspec.T2Volume')
    MasterDWIWorkflow.connect(GetFileNamesNode[localSessionID],'FixMaskImage',all_workflows[localSessionID],'inputspec.BrainMask')
    MasterDWIWorkflow.connect(GetFileNamesNode[localSessionID],'MovingDWI',all_workflows[localSessionID],'inputspec.DWIVolume')

    currentIn='in'+str(index)
    index += 1
    MasterDWIWorkflow.connect(all_workflows[localSessionID],'outputspec.FAImage',MergeFAsNode,currentIn)


## Now do template building with FA's
import nipype.interfaces.ants as ants

initAvg = pe.Node(interface=ants.AverageImages(), name ='initAvg')
initAvg.inputs.dimension = 3
initAvg.inputs.normalize = True
MasterDWIWorkflow.connect(MergeFAsNode, "out", initAvg, "images")

CLUSTER_QUEUE_LONG = '-q OSX'
CLUSTER_QUEUE= '-q OSX'


def MergeByExtendListElements(FAImageList):
    ## Initial list with empty dictionaries

    ListOfImagesDictionaries=list()
    for ff in FAImageList:
        ListOfImagesDictionaries.append( { 'FA': ff, 'DUMMY': ff } )

    ## HACK:  Need to make it so that AVG_AIR.nii.gz is has a background value of 1
    registrationImageTypes = ['FA']  # ['T1','T2'] someday.
    # DefaultContinuousInterpolationType='LanczosWindowedSinc' ## Could also be Linear for speed.
    DefaultContinuousInterpolationType = 'Linear'
    interpolationMapping = {'T1': DefaultContinuousInterpolationType,
                            'T2': DefaultContinuousInterpolationType,
                            'PD': DefaultContinuousInterpolationType,
                            'FL': DefaultContinuousInterpolationType,
                            'FA': DefaultContinuousInterpolationType,
                            'DUMMY': DefaultContinuousInterpolationType,
                            'BRAINMASK': 'MultiLabel'}
    return ListOfImagesDictionaries,registrationImageTypes,interpolationMapping

MergeByExtendListElementsNode = pe.Node(interface=Function(function = MergeByExtendListElements, input_names=['FAImageList'],
                                                output_names=['ListOfImagesDictionaries', 'registrationImageTypes', 'interpolationMapping']),
                                                run_without_submitting=True, name="99_FAMergeByExtendListElements")

MasterDWIWorkflow.connect(MergeFAsNode, "out",MergeByExtendListElementsNode,'FAImageList')

### USE ANTS REGISTRATION
# from nipype.workflows.smri.ants import antsRegistrationTemplateBuildSingleIterationWF
from BAWantsRegistrationBuildTemplate import BAWantsRegistrationTemplateBuildSingleIterationWF
buildTemplateIteration1 = BAWantsRegistrationTemplateBuildSingleIterationWF('iteration01')
## TODO:  Change these parameters
BeginANTS_iter1 = buildTemplateIteration1.get_node("BeginANTS")
BeginANTS_iter1.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 4-8 -l mem_free=9000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE_LONG), 'overwrite': True}
wimtdeformed_iter1 = buildTemplateIteration1.get_node("wimtdeformed")
wimtdeformed_iter1.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1-2 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}
AvgAffineTransform_iter1 = buildTemplateIteration1.get_node("AvgAffineTransform")
AvgAffineTransform_iter1.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}
wimtPassivedeformed_iter1 = buildTemplateIteration1.get_node("wimtPassivedeformed")
wimtPassivedeformed_iter1.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1-2 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}

MasterDWIWorkflow.connect(initAvg, 'output_average_image', buildTemplateIteration1, 'inputspec.fixed_image')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'ListOfImagesDictionaries', buildTemplateIteration1, 'inputspec.ListOfImagesDictionaries')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'registrationImageTypes', buildTemplateIteration1, 'inputspec.registrationImageTypes')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'interpolationMapping', buildTemplateIteration1, 'inputspec.interpolationMapping')

buildTemplateIteration2 = buildTemplateIteration1.clone(name='buildTemplateIteration2')
buildTemplateIteration2 = BAWantsRegistrationTemplateBuildSingleIterationWF('Iteration02')
## TODO:  Change these parameters
BeginANTS_iter2 = buildTemplateIteration2.get_node("BeginANTS")
BeginANTS_iter2.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 4-8 -l mem_free=9000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE_LONG), 'overwrite': True}
wimtdeformed_iter2 = buildTemplateIteration2.get_node("wimtdeformed")
wimtdeformed_iter2.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1-2 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}
AvgAffineTransform_iter2 = buildTemplateIteration2.get_node("AvgAffineTransform")
AvgAffineTransform_iter2.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}
wimtPassivedeformed_iter2 = buildTemplateIteration2.get_node("wimtPassivedeformed")
wimtPassivedeformed_iter2.plugin_args = {'template': SGE_JOB_SCRIPT, 'qsub_args': '-S /bin/bash -cwd -pe smp1 1-2 -l mem_free=2000M -o /dev/null -e /dev/null {QUEUE_OPTIONS}'.format(QUEUE_OPTIONS=CLUSTER_QUEUE), 'overwrite': True}

MasterDWIWorkflow.connect(buildTemplateIteration1, 'outputspec.template', buildTemplateIteration2, 'inputspec.fixed_image')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'ListOfImagesDictionaries', buildTemplateIteration2, 'inputspec.ListOfImagesDictionaries')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'registrationImageTypes', buildTemplateIteration2, 'inputspec.registrationImageTypes')
MasterDWIWorkflow.connect(MergeByExtendListElementsNode, 'interpolationMapping', buildTemplateIteration2, 'inputspec.interpolationMapping')

Now Start Processing

====================

import multiprocessing total_CPUS = multiprocessing.cpu_count() NUMPARALLEL=1 os.environ['NSLOTS'] = "{0}".format(total_CPUS / NUMPARALLEL)

MasterDWIWorkflow.write_graph()

SGEFlavor = 'SGE'

if False: MasterDWIWorkflow.run() else: MasterDWIWorkflow.run(plugin=SGEFlavor, plugin_args=dict(template=JOB_SCRIPT, qsub_args="-S /bin/bash -cwd -pe smp1 1-12 -l h_vmem=19G,mem_free=2G -o /dev/null -e /dev/null " + "-q OSX"))

print sys.argv print sys.api_version

Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.