Images

SimpleITK conventions:

Image access is in x,y,z order, image.GetPixel(x,y,z) or image[x,y,z], with zero based indexing.
If the output of an ITK filter has non-zero starting index, then the index will be set to 0, and the origin adjusted accordingly.

The unique feature of SimpleITK (derived from ITK) as a toolkit for image manipulation and analysis is that it views images as physical objects occupying a bounded region in physical space. In addition images can have different spacing between pixels along each axis, and the axes are not necessarily orthogonal. The following figure illustrates these concepts.

Pixel Types

The pixel type is represented as an enumerated type. The following is a table of the enumerated list.

sitkUInt8	Unsigned 8 bit integer
sitkInt8	Signed 8 bit integer
sitkUInt16	Unsigned 16 bit integer
sitkInt16	Signed 16 bit integer
sitkUInt32	Unsigned 32 bit integer
sitkInt32	Signed 32 bit integer
sitkUInt64	Unsigned 64 bit integer
sitkInt64	Signed 64 bit integer
sitkFloat32	32 bit float
sitkFloat64	64 bit float
sitkComplexFloat32	complex number of 32 bit float
sitkComplexFloat64	complex number of 64 bit float
sitkVectorUInt8	Multi-component of unsigned 8 bit integer
sitkVectorInt8	Multi-component of signed 8 bit integer
sitkVectorUInt16	Multi-component of unsigned 16 bit integer
sitkVectorInt16	Multi-component of signed 16 bit integer
sitkVectorUInt32	Multi-component of unsigned 32 bit integer
sitkVectorInt32	Multi-component of signed 32 bit integer
sitkVectorUInt64	Multi-component of unsigned 64 bit integer
sitkVectorInt64	Multi-component of signed 64 bit integer
sitkVectorFloat32	Multi-component of 32 bit float
sitkVectorFloat64	Multi-component of 64 bit float
sitkLabelUInt8	RLE label of unsigned 8 bit integers
sitkLabelUInt16	RLE label of unsigned 16 bit integers
sitkLabelUInt32	RLE label of unsigned 32 bit integers
sitkLabelUInt64	RLE label of unsigned 64 bit integers

There is also sitkUnknown, which is used for undefined or erroneous pixel ID's. It has a value of -1.

The 64-bit integer types are not available on all distributions. When not available the value is sitkUnknown.



In [ ]:

    
import SimpleITK as sitk

from __future__ import print_function

import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

from ipywidgets import interact, fixed
import os

OUTPUT_DIR = 'Output'

# Utility method that either downloads data from the MIDAS repository or
# if already downloaded returns the file name for reading from disk (cached data).
from downloaddata import fetch_data as fdata

Load your first image and display it



In [ ]:

    
logo = sitk.ReadImage(fdata('SimpleITK.jpg'))

plt.imshow(sitk.GetArrayFromImage(logo))
plt.axis('off');

Image Construction

There are a variety of ways to create an image.

The following components are required for a complete definition of an image:

Pixel type [fixed on creation, no default]: unsigned 32 bit integer, sitkVectorUInt8, etc., see list above.
Sizes [fixed on creation, no default]: number of pixels/voxels in each dimension. This quantity implicitly defines the image dimension.
Origin [default is zero]: coordinates of the pixel/voxel with index (0,0,0) in physical units (i.e. mm).
Spacing [default is one]: Distance between adjacent pixels/voxels in each dimension given in physical units.
Direction matrix [default is identity]: mapping, rotation, between direction of the pixel/voxel axes and physical directions.

Initial pixel/voxel values are set to zero.



In [ ]:

    
image_3D = sitk.Image(256, 128, 64, sitk.sitkInt16)
image_2D = sitk.Image(64, 64, sitk.sitkFloat32)
image_2D = sitk.Image([32,32], sitk.sitkUInt32)
image_RGB = sitk.Image([128,64], sitk.sitkVectorUInt8, 3)

Basic Image Attributes

You can change the image origin, spacing and direction. Making such changes to an image already containing data should be done cautiously.



In [ ]:

    
image_3D.SetOrigin((78.0, 76.0, 77.0))
image_3D.SetSpacing([0.5,0.5,3.0])

print(image_3D.GetOrigin())
print(image_3D.GetSize())
print(image_3D.GetSpacing())
print(image_3D.GetDirection())

Image dimension queries:



In [ ]:

    
print(image_3D.GetDimension())
print(image_3D.GetWidth())
print(image_3D.GetHeight())
print(image_3D.GetDepth())

What is the depth of a 2D image?



In [ ]:

    
print(image_2D.GetSize())
print(image_2D.GetDepth())

Pixel/voxel type queries:



In [ ]:

    
print(image_3D.GetPixelIDValue())
print(image_3D.GetPixelIDTypeAsString())
print(image_3D.GetNumberOfComponentsPerPixel())

What is the dimension and size of a Vector image and its data?



In [ ]:

    
print(image_RGB.GetDimension())
print(image_RGB.GetSize())
print(image_RGB.GetNumberOfComponentsPerPixel())

Accessing Pixels and Slicing

The Image class's member functions GetPixel and SetPixel provide an ITK-like interface for pixel access.



In [ ]:

    
help(image_3D.GetPixel)



In [ ]:

    
print(image_3D.GetPixel(0, 0, 0))
image_3D.SetPixel(0, 0, 0, 1)
print(image_3D.GetPixel(0, 0, 0))

# This can also be done using pythonic notation.
print(image_3D[0,0,1])
image_3D[0,0,1] = 2
print(image_3D[0,0,1])

Slicing of SimpleITK images returns a copy of the image data.

This is similar to slicing Python lists and differs from the "view" returned by slicing numpy arrays.



In [ ]:

    
# Brute force subsampling 
logo_subsampled = logo[::2,::2]

# Get the sub-image containing the word Simple
simple = logo[0:115,:]

# Get the sub-image containing the word Simple and flip it
simple_flipped = logo[115:0:-1,:]

n = 4

plt.subplot(n,1,1)
plt.imshow(sitk.GetArrayFromImage(logo))
plt.axis('off');

plt.subplot(n,1,2)
plt.imshow(sitk.GetArrayFromImage(logo_subsampled))
plt.axis('off');

plt.subplot(n,1,3)
plt.imshow(sitk.GetArrayFromImage(simple))
plt.axis('off')

plt.subplot(n,1,4)
plt.imshow(sitk.GetArrayFromImage(simple_flipped))
plt.axis('off');

Draw a square on top of the logo image: After running this cell, uncomment "Version 3" and see its effect.



In [ ]:

    
# Version 0: get the numpy array and assign the value via broadcast - later on you will need to construct 
# a new image from the array 
logo_pixels = sitk.GetArrayFromImage(logo)
logo_pixels[0:10,0:10] = [0,255,0]

# Version 1: generates an error, the image slicing returns a new image and you cannot assign a value to an image 
#logo[0:10,0:10] = [255,0,0]

# Version 2: image slicing returns a new image, so all assignments here will not have any effect on the original
# 'logo' image
logo_subimage = logo[0:10, 0:10]
for x in range(0,10):
    for y in range(0,10):
        logo_subimage[x,y] = [255,0,0]

# Version 3: modify the original image, iterate and assing a value to each pixel
#for x in range(0,10):
#    for y in range(0,10):
#        logo[x,y] = [255,0,0]

        
plt.subplot(2,1,1)
plt.imshow(sitk.GetArrayFromImage(logo))
plt.axis('off')

plt.subplot(2,1,2)
plt.imshow(logo_pixels)
plt.axis('off');

Conversion between numpy and SimpleITK

SimpleITK and numpy indexing access is in opposite order!

SimpleITK: image[x,y,z]
numpy: image_numpy_array[z,y,x]

From SimpleITK to numpy



In [ ]:

    
nda = sitk.GetArrayFromImage(image_3D)
print(image_3D.GetSize())
print(nda.shape)

nda = sitk.GetArrayFromImage(image_RGB)
print(image_RGB.GetSize())
print(nda.shape)

From numpy to SimpleITK

Remember to to set the image's origin, spacing, and possibly direction cosine matrix. The default values may not match the physical dimensions of your image.



In [ ]:

    
nda = np.zeros((10,20,3))

        #if this is supposed to be a 3D gray scale image [x=3, y=20, z=10]
img = sitk.GetImageFromArray(nda)
print(img.GetSize())

      #if this is supposed to be a 2D color image [x=20,y=10]
img = sitk.GetImageFromArray(nda, isVector=True)
print(img.GetSize())

Image operations

SimpleITK supports basic arithmetic operations between images, taking into account their physical space.

Repeatedly run this cell. Fix the error (comment out the SetDirection, then SetSpacing). Why doesn't the SetOrigin line cause a problem? How close do two physical attributes need to be in order to be considered equivalent?



In [ ]:

    
img1 = sitk.Image(24,24, sitk.sitkUInt8)
img1[0,0] = 0

img2 = sitk.Image(img1.GetSize(), sitk.sitkUInt8)
img2.SetDirection([0,1,0.5,0.5])
img2.SetSpacing([0.5,0.8])
img2.SetOrigin([0.000001,0.000001])
img2[0,0] = 255

img3 = img1 + img2
print(img3[0,0])

Reading and Writing

SimpleITK can read and write images stored in a single file, or a set of files (e.g. DICOM series).

Images stored in the DICOM format have a meta-data dictionary associated with them, which is populated with the DICOM tags. When a DICOM series is read as a single image, the meta-data information is not available since DICOM tags are specific to a file. If you need the meta-data, access the dictionary for each file by reading them seperately.

In the following cell, we read an image in JPEG format, and write it as PNG and BMP. File formats are deduced from the file extension. Appropriate pixel type is also set - you can override this and force a pixel type of your choice.



In [ ]:

    
img = sitk.ReadImage(fdata('SimpleITK.jpg'))
print(img.GetPixelIDTypeAsString())

# write as PNG and BMP
sitk.WriteImage(img, os.path.join(OUTPUT_DIR, 'SimpleITK.png'))
sitk.WriteImage(img, os.path.join(OUTPUT_DIR, 'SimpleITK.bmp'))

Read an image in JPEG format and cast the pixel type according to user selection.



In [ ]:

    
# Several pixel types, some make sense in this case (vector types) and some are just show
# that the user's choice will force the pixel type even when it doesn't make sense.
pixel_types = { 'sitkUInt8': sitk.sitkUInt8,
                'sitkUInt16' : sitk.sitkUInt16,
                'sitkFloat64' : sitk.sitkFloat64,
                'sitkVectorUInt8' : sitk.sitkVectorUInt8,
                'sitkVectorUInt16' : sitk.sitkVectorUInt16,
                'sitkVectorFloat64' : sitk.sitkVectorFloat64}

def pixel_type_dropdown_callback(pixel_type, pixel_types_dict):
    #specify the file location and the pixel type we want
    img = sitk.ReadImage(fdata('SimpleITK.jpg'), pixel_types_dict[pixel_type])
    
    print(img.GetPixelIDTypeAsString())
    print(img[0,0])
    plt.imshow(sitk.GetArrayFromImage(img))
    plt.axis('off')
 
interact(pixel_type_dropdown_callback, pixel_type=pixel_types.keys(), pixel_types_dict=fixed(pixel_types));

Read a DICOM series and write it as a single mha file



In [ ]:

    
data_directory = os.path.dirname(fdata("CIRS057A_MR_CT_DICOM/readme.txt"))
series_ID = '1.2.840.113619.2.290.3.3233817346.783.1399004564.515'

# Get the list of files belonging to a specific series ID.
reader = sitk.ImageSeriesReader()
# Use the functional interface to read the image series.
original_image = sitk.ReadImage(reader.GetGDCMSeriesFileNames(data_directory, series_ID))

# Write the image.
output_file_name_3D = os.path.join(OUTPUT_DIR, '3DImage.mha')
sitk.WriteImage(original_image, output_file_name_3D)

# Read it back again.
written_image = sitk.ReadImage(output_file_name_3D)

# Check that the original and written image are the same.
statistics_image_filter = sitk.StatisticsImageFilter()
statistics_image_filter.Execute(original_image - written_image)

# Check that the original and written files are the same
print('Max, Min differences are : {0}, {1}'.format(statistics_image_filter.GetMaximum(), statistics_image_filter.GetMinimum()))

Write an image series as JPEG. The WriteImage function receives a volume and a list of images names and writes the volume according to the z axis. For a displayable result we need to rescale the image intensities (default is [0,255]) since the JPEG format requires a cast to the UInt8 pixel type.



In [ ]:

    
sitk.WriteImage(sitk.Cast(sitk.RescaleIntensity(written_image), sitk.sitkUInt8), 
                [os.path.join(OUTPUT_DIR, 'slice{0:03d}.jpg'.format(i)) for i in range(written_image.GetSize()[2])])

Select a specific DICOM series from a directory and only then load user selection.



In [ ]:

    
data_directory = os.path.dirname(fdata("CIRS057A_MR_CT_DICOM/readme.txt"))
# Global variable 'selected_series' is updated by the interact function
selected_series = ''
def DICOM_series_dropdown_callback(series_to_load, series_dictionary):
    global selected_series
               # Print some information about the series from the meta-data dictionary
              # DICOM standard part 6, Data Dictionary: http://medical.nema.org/medical/dicom/current/output/pdf/part06.pdf
    img = sitk.ReadImage(series_dictionary[series_to_load][0])
    tags_to_print = {'0010|0010': 'Patient name: ', 
                     '0008|0060' : 'Modality: ',
                     '0008|0021' : 'Series date: ',
                     '0008|0080' : 'Institution name: ',
                     '0008|1050' : 'Performing physician\'s name: '}
    for tag in tags_to_print:
        try:
            print(tags_to_print[tag] + img.GetMetaData(tag))
        except: # Ignore if the tag isn't in the dictionary
            pass
    selected_series = series_to_load                    

# Directory contains multiple DICOM studies/series, store
             # in dictionary with key being the seriesID
reader = sitk.ImageSeriesReader()
series_file_names = {}
series_IDs = reader.GetGDCMSeriesIDs(data_directory)
            # Check that we have at least one series
if series_IDs:
    for series in series_IDs:
        series_file_names[series] = reader.GetGDCMSeriesFileNames(data_directory, series)
    
    interact(DICOM_series_dropdown_callback, series_to_load=series_IDs, series_dictionary=fixed(series_file_names)); 
else:
    print('Data directory does not contain any DICOM series.')



In [ ]:

    
reader.SetFileNames(series_file_names[selected_series])
img = reader.Execute()
npa = sitk.GetArrayFromImage(img)
# Display the image slice from the middle of the stack, z axis
z = img.GetDepth()/2
plt.imshow(sitk.GetArrayFromImage(img)[z,:,:], cmap=plt.cm.Greys_r)
plt.axis('off');

Image Display

While SimpleITK does not do visualization, it does contain a built in Show method. This function writes the image out to disk and than launches a program for visualization. By default it is configured to use ImageJ, because it readily supports many medical image formats and loads quickly. However, the Show visualization program is easily customizable by an enviroment variable:

SITK_SHOW_COMMAND: Viewer to use (ITK-SNAP, 3D Slicer...)
SITK_SHOW_COLOR_COMMAND: Viewer to use when displaying color images.
SITK_SHOW_3D_COMMAND: Viewer to use for 3D images.



In [ ]:

    
sitk.Show?

By converting into a numpy array, matplotlib can be used for visualization for integration into the scientifc python enviroment. This is good for illustrative purposes, but is problematic when working with images that have a high dynamic range or non-isotropic spacing - most 3D medical images.

When working with medical images it is recommended to visualize them using dedicated software such as the freely available 3D Slicer or ITK-SNAP.



In [ ]:

    
mr_image = sitk.ReadImage(fdata('training_001_mr_T1.mha'))
npa = sitk.GetArrayFromImage(mr_image)

# Display the image slice from the middle of the stack, z axis
z = mr_image.GetDepth()/2
npa_zslice = sitk.GetArrayFromImage(mr_image)[z,:,:]

# Three plots displaying the same data, how do we deal with the high dynamic range?
fig = plt.figure()
fig.set_size_inches(15,30)

fig.add_subplot(1,3,1)
plt.imshow(npa_zslice)
plt.title('default colormap')
plt.axis('off')

fig.add_subplot(1,3,2)
plt.imshow(npa_zslice,cmap=plt.cm.Greys_r);
plt.title('grey colormap')
plt.axis('off')

fig.add_subplot(1,3,3)
plt.title('grey colormap,\n scaling based on volumetric min and max values')
plt.imshow(npa_zslice,cmap=plt.cm.Greys_r, vmin=npa.min(), vmax=npa.max())
plt.axis('off');



In [ ]:

    
# Display the image slice in the middle of the stack, x axis
    
x = mr_image.GetWidth()/2

npa_xslice = npa[:,:,x]
plt.imshow(npa_xslice, cmap=plt.cm.Greys_r)
plt.axis('off')

print('Image spacing: {0}'.format(mr_image.GetSpacing()))



In [ ]:

    
# Collapse along the x axis
extractSliceFilter = sitk.ExtractImageFilter()     
size = list(mr_image.GetSize())
size[0] = 0
extractSliceFilter.SetSize( size )
         
index = (x, 0, 0)
extractSliceFilter.SetIndex(index)
sitk_xslice = extractSliceFilter.Execute(mr_image)

# Resample slice to isotropic
original_spacing = sitk_xslice.GetSpacing()
original_size = sitk_xslice.GetSize()

min_spacing = min(sitk_xslice.GetSpacing())
new_spacing = [min_spacing, min_spacing]
new_size = [int(round(original_size[0]*(original_spacing[0]/min_spacing))), 
            int(round(original_size[1]*(original_spacing[1]/min_spacing)))]
resampleSliceFilter = sitk.ResampleImageFilter()

# Why is the image pixelated?
sitk_isotropic_xslice = resampleSliceFilter.Execute(sitk_xslice, new_size, sitk.Transform(), sitk.sitkNearestNeighbor, sitk_xslice.GetOrigin(),
                                                    new_spacing, sitk_xslice.GetDirection(), 0, sitk_xslice.GetPixelIDValue())

plt.imshow(sitk.GetArrayFromImage(sitk_isotropic_xslice), cmap=plt.cm.Greys_r)
plt.axis('off')
print('Image spacing: {0}'.format(sitk_isotropic_xslice.GetSpacing()))

So if you really want to look at your images, use the sitk.Show command:



In [ ]:

    
try:
    sitk.Show(mr_image)
except RuntimeError:
    print('SimpleITK Show method could not find the viewer (ImageJ not installed or ' +
          'environment variable pointing to non existant viewer).')

Use a different viewer by setting environment variable(s). Do this from within your ipython notebook using 'magic' functions, or set in a more permanent manner using your OS specific convention.



In [ ]:

    
%env SITK_SHOW_COMMAND /Applications/ITK-SNAP.app/Contents/MacOS/ITK-SNAP 

try:
    sitk.Show(mr_image)
except RuntimeError:
    print('SimpleITK Show method could not find the viewer (ImageJ not installed or ' +
          'environment variable pointing to non existant viewer).')



In [ ]:

    
%env SITK_SHOW_COMMAND '/Applications/ImageJ/ImageJ.app/Contents/MacOS/JavaApplicationStub'
try:
    sitk.Show(mr_image)
except RuntimeError:
    print('SimpleITK Show method could not find the viewer (ImageJ not installed or ' +
          'environment variable pointing to non existant viewer).')



In [ ]:

    
%env SITK_SHOW_COMMAND '/Applications/Slicer.app/Contents/MacOS/Slicer'
sitk.Show(mr_image)