Spatial Images are images which use their channels to encode spatial information. These are the natural output of most depth cameras, and also have uses in establishing correspondence between shapes.

Currently, only 2D spatial images (i, j)are supported. This covers must use cases, outside of volumetric alignment. Such support could be added at a later date.

There are two kinds of spatial image that we support:

ShapeImage (i, j, <x,y,z>)
- 2D image where each pixel encodes a 3D (x,y,z) point in 3 channels.
- Useful as shape can be fully described - no ambiguity in (x,y) units
- Higher storage requirements, potential redundency in data as (x,y) be inferred from (i,j)

DepthImage (i, j, <z>)
- 2D image where each pixel encodes a depth z point in a single channel
- 3x lighter memory requirement
- Ambiguous scale on inferred (x,y)
- Images cannot be warped into same reference space to establish correspondence

Both of these classes provide the same functionality, largely encoded in their shared superclass, AbstractSpatialImage:

AbstractSpatialImage
- Direct subclass of MaskedNDImage, so supports landmarks, masking, gradients, cropping, viewing, and vectorization as usual
- has mesh property which is a TriMesh instance visualizing the spatial data.
  - points used in .mesh are taken from the masked region of the image only
  - connectivity can be provided as a trilist kwarg - if not is automatically generated using Delaunay on the points
  - if a texture kwarg is provided, mesh will instead be a TexturedTriMesh
  - tcoords kwarg can be used to provide texture coordinates - if none are provided they are generated using the assumption that there is a 1:1 mapping between texture and the spatial image

note that .mesh is generated once on creation of the spatial image. If you want to alter the spatial image and see how the mesh changes, you will need to construct a fresh object from the changed state.

The easiest way to build spatial images is using any of the import tools from the menpo.io package. As we are only importing individual images, here we use the import_image(filepath) function. Refer to the specific importers for details of how the data is extracted from the files.



In [ ]:

    
import menpo.io as pio
abs_path = '/vol/atlas/databases/frgc/fall2003/02463d546.abs'
bnt_path = '/vol/hci2/Databases/video/Bosphorus/BosphorusDB/bs000/bs000_CR_RD_0.bnt'
abs_image = pio.import_image(abs_path)
bnt_image = pio.import_image(bnt_path)

Remember that spatial images are just normal MaskedNDImages



In [ ]:

    
print bnt_image
print bnt_image.mesh

But they have special support for viewing. By default, you get an image as normal. You can explicitly get this with mode='image' too.



In [ ]:

    
%matplotlib inline
abs_image.view()

As with all images, you can use the channel kwarg to see one channel in isolation



In [ ]:

    
%matplotlib inline
abs_image.view(channels=2)  # just the z values

Pass mode='mesh' to see the mesh



In [ ]:

    
%matplotlib wx
abs_image.view(mode='mesh')

Pass mode='height' to see the z height point cloud



In [ ]:

    
%matplotlib wx
abs_image.view(mode='height')

The mesh is a full blown TriMesh instance, in this case a TexturedTriMesh



In [ ]:

    
print abs_image.mesh

Calculating per-face normals is trivial and fast



In [ ]:

    
%timeit abs_image.mesh.face_normals

The texture is also accessable on the .mesh at .mesh.texture



In [ ]:

    
%matplotlib inline
texture = abs_image.mesh.texture
print texture
texture.view()