In the Introductory Tutorial and the Element and Container overviews you can see how HoloViews allows you to wrap your data into annotated Elements that can be composed easily into complex visualizations.

In this tutorial, we will see how all of your data can be embedded as Elements into a nested, sparsely populated multi-dimensional data structure that gives you maximum flexibility to slice, select, and combine your data for visualization and analysis. With the HoloViews data structures, you can visualize your multi-dimensional data as animations, images, charts, and parameter spaces with ease, allowing you to quickly discover the important features interactively and then prepare corresponding plots for reports or publications.

We will first start with the very powerful HoloMap container, and then show how HoloMap objects can be nested inside the other Container objects to make all of your data available easily.

To start, here are some general imports we will be using, mainly from the Python standard library:

HoloMap Basics

Python users will be familiar with dictionaries as a way to collect data together in a conveniently accessible manner. Unlike Numpy arrays, dictionaries are sparse and heterogeneous and do not have to be declared with a fixed size.

HoloMaps are a core part of HoloViews and are essential for generating animated visualizations. They also provide highly useful ways to manipulate your data for display and have several useful properties:

• HoloMaps are ordered (internally they use OrderedDictionary, or if installed, the optimized cyordereddict).
• HoloMaps let you index your data with an arbitrary number of dimensions (e.g. date and batch-number), not just one like a Python dictionary.
• The dimensions used may be simple strings, or objects recording the name, type and physical units of the dimension.
• HoloMaps let you select portions of your data by slicing each available dimension independently.
• HoloMaps also provide ways to transform your data by sampling, reducing and collapsing the data Elements.
• Dimensions in a HoloMap may be mapped onto parameter spaces for easy visualization of a portion of your multidimensional data space.

Loading data

In this notebook we will be exploring weather data from Hurricane Sandy, which swept across the Caribbean and Eastern US seaboard in late October 2012. We will scrape our data from various online sources, exploring not only how we can quickly generate animations using HoloMaps, but also how we can deal with very high-dimensional data.

We've already downloaded and cropped a number of frames of the satellite-imagery-based wind speed models from NASA and cached them on the HoloViews website. If you want to select a different cropping region or sample more frames you can find out how to get the raw data directly from NASA in this Wiki entry. For now though we'll just get the preprocessed data:

Constructing a HoloMap

Declaring Dimensions

Now that we have loaded the data we can store the raw image arrays as RGB Elements and create a HoloMap. We begin by declaring the key dimensions (kdims) of the HoloMap, which determine how the data will be stored and thus how you will be able to index and select it most easily. In this case we will index our HoloMap both by the frame number and the date:

Dimensions can be specified as a simple string, or as a Dimension object with additional information to give HoloViews some hints about how to format and display values along that Dimension.

Populating the HoloMap

Creating a HoloMap is just like creating a Python dictionary, and so you can either pass a dictionary object or a list of (key,value) pairs. The keys can each be a single value for a one-dimensional HoloMap, or tuples for multiple Dimensions.

Not only is the HoloMap constructor similar to Python dictionaries, HoloMaps also provide __getitem__, __setitem__, update, get, pop, keys, values and items just as for normal dictionaries. In addition, HoloMap provides a .clone method that will return a copy of the HoloMap containing the same data, where the data and all the parameters may now be overridden.

Basic usage and attributes on HoloMaps

A HoloMap must be uniform in the type, group, label, and key dimensions of its Elements, because it defines a parameter space of Elements varying only in their n-dimensional index and data. This also allows HoloMaps to inherit the value and label of its Elements, which we can see by inspecting the HoloMap repr() for satellite_map:

Since the RGB elements we have created are not square we can declare that RGB Elements should be displayed with an aspect ratio of 1.0 using the %opts line magic, which will apply to all subsequent cells:

To get a quick glimpse at the data we have collected, you can access the .last property, which will return the last Element in the HoloMap:

If you are unsure how large the HoloMap is or want to know a bit more about the Dimension ranges, you can use the .info property. For a HoloMap, .info will list the dimensions, their ranges for the key dimensions on the HoloMap, and even the deep_dimensions, i.e. any Dimensions contained within the Elements of the HoloMap.

Indexing and slicing HoloMaps

Having found out a bit about the HoloMap, we can look at a few frames, starting with selecting just the first three:

Because HoloMaps support all the slicing semantics including steps, we can do things like select every second frame in the second half of the animation:

As you may have noticed, the slices are not simply by whole-number index, as for a numpy array. A HoloMap, like all other Dimensioned objects (i.e., most HoloViews components), is always sliceable by the values along its key dimensions, in whatever units they are expressed.

Apart from simple slicing semantics, you can also select Elements by passing the Dimension values as a set. Since our Elements are guaranteed to be uniform, a HoloMap also allows deep indexing into the key dimensions of its Elements, allowing us to easily select a subregion of each satellite frame (where : alone means to select the entire range of that dimension)

Finally let's put together everything we've learned about indexing and go one step further. So far we've been looking at just the surface wind speed plots, but now let's combine them into a Layout. Just like Elements, HoloMaps can be grouped into a Layout using the + operator. Since the Layout is a Tree-based data structure it doesn't have any Dimensions of its own and we can't use __getitem__. Instead we may use select, which can be found on all HoloViews components. The .select method may be supplied with any number of dimension and value slice pairs. Slices may be supplied either as explicit slice objects or as tuples.

Grouping HoloMaps

HoloMaps provide the starting point to display your data in any number of ways. While HoloMap dimensions are displayed as frames of an animation by default, you can easily transform a HoloMap into another n-D component type, such as an NdLayout, GridSpace, or NdOverlay, via the .layout, .grid, and .overlay methods.

Each of these methods groups the data along the values of the dimensions you specify and return the newly grouped object. These methods are however just convenience methods around the .groupby method, which can split a HoloMap into whatever container and group types you specify.

Before we can start grouping however, we hit a snag in our indexing: the Frame and Date dimensions we specified above are redundant, because for each frame there is only one corresponding date. As a result, any groupby operation will fail. But we can easily solve this problem by reindexing the HoloMap:

Now that we have removed the redundant Dimension we can create an NdLayout indexed just by the date:

For a more compact representation, you may also create a GridSpace using the .grid method. In a GridSpace, each dimension maps onto an axis, which limits it to a maximum of two Dimensions, but redundant data like the shared axes and axis labels are suppressed. To avoid the tick labels overlapping we will also define a rotation of the tick marks by a few degrees.

Adding Dimensions

Now how do we go about combining the two HoloMaps into a single GridSpace? First let us reindex the near-surface data as well.

The two HoloMaps we have represent wind speed at different heights. Meteorologists state the height of different air masses by their pressure. The near-surface imagery is at 850 hPa, while the surface level images are at 1000 hPa.

We can add this Dimension to the HoloMaps via the add_dimension method, which accepts the new dimension, the index position at which to insert that dimension and the dimension value as arguments:

Now we can combine the two HoloMaps by creating a clone and updating it with the other HoloMap:

Using .info we can confirm the two HoloMaps have been successfully merged.

Merging multiple HoloMaps in this step-by-step way would be cumbersome, and avoiding this complexity is why the Collator object (another instance of Dimensioned) has been provided. Collator is described in the Transforming Data tutorial.

Now that both the Date and Layer Height are Dimensions on the HoloMap we have various options for laying out our data. We can simply map each Dimension to an axis of a GridSpace:

Or we can choose to animate one Dimension but not the other:

Handling missing data

Another powerful property of HoloMaps is that when combined into a Layout via the + operator, their Dimensions are coordinated across each frame. This allows you to handle missing values, because HoloViews will blank out any frames without matching dimension values when combining overlapping dimensions:

This feature becomes particularly important when combining data from different sources, which shares common dimensions but may not be sampled in precisely the same. To demonstrate this let's load some additional data.

Combining heterogenous data

With the existing timestamps we can look up weather data about different cities via the REST API provided by openweathermap.org as shown below. For the purposes of this Tutorial, we can load the data a lot quicker if we get it directly from the HoloViews website.

First we define the new dimensions we will be adding:

Now we can load the data into a HoloMap of ItemTables. ItemTables simply associate a value with each of the value dimensions we defined above. We will collect data for a few cities on the East Coast at all the timestamps associated with the satellite wind imagery. As for the satellite imagery we've prefetched this data to find out how to do that yourself just go here.

Since the two datasets share the same timestamps we can now combine them into a Layout.

The Date dimension in the satellite data and the City and Date dimensions on the weather data combined seamlessly to give us this multi-dimensional selection widget. Since the satellite data is independent of the City it stays fixed when selecting a different city, while the Date, which is present on both, controls both components of the plot. You can play with the sliders a little bit and explore the data; once you've selected a slider you can also press R and P to play an animation back and forth respectively.

Now let's put together some of what we've learned. By making use of the slicing we can zoom in for each city on the satellite imagery and place the ItemTable containing the weather data next to it. Then we'll arrange the layout in three columns by calling the .cols method:

Now that you see how to assemble your data into an organization that lets you explore and analyze it, you can study the various Container types that make this possible, especially the section on nested containers. And then just try it out!