Programmatically accessing data through THREDDS and the VDI

...using Python 3

In this notebook:

Using the Siphon Python package to programmatically access THREDDS data service endpoints
Programmatically accessing files from the VDI

The following material uses CSIRO IMOS TERN-AusCover MODIS Data Collection. For more information on the collection and licensing, please click here.

Prerequisites:

A python 3 virtual environment with the following Python modules loaded:
- matplotlib
- netcdf4
- siphon
- shapely
- requests
Some knowledge of navigating the NCI data catalogues to find a dataset. Screenshots at the start of this notebook are a useful example, although using a different dataset.

Setup instructions for python 3 virtual environments can be found here.

Import python packages



In [1]:

    
from netCDF4 import Dataset
import matplotlib.pyplot as plt 
from siphon import catalog, ncss
import datetime
%matplotlib inline

Start by defining the parent catalog URL from NCI's THREDDS Data Server

Note: Switch the '.html' ending on the URL to '.xml'



In [2]:

    
url = 'http://dapds00.nci.org.au/thredds/catalog/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/catalog.xml'

Using Siphon

Siphon is a collection of Python utilities for downloading data from Unidata data technologies. More information on installing and using Unidata's Siphon can be found: https://github.com/Unidata/siphon

Once selecting a parent dataset directory, Siphon can be used to search and use the data access methods and services provided by THREDDS. For example, Siphon will return a list of data endpoints for the OPeNDAP data URL, NetCDF Subset Service (NCSS), Web Map Service (WMS), Web Coverage Service (WCS), and the HTTP link for direct download.

In this Notebook, we'll be demonstrating the Netcdf Subset Service (NCSS).



In [4]:

    
tds = catalog.TDSCatalog(url)
datasets = list(tds.datasets)
endpts = list(tds.datasets.values())



In [5]:

    
list(tds.datasets.keys())









    Out[5]:





['FractCover.V2_2.2000.049.aust.005.nc.gz',
 'FractCover.V2_2.2000.057.aust.005.nc.gz',
 'FractCover.V2_2.2000.065.aust.005.nc.gz',
 'FractCover.V2_2.2000.073.aust.005.nc.gz',
 'FractCover.V2_2.2000.081.aust.005.nc.gz',
 'FractCover.V2_2.2000.089.aust.005.nc.gz',
 'FractCover.V2_2.2000.097.aust.005.nc.gz',
 'FractCover.V2_2.2000.105.aust.005.nc.gz',
 'FractCover.V2_2.2000.113.aust.005.nc.gz',
 'FractCover.V2_2.2000.121.aust.005.nc.gz',
 'FractCover.V2_2.2000.129.aust.005.nc.gz',
 'FractCover.V2_2.2000.137.aust.005.nc.gz',
 'FractCover.V2_2.2000.145.aust.005.nc.gz',
 'FractCover.V2_2.2000.153.aust.005.nc.gz',
 'FractCover.V2_2.2000.161.aust.005.nc.gz',
 'FractCover.V2_2.2000.169.aust.005.nc.gz',
 'FractCover.V2_2.2000.177.aust.005.nc.gz',
 'FractCover.V2_2.2000.185.aust.005.nc.gz',
 'FractCover.V2_2.2000.193.aust.005.nc.gz',
 'FractCover.V2_2.2000.201.aust.005.nc.gz',
 'FractCover.V2_2.2000.209.aust.005.nc.gz',
 'FractCover.V2_2.2000.217.aust.005.nc.gz',
 'FractCover.V2_2.2000.225.aust.005.nc.gz',
 'FractCover.V2_2.2000.233.aust.005.nc.gz',
 'FractCover.V2_2.2000.241.aust.005.nc.gz',
 'FractCover.V2_2.2000.249.aust.005.nc.gz',
 'FractCover.V2_2.2000.257.aust.005.nc.gz',
 'FractCover.V2_2.2000.265.aust.005.nc.gz',
 'FractCover.V2_2.2000.273.aust.005.nc.gz',
 'FractCover.V2_2.2000.281.aust.005.nc.gz',
 'FractCover.V2_2.2000.289.aust.005.nc.gz',
 'FractCover.V2_2.2000.297.aust.005.nc.gz',
 'FractCover.V2_2.2000.305.aust.005.nc.gz',
 'FractCover.V2_2.2000.313.aust.005.nc.gz',
 'FractCover.V2_2.2000.321.aust.005.nc.gz',
 'FractCover.V2_2.2000.329.aust.005.nc.gz',
 'FractCover.V2_2.2000.337.aust.005.nc.gz',
 'FractCover.V2_2.2000.345.aust.005.nc.gz',
 'FractCover.V2_2.2000.353.aust.005.nc.gz',
 'FractCover.V2_2.2000.361.aust.005.nc.gz',
 'FractCover.V2_2.2001.001.aust.005.nc.gz',
 'FractCover.V2_2.2001.009.aust.005.nc.gz',
 'FractCover.V2_2.2001.017.aust.005.nc.gz',
 'FractCover.V2_2.2001.025.aust.005.nc.gz',
 'FractCover.V2_2.2001.033.aust.005.nc.gz',
 'FractCover.V2_2.2001.041.aust.005.nc.gz',
 'FractCover.V2_2.2001.049.aust.005.nc.gz',
 'FractCover.V2_2.2001.057.aust.005.nc.gz',
 'FractCover.V2_2.2001.065.aust.005.nc.gz',
 'FractCover.V2_2.2001.073.aust.005.nc.gz',
 'FractCover.V2_2.2001.081.aust.005.nc.gz',
 'FractCover.V2_2.2001.089.aust.005.nc.gz',
 'FractCover.V2_2.2001.097.aust.005.nc.gz',
 'FractCover.V2_2.2001.105.aust.005.nc.gz',
 'FractCover.V2_2.2001.113.aust.005.nc.gz',
 'FractCover.V2_2.2001.121.aust.005.nc.gz',
 'FractCover.V2_2.2001.129.aust.005.nc.gz',
 'FractCover.V2_2.2001.137.aust.005.nc.gz',
 'FractCover.V2_2.2001.145.aust.005.nc.gz',
 'FractCover.V2_2.2001.153.aust.005.nc.gz',
 'FractCover.V2_2.2001.161.aust.005.nc.gz',
 'FractCover.V2_2.2001.169.aust.005.nc.gz',
 'FractCover.V2_2.2001.177.aust.005.nc.gz',
 'FractCover.V2_2.2001.185.aust.005.nc.gz',
 'FractCover.V2_2.2001.193.aust.005.nc.gz',
 'FractCover.V2_2.2001.201.aust.005.nc.gz',
 'FractCover.V2_2.2001.209.aust.005.nc.gz',
 'FractCover.V2_2.2001.217.aust.005.nc.gz',
 'FractCover.V2_2.2001.225.aust.005.nc.gz',
 'FractCover.V2_2.2001.233.aust.005.nc.gz',
 'FractCover.V2_2.2001.241.aust.005.nc.gz',
 'FractCover.V2_2.2001.249.aust.005.nc.gz',
 'FractCover.V2_2.2001.257.aust.005.nc.gz',
 'FractCover.V2_2.2001.265.aust.005.nc.gz',
 'FractCover.V2_2.2001.273.aust.005.nc.gz',
 'FractCover.V2_2.2001.281.aust.005.nc.gz',
 'FractCover.V2_2.2001.289.aust.005.nc.gz',
 'FractCover.V2_2.2001.297.aust.005.nc.gz',
 'FractCover.V2_2.2001.305.aust.005.nc.gz',
 'FractCover.V2_2.2001.313.aust.005.nc.gz',
 'FractCover.V2_2.2001.321.aust.005.nc.gz',
 'FractCover.V2_2.2001.329.aust.005.nc.gz',
 'FractCover.V2_2.2001.337.aust.005.nc.gz',
 'FractCover.V2_2.2001.345.aust.005.nc.gz',
 'FractCover.V2_2.2001.353.aust.005.nc.gz',
 'FractCover.V2_2.2001.361.aust.005.nc.gz',
 'FractCover.V2_2.2002.001.aust.005.nc.gz',
 'FractCover.V2_2.2002.009.aust.005.nc.gz',
 'FractCover.V2_2.2002.017.aust.005.nc.gz',
 'FractCover.V2_2.2002.025.aust.005.nc.gz',
 'FractCover.V2_2.2002.033.aust.005.nc.gz',
 'FractCover.V2_2.2002.041.aust.005.nc.gz',
 'FractCover.V2_2.2002.049.aust.005.nc.gz',
 'FractCover.V2_2.2002.057.aust.005.nc.gz',
 'FractCover.V2_2.2002.065.aust.005.nc.gz',
 'FractCover.V2_2.2002.073.aust.005.nc.gz',
 'FractCover.V2_2.2002.081.aust.005.nc.gz',
 'FractCover.V2_2.2002.089.aust.005.nc.gz',
 'FractCover.V2_2.2002.097.aust.005.nc.gz',
 'FractCover.V2_2.2002.105.aust.005.nc.gz',
 'FractCover.V2_2.2002.113.aust.005.nc.gz',
 'FractCover.V2_2.2002.121.aust.005.nc.gz',
 'FractCover.V2_2.2002.129.aust.005.nc.gz',
 'FractCover.V2_2.2002.137.aust.005.nc.gz',
 'FractCover.V2_2.2002.145.aust.005.nc.gz',
 'FractCover.V2_2.2002.153.aust.005.nc.gz',
 'FractCover.V2_2.2002.161.aust.005.nc.gz',
 'FractCover.V2_2.2002.169.aust.005.nc.gz',
 'FractCover.V2_2.2002.177.aust.005.nc.gz',
 'FractCover.V2_2.2002.185.aust.005.nc.gz',
 'FractCover.V2_2.2002.193.aust.005.nc.gz',
 'FractCover.V2_2.2002.201.aust.005.nc.gz',
 'FractCover.V2_2.2002.209.aust.005.nc.gz',
 'FractCover.V2_2.2002.217.aust.005.nc.gz',
 'FractCover.V2_2.2002.225.aust.005.nc.gz',
 'FractCover.V2_2.2002.233.aust.005.nc.gz',
 'FractCover.V2_2.2002.241.aust.005.nc.gz',
 'FractCover.V2_2.2002.249.aust.005.nc.gz',
 'FractCover.V2_2.2002.257.aust.005.nc.gz',
 'FractCover.V2_2.2002.265.aust.005.nc.gz',
 'FractCover.V2_2.2002.273.aust.005.nc.gz',
 'FractCover.V2_2.2002.281.aust.005.nc.gz',
 'FractCover.V2_2.2002.289.aust.005.nc.gz',
 'FractCover.V2_2.2002.297.aust.005.nc.gz',
 'FractCover.V2_2.2002.305.aust.005.nc.gz',
 'FractCover.V2_2.2002.313.aust.005.nc.gz',
 'FractCover.V2_2.2002.321.aust.005.nc.gz',
 'FractCover.V2_2.2002.329.aust.005.nc.gz',
 'FractCover.V2_2.2002.337.aust.005.nc.gz',
 'FractCover.V2_2.2002.345.aust.005.nc.gz',
 'FractCover.V2_2.2002.353.aust.005.nc.gz',
 'FractCover.V2_2.2002.361.aust.005.nc.gz',
 'FractCover.V2_2.2003.001.aust.005.nc.gz',
 'FractCover.V2_2.2003.009.aust.005.nc.gz',
 'FractCover.V2_2.2003.017.aust.005.nc.gz',
 'FractCover.V2_2.2003.025.aust.005.nc.gz',
 'FractCover.V2_2.2003.033.aust.005.nc.gz',
 'FractCover.V2_2.2003.041.aust.005.nc.gz',
 'FractCover.V2_2.2003.049.aust.005.nc.gz',
 'FractCover.V2_2.2003.057.aust.005.nc.gz',
 'FractCover.V2_2.2003.065.aust.005.nc.gz',
 'FractCover.V2_2.2003.073.aust.005.nc.gz',
 'FractCover.V2_2.2003.081.aust.005.nc.gz',
 'FractCover.V2_2.2003.089.aust.005.nc.gz',
 'FractCover.V2_2.2003.097.aust.005.nc.gz',
 'FractCover.V2_2.2003.105.aust.005.nc.gz',
 'FractCover.V2_2.2003.113.aust.005.nc.gz',
 'FractCover.V2_2.2003.121.aust.005.nc.gz',
 'FractCover.V2_2.2003.129.aust.005.nc.gz',
 'FractCover.V2_2.2003.137.aust.005.nc.gz',
 'FractCover.V2_2.2003.145.aust.005.nc.gz',
 'FractCover.V2_2.2003.153.aust.005.nc.gz',
 'FractCover.V2_2.2003.161.aust.005.nc.gz',
 'FractCover.V2_2.2003.169.aust.005.nc.gz',
 'FractCover.V2_2.2003.177.aust.005.nc.gz',
 'FractCover.V2_2.2003.185.aust.005.nc.gz',
 'FractCover.V2_2.2003.193.aust.005.nc.gz',
 'FractCover.V2_2.2003.201.aust.005.nc.gz',
 'FractCover.V2_2.2003.209.aust.005.nc.gz',
 'FractCover.V2_2.2003.217.aust.005.nc.gz',
 'FractCover.V2_2.2003.225.aust.005.nc.gz',
 'FractCover.V2_2.2003.233.aust.005.nc.gz',
 'FractCover.V2_2.2003.241.aust.005.nc.gz',
 'FractCover.V2_2.2003.249.aust.005.nc.gz',
 'FractCover.V2_2.2003.257.aust.005.nc.gz',
 'FractCover.V2_2.2003.265.aust.005.nc.gz',
 'FractCover.V2_2.2003.273.aust.005.nc.gz',
 'FractCover.V2_2.2003.281.aust.005.nc.gz',
 'FractCover.V2_2.2003.289.aust.005.nc.gz',
 'FractCover.V2_2.2003.297.aust.005.nc.gz',
 'FractCover.V2_2.2003.305.aust.005.nc.gz',
 'FractCover.V2_2.2003.313.aust.005.nc.gz',
 'FractCover.V2_2.2003.321.aust.005.nc.gz',
 'FractCover.V2_2.2003.329.aust.005.nc.gz',
 'FractCover.V2_2.2003.337.aust.005.nc.gz',
 'FractCover.V2_2.2003.345.aust.005.nc.gz',
 'FractCover.V2_2.2003.353.aust.005.nc.gz',
 'FractCover.V2_2.2003.361.aust.005.nc.gz',
 'FractCover.V2_2.2004.001.aust.005.nc.gz',
 'FractCover.V2_2.2004.009.aust.005.nc.gz',
 'FractCover.V2_2.2004.017.aust.005.nc.gz',
 'FractCover.V2_2.2004.025.aust.005.nc.gz',
 'FractCover.V2_2.2004.033.aust.005.nc.gz',
 'FractCover.V2_2.2004.041.aust.005.nc.gz',
 'FractCover.V2_2.2004.049.aust.005.nc.gz',
 'FractCover.V2_2.2004.057.aust.005.nc.gz',
 'FractCover.V2_2.2004.065.aust.005.nc.gz',
 'FractCover.V2_2.2004.073.aust.005.nc.gz',
 'FractCover.V2_2.2004.081.aust.005.nc.gz',
 'FractCover.V2_2.2004.089.aust.005.nc.gz',
 'FractCover.V2_2.2004.097.aust.005.nc.gz',
 'FractCover.V2_2.2004.105.aust.005.nc.gz',
 'FractCover.V2_2.2004.113.aust.005.nc.gz',
 'FractCover.V2_2.2004.121.aust.005.nc.gz',
 'FractCover.V2_2.2004.129.aust.005.nc.gz',
 'FractCover.V2_2.2004.137.aust.005.nc.gz',
 'FractCover.V2_2.2004.145.aust.005.nc.gz',
 'FractCover.V2_2.2004.153.aust.005.nc.gz',
 'FractCover.V2_2.2004.161.aust.005.nc.gz',
 'FractCover.V2_2.2004.169.aust.005.nc.gz',
 'FractCover.V2_2.2004.177.aust.005.nc.gz',
 'FractCover.V2_2.2004.185.aust.005.nc.gz',
 'FractCover.V2_2.2004.193.aust.005.nc.gz',
 'FractCover.V2_2.2004.201.aust.005.nc.gz',
 'FractCover.V2_2.2004.209.aust.005.nc.gz',
 'FractCover.V2_2.2004.217.aust.005.nc.gz',
 'FractCover.V2_2.2004.225.aust.005.nc.gz',
 'FractCover.V2_2.2004.233.aust.005.nc.gz',
 'FractCover.V2_2.2004.241.aust.005.nc.gz',
 'FractCover.V2_2.2004.249.aust.005.nc.gz',
 'FractCover.V2_2.2004.257.aust.005.nc.gz',
 'FractCover.V2_2.2004.265.aust.005.nc.gz',
 'FractCover.V2_2.2004.273.aust.005.nc.gz',
 'FractCover.V2_2.2004.281.aust.005.nc.gz',
 'FractCover.V2_2.2004.289.aust.005.nc.gz',
 'FractCover.V2_2.2004.297.aust.005.nc.gz',
 'FractCover.V2_2.2004.305.aust.005.nc.gz',
 'FractCover.V2_2.2004.313.aust.005.nc.gz',
 'FractCover.V2_2.2004.321.aust.005.nc.gz',
 'FractCover.V2_2.2004.329.aust.005.nc.gz',
 'FractCover.V2_2.2004.337.aust.005.nc.gz',
 'FractCover.V2_2.2004.345.aust.005.nc.gz',
 'FractCover.V2_2.2004.353.aust.005.nc.gz',
 'FractCover.V2_2.2004.361.aust.005.nc.gz',
 'FractCover.V2_2.2005.001.aust.005.nc.gz',
 'FractCover.V2_2.2005.009.aust.005.nc.gz',
 'FractCover.V2_2.2005.017.aust.005.nc.gz',
 'FractCover.V2_2.2005.025.aust.005.nc.gz',
 'FractCover.V2_2.2005.033.aust.005.nc.gz',
 'FractCover.V2_2.2005.041.aust.005.nc.gz',
 'FractCover.V2_2.2005.049.aust.005.nc.gz',
 'FractCover.V2_2.2005.057.aust.005.nc.gz',
 'FractCover.V2_2.2005.065.aust.005.nc.gz',
 'FractCover.V2_2.2005.073.aust.005.nc.gz',
 'FractCover.V2_2.2005.081.aust.005.nc.gz',
 'FractCover.V2_2.2005.089.aust.005.nc.gz',
 'FractCover.V2_2.2005.097.aust.005.nc.gz',
 'FractCover.V2_2.2005.105.aust.005.nc.gz',
 'FractCover.V2_2.2005.113.aust.005.nc.gz',
 'FractCover.V2_2.2005.121.aust.005.nc.gz',
 'FractCover.V2_2.2005.129.aust.005.nc.gz',
 'FractCover.V2_2.2005.137.aust.005.nc.gz',
 'FractCover.V2_2.2005.145.aust.005.nc.gz',
 'FractCover.V2_2.2005.153.aust.005.nc.gz',
 'FractCover.V2_2.2005.161.aust.005.nc.gz',
 'FractCover.V2_2.2005.169.aust.005.nc.gz',
 'FractCover.V2_2.2005.177.aust.005.nc.gz',
 'FractCover.V2_2.2005.185.aust.005.nc.gz',
 'FractCover.V2_2.2005.193.aust.005.nc.gz',
 'FractCover.V2_2.2005.201.aust.005.nc.gz',
 'FractCover.V2_2.2005.209.aust.005.nc.gz',
 'FractCover.V2_2.2005.217.aust.005.nc.gz',
 'FractCover.V2_2.2005.225.aust.005.nc.gz',
 'FractCover.V2_2.2005.233.aust.005.nc.gz',
 'FractCover.V2_2.2005.241.aust.005.nc.gz',
 'FractCover.V2_2.2005.249.aust.005.nc.gz',
 'FractCover.V2_2.2005.257.aust.005.nc.gz',
 'FractCover.V2_2.2005.265.aust.005.nc.gz',
 'FractCover.V2_2.2005.273.aust.005.nc.gz',
 'FractCover.V2_2.2005.281.aust.005.nc.gz',
 'FractCover.V2_2.2005.289.aust.005.nc.gz',
 'FractCover.V2_2.2005.297.aust.005.nc.gz',
 'FractCover.V2_2.2005.305.aust.005.nc.gz',
 'FractCover.V2_2.2005.313.aust.005.nc.gz',
 'FractCover.V2_2.2005.321.aust.005.nc.gz',
 'FractCover.V2_2.2005.329.aust.005.nc.gz',
 'FractCover.V2_2.2005.337.aust.005.nc.gz',
 'FractCover.V2_2.2005.345.aust.005.nc.gz',
 'FractCover.V2_2.2005.353.aust.005.nc.gz',
 'FractCover.V2_2.2005.361.aust.005.nc.gz',
 'FractCover.V2_2.2006.001.aust.005.nc.gz',
 'FractCover.V2_2.2006.009.aust.005.nc.gz',
 'FractCover.V2_2.2006.017.aust.005.nc.gz',
 'FractCover.V2_2.2006.025.aust.005.nc.gz',
 'FractCover.V2_2.2006.033.aust.005.nc.gz',
 'FractCover.V2_2.2006.041.aust.005.nc.gz',
 'FractCover.V2_2.2006.049.aust.005.nc.gz',
 'FractCover.V2_2.2006.057.aust.005.nc.gz',
 'FractCover.V2_2.2006.065.aust.005.nc.gz',
 'FractCover.V2_2.2006.073.aust.005.nc.gz',
 'FractCover.V2_2.2006.081.aust.005.nc.gz',
 'FractCover.V2_2.2006.089.aust.005.nc.gz',
 'FractCover.V2_2.2006.097.aust.005.nc.gz',
 'FractCover.V2_2.2006.105.aust.005.nc.gz',
 'FractCover.V2_2.2006.113.aust.005.nc.gz',
 'FractCover.V2_2.2006.121.aust.005.nc.gz',
 'FractCover.V2_2.2006.129.aust.005.nc.gz',
 'FractCover.V2_2.2006.137.aust.005.nc.gz',
 'FractCover.V2_2.2006.145.aust.005.nc.gz',
 'FractCover.V2_2.2006.153.aust.005.nc.gz',
 'FractCover.V2_2.2006.161.aust.005.nc.gz',
 'FractCover.V2_2.2006.169.aust.005.nc.gz',
 'FractCover.V2_2.2006.177.aust.005.nc.gz',
 'FractCover.V2_2.2006.185.aust.005.nc.gz',
 'FractCover.V2_2.2006.193.aust.005.nc.gz',
 'FractCover.V2_2.2006.201.aust.005.nc.gz',
 'FractCover.V2_2.2006.209.aust.005.nc.gz',
 'FractCover.V2_2.2006.217.aust.005.nc.gz',
 'FractCover.V2_2.2006.225.aust.005.nc.gz',
 'FractCover.V2_2.2006.233.aust.005.nc.gz',
 'FractCover.V2_2.2006.241.aust.005.nc.gz',
 'FractCover.V2_2.2006.249.aust.005.nc.gz',
 'FractCover.V2_2.2006.257.aust.005.nc.gz',
 'FractCover.V2_2.2006.265.aust.005.nc.gz',
 'FractCover.V2_2.2006.273.aust.005.nc.gz',
 'FractCover.V2_2.2006.281.aust.005.nc.gz',
 'FractCover.V2_2.2006.289.aust.005.nc.gz',
 'FractCover.V2_2.2006.297.aust.005.nc.gz',
 'FractCover.V2_2.2006.305.aust.005.nc.gz',
 'FractCover.V2_2.2006.313.aust.005.nc.gz',
 'FractCover.V2_2.2006.321.aust.005.nc.gz',
 'FractCover.V2_2.2006.329.aust.005.nc.gz',
 'FractCover.V2_2.2006.337.aust.005.nc.gz',
 'FractCover.V2_2.2006.345.aust.005.nc.gz',
 'FractCover.V2_2.2006.353.aust.005.nc.gz',
 'FractCover.V2_2.2006.361.aust.005.nc.gz',
 'FractCover.V2_2.2007.001.aust.005.nc.gz',
 'FractCover.V2_2.2007.009.aust.005.nc.gz',
 'FractCover.V2_2.2007.017.aust.005.nc.gz',
 'FractCover.V2_2.2007.025.aust.005.nc.gz',
 'FractCover.V2_2.2007.033.aust.005.nc.gz',
 'FractCover.V2_2.2007.041.aust.005.nc.gz',
 'FractCover.V2_2.2007.049.aust.005.nc.gz',
 'FractCover.V2_2.2007.057.aust.005.nc.gz',
 'FractCover.V2_2.2007.065.aust.005.nc.gz',
 'FractCover.V2_2.2007.073.aust.005.nc.gz',
 'FractCover.V2_2.2007.081.aust.005.nc.gz',
 'FractCover.V2_2.2007.089.aust.005.nc.gz',
 'FractCover.V2_2.2007.097.aust.005.nc.gz',
 'FractCover.V2_2.2007.105.aust.005.nc.gz',
 'FractCover.V2_2.2007.113.aust.005.nc.gz',
 'FractCover.V2_2.2007.121.aust.005.nc.gz',
 'FractCover.V2_2.2007.129.aust.005.nc.gz',
 'FractCover.V2_2.2007.137.aust.005.nc.gz',
 'FractCover.V2_2.2007.145.aust.005.nc.gz',
 'FractCover.V2_2.2007.153.aust.005.nc.gz',
 'FractCover.V2_2.2007.161.aust.005.nc.gz',
 'FractCover.V2_2.2007.169.aust.005.nc.gz',
 'FractCover.V2_2.2007.177.aust.005.nc.gz',
 'FractCover.V2_2.2007.185.aust.005.nc.gz',
 'FractCover.V2_2.2007.193.aust.005.nc.gz',
 'FractCover.V2_2.2007.201.aust.005.nc.gz',
 'FractCover.V2_2.2007.209.aust.005.nc.gz',
 'FractCover.V2_2.2007.217.aust.005.nc.gz',
 'FractCover.V2_2.2007.225.aust.005.nc.gz',
 'FractCover.V2_2.2007.233.aust.005.nc.gz',
 'FractCover.V2_2.2007.241.aust.005.nc.gz',
 'FractCover.V2_2.2007.249.aust.005.nc.gz',
 'FractCover.V2_2.2007.257.aust.005.nc.gz',
 'FractCover.V2_2.2007.265.aust.005.nc.gz',
 'FractCover.V2_2.2007.273.aust.005.nc.gz',
 'FractCover.V2_2.2007.281.aust.005.nc.gz',
 'FractCover.V2_2.2007.289.aust.005.nc.gz',
 'FractCover.V2_2.2007.297.aust.005.nc.gz',
 'FractCover.V2_2.2007.305.aust.005.nc.gz',
 'FractCover.V2_2.2007.313.aust.005.nc.gz',
 'FractCover.V2_2.2007.321.aust.005.nc.gz',
 'FractCover.V2_2.2007.329.aust.005.nc.gz',
 'FractCover.V2_2.2007.337.aust.005.nc.gz',
 'FractCover.V2_2.2007.345.aust.005.nc.gz',
 'FractCover.V2_2.2007.353.aust.005.nc.gz',
 'FractCover.V2_2.2007.361.aust.005.nc.gz',
 'FractCover.V2_2.2008.001.aust.005.nc.gz',
 'FractCover.V2_2.2008.009.aust.005.nc.gz',
 'FractCover.V2_2.2008.017.aust.005.nc.gz',
 'FractCover.V2_2.2008.025.aust.005.nc.gz',
 'FractCover.V2_2.2008.033.aust.005.nc.gz',
 'FractCover.V2_2.2008.041.aust.005.nc.gz',
 'FractCover.V2_2.2008.049.aust.005.nc.gz',
 'FractCover.V2_2.2008.057.aust.005.nc.gz',
 'FractCover.V2_2.2008.065.aust.005.nc.gz',
 'FractCover.V2_2.2008.073.aust.005.nc.gz',
 'FractCover.V2_2.2008.081.aust.005.nc.gz',
 'FractCover.V2_2.2008.089.aust.005.nc.gz',
 'FractCover.V2_2.2008.097.aust.005.nc.gz',
 'FractCover.V2_2.2008.105.aust.005.nc.gz',
 'FractCover.V2_2.2008.113.aust.005.nc.gz',
 'FractCover.V2_2.2008.121.aust.005.nc.gz',
 'FractCover.V2_2.2008.129.aust.005.nc.gz',
 'FractCover.V2_2.2008.137.aust.005.nc.gz',
 'FractCover.V2_2.2008.145.aust.005.nc.gz',
 'FractCover.V2_2.2008.153.aust.005.nc.gz',
 'FractCover.V2_2.2008.161.aust.005.nc.gz',
 'FractCover.V2_2.2008.169.aust.005.nc.gz',
 'FractCover.V2_2.2008.177.aust.005.nc.gz',
 'FractCover.V2_2.2008.185.aust.005.nc.gz',
 'FractCover.V2_2.2008.193.aust.005.nc.gz',
 'FractCover.V2_2.2008.201.aust.005.nc.gz',
 'FractCover.V2_2.2008.209.aust.005.nc.gz',
 'FractCover.V2_2.2008.217.aust.005.nc.gz',
 'FractCover.V2_2.2008.225.aust.005.nc.gz',
 'FractCover.V2_2.2008.233.aust.005.nc.gz',
 'FractCover.V2_2.2008.241.aust.005.nc.gz',
 'FractCover.V2_2.2008.249.aust.005.nc.gz',
 'FractCover.V2_2.2008.257.aust.005.nc.gz',
 'FractCover.V2_2.2008.265.aust.005.nc.gz',
 'FractCover.V2_2.2008.273.aust.005.nc.gz',
 'FractCover.V2_2.2008.281.aust.005.nc.gz',
 'FractCover.V2_2.2008.289.aust.005.nc.gz',
 'FractCover.V2_2.2008.297.aust.005.nc.gz',
 'FractCover.V2_2.2008.305.aust.005.nc.gz',
 'FractCover.V2_2.2008.313.aust.005.nc.gz',
 'FractCover.V2_2.2008.321.aust.005.nc.gz',
 'FractCover.V2_2.2008.329.aust.005.nc.gz',
 'FractCover.V2_2.2008.337.aust.005.nc.gz',
 'FractCover.V2_2.2008.345.aust.005.nc.gz',
 'FractCover.V2_2.2008.353.aust.005.nc.gz',
 'FractCover.V2_2.2008.361.aust.005.nc.gz',
 'FractCover.V2_2.2009.001.aust.005.nc.gz',
 'FractCover.V2_2.2009.009.aust.005.nc.gz',
 'FractCover.V2_2.2009.017.aust.005.nc.gz',
 'FractCover.V2_2.2009.025.aust.005.nc.gz',
 'FractCover.V2_2.2009.033.aust.005.nc.gz',
 'FractCover.V2_2.2009.041.aust.005.nc.gz',
 'FractCover.V2_2.2009.049.aust.005.nc.gz',
 'FractCover.V2_2.2009.057.aust.005.nc.gz',
 'FractCover.V2_2.2009.065.aust.005.nc.gz',
 'FractCover.V2_2.2009.073.aust.005.nc.gz',
 'FractCover.V2_2.2009.081.aust.005.nc.gz',
 'FractCover.V2_2.2009.089.aust.005.nc.gz',
 'FractCover.V2_2.2009.097.aust.005.nc.gz',
 'FractCover.V2_2.2009.105.aust.005.nc.gz',
 'FractCover.V2_2.2009.113.aust.005.nc.gz',
 'FractCover.V2_2.2009.121.aust.005.nc.gz',
 'FractCover.V2_2.2009.129.aust.005.nc.gz',
 'FractCover.V2_2.2009.137.aust.005.nc.gz',
 'FractCover.V2_2.2009.145.aust.005.nc.gz',
 'FractCover.V2_2.2009.153.aust.005.nc.gz',
 'FractCover.V2_2.2009.161.aust.005.nc.gz',
 'FractCover.V2_2.2009.169.aust.005.nc.gz',
 'FractCover.V2_2.2009.177.aust.005.nc.gz',
 'FractCover.V2_2.2009.185.aust.005.nc.gz',
 'FractCover.V2_2.2009.193.aust.005.nc.gz',
 'FractCover.V2_2.2009.201.aust.005.nc.gz',
 'FractCover.V2_2.2009.209.aust.005.nc.gz',
 'FractCover.V2_2.2009.217.aust.005.nc.gz',
 'FractCover.V2_2.2009.225.aust.005.nc.gz',
 'FractCover.V2_2.2009.233.aust.005.nc.gz',
 'FractCover.V2_2.2009.241.aust.005.nc.gz',
 'FractCover.V2_2.2009.249.aust.005.nc.gz',
 'FractCover.V2_2.2009.257.aust.005.nc.gz',
 'FractCover.V2_2.2009.265.aust.005.nc.gz',
 'FractCover.V2_2.2009.273.aust.005.nc.gz',
 'FractCover.V2_2.2009.281.aust.005.nc.gz',
 'FractCover.V2_2.2009.289.aust.005.nc.gz',
 'FractCover.V2_2.2009.297.aust.005.nc.gz',
 'FractCover.V2_2.2009.305.aust.005.nc.gz',
 'FractCover.V2_2.2009.313.aust.005.nc.gz',
 'FractCover.V2_2.2009.321.aust.005.nc.gz',
 'FractCover.V2_2.2009.329.aust.005.nc.gz',
 'FractCover.V2_2.2009.337.aust.005.nc.gz',
 'FractCover.V2_2.2009.345.aust.005.nc.gz',
 'FractCover.V2_2.2009.353.aust.005.nc.gz',
 'FractCover.V2_2.2009.361.aust.005.nc.gz',
 'FractCover.V2_2.2010.001.aust.005.nc.gz',
 'FractCover.V2_2.2010.009.aust.005.nc.gz',
 'FractCover.V2_2.2010.017.aust.005.nc.gz',
 'FractCover.V2_2.2010.025.aust.005.nc.gz',
 'FractCover.V2_2.2010.033.aust.005.nc.gz',
 'FractCover.V2_2.2010.041.aust.005.nc.gz',
 'FractCover.V2_2.2010.049.aust.005.nc.gz',
 'FractCover.V2_2.2010.057.aust.005.nc.gz',
 'FractCover.V2_2.2010.065.aust.005.nc.gz',
 'FractCover.V2_2.2010.073.aust.005.nc.gz',
 'FractCover.V2_2.2010.081.aust.005.nc.gz',
 'FractCover.V2_2.2010.089.aust.005.nc.gz',
 'FractCover.V2_2.2010.097.aust.005.nc.gz',
 'FractCover.V2_2.2010.105.aust.005.nc.gz',
 'FractCover.V2_2.2010.113.aust.005.nc.gz',
 'FractCover.V2_2.2010.121.aust.005.nc.gz',
 'FractCover.V2_2.2010.129.aust.005.nc.gz',
 'FractCover.V2_2.2010.137.aust.005.nc.gz',
 'FractCover.V2_2.2010.145.aust.005.nc.gz',
 'FractCover.V2_2.2010.153.aust.005.nc.gz',
 'FractCover.V2_2.2010.161.aust.005.nc.gz',
 'FractCover.V2_2.2010.169.aust.005.nc.gz',
 'FractCover.V2_2.2010.177.aust.005.nc.gz',
 'FractCover.V2_2.2010.185.aust.005.nc.gz',
 'FractCover.V2_2.2010.193.aust.005.nc.gz',
 'FractCover.V2_2.2010.201.aust.005.nc.gz',
 'FractCover.V2_2.2010.209.aust.005.nc.gz',
 'FractCover.V2_2.2010.217.aust.005.nc.gz',
 'FractCover.V2_2.2010.225.aust.005.nc.gz',
 'FractCover.V2_2.2010.233.aust.005.nc.gz',
 'FractCover.V2_2.2010.241.aust.005.nc.gz',
 'FractCover.V2_2.2010.249.aust.005.nc.gz',
 'FractCover.V2_2.2010.257.aust.005.nc.gz',
 'FractCover.V2_2.2010.265.aust.005.nc.gz',
 'FractCover.V2_2.2010.273.aust.005.nc.gz',
 'FractCover.V2_2.2010.281.aust.005.nc.gz',
 'FractCover.V2_2.2010.289.aust.005.nc.gz',
 'FractCover.V2_2.2010.297.aust.005.nc.gz',
 'FractCover.V2_2.2010.305.aust.005.nc.gz',
 'FractCover.V2_2.2010.313.aust.005.nc.gz',
 'FractCover.V2_2.2010.321.aust.005.nc.gz',
 'FractCover.V2_2.2010.329.aust.005.nc.gz',
 'FractCover.V2_2.2010.337.aust.005.nc.gz',
 'FractCover.V2_2.2010.345.aust.005.nc.gz',
 'FractCover.V2_2.2010.353.aust.005.nc.gz',
 'FractCover.V2_2.2010.361.aust.005.nc.gz',
 'FractCover.V2_2.2011.001.aust.005.nc.gz',
 'FractCover.V2_2.2011.009.aust.005.nc.gz',
 'FractCover.V2_2.2011.017.aust.005.nc.gz',
 'FractCover.V2_2.2011.025.aust.005.nc.gz',
 'FractCover.V2_2.2011.033.aust.005.nc.gz',
 'FractCover.V2_2.2011.041.aust.005.nc.gz',
 'FractCover.V2_2.2011.049.aust.005.nc.gz',
 'FractCover.V2_2.2011.057.aust.005.nc.gz',
 'FractCover.V2_2.2011.065.aust.005.nc.gz',
 'FractCover.V2_2.2011.073.aust.005.nc.gz',
 'FractCover.V2_2.2011.081.aust.005.nc.gz',
 'FractCover.V2_2.2011.089.aust.005.nc.gz',
 'FractCover.V2_2.2011.097.aust.005.nc.gz',
 'FractCover.V2_2.2011.105.aust.005.nc.gz',
 'FractCover.V2_2.2011.113.aust.005.nc.gz',
 'FractCover.V2_2.2011.121.aust.005.nc.gz',
 'FractCover.V2_2.2011.129.aust.005.nc.gz',
 'FractCover.V2_2.2011.137.aust.005.nc.gz',
 'FractCover.V2_2.2011.145.aust.005.nc.gz',
 'FractCover.V2_2.2011.153.aust.005.nc.gz',
 'FractCover.V2_2.2011.161.aust.005.nc.gz',
 'FractCover.V2_2.2011.169.aust.005.nc.gz',
 'FractCover.V2_2.2011.177.aust.005.nc.gz',
 'FractCover.V2_2.2011.185.aust.005.nc.gz',
 'FractCover.V2_2.2011.193.aust.005.nc.gz',
 'FractCover.V2_2.2011.201.aust.005.nc.gz',
 'FractCover.V2_2.2011.209.aust.005.nc.gz',
 'FractCover.V2_2.2011.217.aust.005.nc.gz',
 'FractCover.V2_2.2011.225.aust.005.nc.gz',
 'FractCover.V2_2.2011.233.aust.005.nc.gz',
 'FractCover.V2_2.2011.241.aust.005.nc.gz',
 'FractCover.V2_2.2011.249.aust.005.nc.gz',
 'FractCover.V2_2.2011.257.aust.005.nc.gz',
 'FractCover.V2_2.2011.265.aust.005.nc.gz',
 'FractCover.V2_2.2011.273.aust.005.nc.gz',
 'FractCover.V2_2.2011.281.aust.005.nc.gz',
 'FractCover.V2_2.2011.289.aust.005.nc.gz',
 'FractCover.V2_2.2011.297.aust.005.nc.gz',
 'FractCover.V2_2.2011.305.aust.005.nc.gz',
 'FractCover.V2_2.2011.313.aust.005.nc.gz',
 'FractCover.V2_2.2011.321.aust.005.nc.gz',
 'FractCover.V2_2.2011.329.aust.005.nc.gz',
 'FractCover.V2_2.2011.337.aust.005.nc.gz',
 'FractCover.V2_2.2011.345.aust.005.nc.gz',
 'FractCover.V2_2.2011.353.aust.005.nc.gz',
 'FractCover.V2_2.2011.361.aust.005.nc.gz',
 'FractCover.V2_2.2012.001.aust.005.nc.gz',
 'FractCover.V2_2.2012.009.aust.005.nc.gz',
 'FractCover.V2_2.2012.017.aust.005.nc.gz',
 'FractCover.V2_2.2012.025.aust.005.nc.gz',
 'FractCover.V2_2.2012.033.aust.005.nc.gz',
 'FractCover.V2_2.2012.041.aust.005.nc.gz',
 'FractCover.V2_2.2012.049.aust.005.nc.gz',
 'FractCover.V2_2.2012.057.aust.005.nc.gz',
 'FractCover.V2_2.2012.065.aust.005.nc.gz',
 'FractCover.V2_2.2012.073.aust.005.nc.gz',
 'FractCover.V2_2.2012.081.aust.005.nc.gz',
 'FractCover.V2_2.2012.089.aust.005.nc.gz',
 'FractCover.V2_2.2012.097.aust.005.nc.gz',
 'FractCover.V2_2.2012.105.aust.005.nc.gz',
 'FractCover.V2_2.2012.113.aust.005.nc.gz',
 'FractCover.V2_2.2012.121.aust.005.nc.gz',
 'FractCover.V2_2.2012.129.aust.005.nc.gz',
 'FractCover.V2_2.2012.137.aust.005.nc.gz',
 'FractCover.V2_2.2012.145.aust.005.nc.gz',
 'FractCover.V2_2.2012.153.aust.005.nc.gz',
 'FractCover.V2_2.2012.161.aust.005.nc.gz',
 'FractCover.V2_2.2012.169.aust.005.nc.gz',
 'FractCover.V2_2.2012.177.aust.005.nc.gz',
 'FractCover.V2_2.2012.185.aust.005.nc.gz',
 'FractCover.V2_2.2012.193.aust.005.nc.gz',
 'FractCover.V2_2.2012.201.aust.005.nc.gz',
 'FractCover.V2_2.2012.209.aust.005.nc.gz',
 'FractCover.V2_2.2012.217.aust.005.nc.gz',
 'FractCover.V2_2.2012.225.aust.005.nc.gz']

The possible data services end points through NCI's THREDDS includes: OPeNDAP, Netcdf Subset Service (NCSS), HTTP download, Web Map Service (WMS), Web Coverage Service (WCS), NetCDF Markup Language (NcML), and a few metadata services (ISO, UDDC).



In [6]:

    
for key, value in endpts[0].access_urls.items():
    print('{}, {}'.format(key, value))









    



WMS, http://dapds00.nci.org.au/thredds/wms/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
WCS, http://dapds00.nci.org.au/thredds/wcs/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
NetcdfSubset, http://dapds00.nci.org.au/thredds/ncss/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
OPENDAP, http://dapds00.nci.org.au/thredds/dodsC/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
UDDC, http://dapds00.nci.org.au/thredds/uddc/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
ISO, http://dapds00.nci.org.au/thredds/iso/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
HTTPServer, http://dapds00.nci.org.au/thredds/fileServer/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz
NCML, http://dapds00.nci.org.au/thredds/ncml/u39/public/data/modis/fractionalcover-clw/v2.2/netcdf/FractCover.V2_2.2000.049.aust.005.nc.gz

We can create a small function that uses Siphon's Netcdf Subset Service (NCSS) to extract a spatial request (defined by a lat/lon box)



In [7]:

    
def get_data(dataset, bbox):    
    nc = ncss.NCSS(dataset.access_urls['NetcdfSubset'])
    query = nc.query()
    query.lonlat_box(north=bbox[3],south=bbox[2],east=bbox[1],west=bbox[0])
    query.variables('bs')
    
    data = nc.get_data(query)
    
    lon = data['longitude'][:]
    lat = data['latitude'][:]
    bs = data['bs'][0,:,:]
    t = data['time'][:]
    
    time_base = datetime.date(year=1800, month=1, day=1)
    time = time_base + datetime.timedelta(t[0])
    
    return lon, lat, bs, time

Query a single file and view result



In [8]:

    
bbox = (135, 140, -31, -27)
lon, lat, bs, t = get_data(endpts[0], bbox)



In [9]:

    
plt.figure(figsize=(10,10))
plt.imshow(bs, extent=bbox, cmap='gist_earth', origin='upper')

plt.xlabel('longitude (degrees)', fontsize=14)
plt.ylabel('latitude (degrees)', fontsize=14)
print("Date: {}".format(t))









    



Date: 2000-02-18

Loop and query over the collection



In [10]:

    
bbox = (135, 140, -31, -27)
plt.figure(figsize=(10,10))

for endpt in endpts[:15]:
    try:
        lon, lat, bs, t = get_data(endpt, bbox)

        plt.imshow(bs, extent=bbox, cmap='gist_earth', origin='upper')
        plt.clim(vmin=-2, vmax=100)

        plt.tick_params(labelsize=14)
        plt.xlabel('longitude (degrees)', fontsize=14)
        plt.ylabel('latitude (degrees)', fontsize=14)

        plt.title("Date: "+str(t), fontsize=16, weight='bold')
        plt.savefig("./images/"+endpt.name+".png")
        plt.cla()
    except:
        pass

plt.close()

Can make an animation of the temporal evolution (this example is by converting the series of *.png files above into a GIF)

Can also use Siphon to extract a single point



In [11]:

    
def get_point(dataset, lat, lon):
    nc = ncss.NCSS(dataset.access_urls['NetcdfSubset'])
    query = nc.query()
    query.lonlat_point(lon, lat)
    query.variables('bs')
    
    data = nc.get_data(query)
    bs = data['bs'][0]
    date = data['date'][0]
    
    return bs, date



In [12]:

    
bs, date = get_point(endpts[4], -27.75, 137)
print("{}, {}".format(bs, date))









    



51.0, 2000-03-21 00:00:00+00:00

Time series example



In [13]:

    
data = []
for endpt in endpts[::20]:
    bs, date = get_point(endpt, -27.75, 137)
    data.append([date, bs])



In [14]:

    
import numpy as np

BS = np.array(data)[:,1]
Date = np.array(data)[:,0]

plt.figure(figsize=(12,6))
plt.plot(Date, BS, '-o', linewidth=2, markersize=8)

plt.tick_params(labelsize=14)
plt.xlabel('date', fontsize=14)
plt.ylabel('fractional cover of bare soil (%)', fontsize=14)
plt.title('Lat, Lon: -27.75, 137', fontsize=16)









    Out[14]:





<matplotlib.text.Text at 0x7f782846fa58>



In [ ]:



In [ ]: