Using Iris to access NCEP CFSR 30-year Wave Hindcast

This demonstrates extracting data from a large aggregated archive via OPeNDAP, taking advantage of CF conventions to enable a plot without specification of lon,lat variables, and save extracted data to GRIB2


In [2]:
from IPython.core.display import HTML
HTML('<iframe src=http://scitools.org.uk/iris/ width=800 height=350></iframe>')


Out[2]:

In [3]:
import numpy
import matplotlib.pyplot as plt
import datetime as dt

import iris
import iris.plot as iplt
import cartopy.crs as ccrs

In [4]:
def time_near(cube,start):
    timevar=cube.coord('time')
    itime = timevar.nearest_neighbour_index(timevar.units.date2num(start))
    return timevar.points[itime]

In [5]:
def myplot(slice,model=None):
    # make the plot
    figure(figsize=(12,8))
    lat=slice.coord(axis='Y').points
    lon=slice.coord(axis='X').points
    time=slice.coord('time')[0]
    subplot(111,aspect=(1.0/cos(mean(lat)*pi/180.0)))
    pcolormesh(lon,lat,masked_invalid(slice.data));
    colorbar()
    grid()
    date_str=time.units.num2date(time.points)
    plt.title('%s: %s: %s' % (model,slice.long_name,date_str));

Extract some data using OPeNDAP


In [6]:
# DAP URL: 30 year East Coast wave hindcast (Wave Watch 3 driven by CFSR Winds) 
#cubes = iris.load('http://geoport.whoi.edu/thredds/dodsC/fmrc/NCEP/ww3/cfsr/4m/best'); # 4 arc minute resolution
cubes = iris.load('http://geoport.whoi.edu/thredds/dodsC/fmrc/NCEP/ww3/cfsr/10m/best'); # 10 arc minute resolution


/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/cf.py:514: UserWarning: Missing CF-netCDF grid mapping variable u'LatLon_Projection', referenced by netCDF variable u'u-component_of_wind_surface'
  warnings.warn(message % (name, nc_var_name))
/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/cf.py:514: UserWarning: Missing CF-netCDF grid mapping variable u'LatLon_Projection', referenced by netCDF variable u'v-component_of_wind_surface'
  warnings.warn(message % (name, nc_var_name))
/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/cf.py:514: UserWarning: Missing CF-netCDF grid mapping variable u'LatLon_Projection', referenced by netCDF variable u'Significant_height_of_combined_wind_waves_and_swell_surface'
  warnings.warn(message % (name, nc_var_name))
/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/cf.py:514: UserWarning: Missing CF-netCDF grid mapping variable u'LatLon_Projection', referenced by netCDF variable u'Primary_wave_direction_degree_true_surface'
  warnings.warn(message % (name, nc_var_name))
/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/cf.py:514: UserWarning: Missing CF-netCDF grid mapping variable u'LatLon_Projection', referenced by netCDF variable u'Primary_wave_mean_period_surface'
  warnings.warn(message % (name, nc_var_name))
/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:1163: UserWarning: Ignoring netCDF variable 'Primary_wave_direction_degree_true_surface' invalid units 'deg'
  warnings.warn(msg.format(msg_name, msg_units))

In [7]:
print cubes


0: Significant height of combined wind waves and swell @ Ground or water surface / (m) (time: 90584; latitude: 331; longitude: 301)
1: u-component of wind @ Ground or water surface / (m/s) (time: 90584; latitude: 331; longitude: 301)
2: v-component of wind @ Ground or water surface / (m/s) (time: 90584; latitude: 331; longitude: 301)
3: Primary wave direction (degree true) @ Ground or water surface / (unknown) (time: 90584; latitude: 331; longitude: 301)
4: Primary wave mean period @ Ground or water surface / (s) (time: 90584; latitude: 331; longitude: 301)

In [8]:
# use contraints to select geographic subset and nearest time
mytime=dt.datetime(1991,10,31,12)  #specified time...   Nov 1, 1991 was the "Perfect Storm"
#mytime=dt.datetime.utcnow()      # .... or now

In [25]:
cubes2 = cubes.extract(iris.Constraint(time=time_near(cubes[0],mytime),
    longitude=lambda cell: -71.5 < cell < -64.0,
    latitude=lambda cell: 40.0 < cell < 46.0))

In [26]:
hsig=cubes2[0]
u=cubes2[1]
v=cubes2[2]

In [27]:
print cubes2


0: Significant height of combined wind waves and swell @ Ground or water surface / (m) (latitude: 35; longitude: 44)
1: u-component of wind @ Ground or water surface / (m/s) (latitude: 35; longitude: 44)
2: v-component of wind @ Ground or water surface / (m/s) (latitude: 35; longitude: 44)
3: Primary wave direction (degree true) @ Ground or water surface / (unknown) (latitude: 35; longitude: 44)
4: Primary wave mean period @ Ground or water surface / (s) (latitude: 35; longitude: 44)

In [29]:
print[coord.name() for coord in hsig.coords()]


['latitude', 'longitude', u'time']

Plot using Iris.plot

There is also Iris.quickplot, but I wanted to add my own title here and control the orientation of the colorbar, so I used the more flexible Iris.plot


In [32]:
def myplot(slice,u=None,v=None,model=None):
    # make the plot
    figure(figsize=(12,8))
    lat=slice.coord(axis='Y').points
    lon=slice.coord(axis='X').points
    time=slice.coord('time')[0]
    subplot(111,aspect=(1.0/cos(mean(lat)*pi/180.0)))
    pcolormesh(lon,lat,slice.data);
    isub = 2
    [lon2d,lat2d]=meshgrid(lon,lat)
    Q = quiver(lon[::isub,::isub],lat[::isub,::isub],u[::isub,::isub],v[::isub,::isub],scale=20)
    maxstr='%3.1f m/s' % maxvel
    qk = quiverkey(Q,0.92,0.08,maxvel,maxstr,labelpos='W')
    colorbar()
    grid()
    date=time.units.num2date(time.points)
    date_str=date[0].strftime('%Y-%m-%d %H:%M:%S %Z')
    plt.title('%s: %s: %s' % (model,slice.long_name,date_str));

In [43]:
lat=hsig.coord(axis='Y').points
lon=hsig.coord(axis='X').points

hsig = ma.masked_invalid(hsig.data)
u = ma.masked_invalid(u.data)
v = ma.masked_invalid(v.data)

In [45]:
[lon2d,lat2d]=meshgrid(lon,lat)
isub=2

In [46]:
Q = quiver(lon2d[::isub,::isub],lat2d[::isub,::isub],u[::isub,::isub],v[::isub,::isub],scale=20)
maxstr='%3.1f m/s' % maxvel
qk = quiverkey(Q,0.92,0.08,maxvel,maxstr,labelpos='W')


---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-46-8ca14d94a330> in <module>()
      1 Q = quiver(lon2d[::isub,::isub],lat2d[::isub,::isub],u[::isub,::isub],v[::isub,::isub],scale=20)
----> 2 maxstr='%3.1f m/s' % maxvel
      3 qk = quiverkey(Q,0.92,0.08,maxvel,maxstr,labelpos='W')

NameError: name 'maxvel' is not defined

In [38]:


In [39]:
Q = quiver(lon2d[::isub,::isub],lat2d[::isub,::isub],u[::isub,::isub],v[::isub,::isub],scale=20)
maxstr='%3.1f m/s' % maxvel
qk = quiverkey(Q,0.92,0.08,maxvel,maxstr,labelpos='W')


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-39-8ca14d94a330> in <module>()
----> 1 Q = quiver(lon2d[::isub,::isub],lat2d[::isub,::isub],u[::isub,::isub],v[::isub,::isub],scale=20)
      2 maxstr='%3.1f m/s' % maxvel
      3 qk = quiverkey(Q,0.92,0.08,maxvel,maxstr,labelpos='W')

/home/local/python27_epd/lib/python2.7/site-packages/matplotlib/pyplot.pyc in quiver(*args, **kw)
   2875         ax.hold(hold)
   2876     try:
-> 2877         ret = ax.quiver(*args, **kw)
   2878         draw_if_interactive()
   2879     finally:

/home/local/python27_epd/lib/python2.7/site-packages/matplotlib/axes.pyc in quiver(self, *args, **kw)
   6625     def quiver(self, *args, **kw):
   6626         if not self._hold: self.cla()
-> 6627         q = mquiver.Quiver(self, *args, **kw)
   6628         self.add_collection(q, False)
   6629         self.update_datalim(q.XY)

/home/local/python27_epd/lib/python2.7/site-packages/matplotlib/quiver.pyc in __init__(self, ax, *args, **kw)
    417                                             **kw)
    418         self.polykw = kw
--> 419         self.set_UVC(U, V, C)
    420         self._initialized = False
    421 

/home/local/python27_epd/lib/python2.7/site-packages/matplotlib/quiver.pyc in set_UVC(self, U, V, C)
    461 
    462     def set_UVC(self, U, V, C=None):
--> 463         U = ma.masked_invalid(U, copy=False).ravel()
    464         V = ma.masked_invalid(V, copy=False).ravel()
    465         mask = ma.mask_or(U.mask, V.mask, copy=False, shrink=True)

/home/local/python27_epd/lib/python2.7/site-packages/numpy/ma/core.pyc in masked_invalid(a, copy)
   2239         cls = type(a)
   2240     else:
-> 2241         condition = ~(np.isfinite(a))
   2242         cls = MaskedArray
   2243     result = a.view(cls)

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

In [44]:
myplot(cubes[0],u=cubes[1].data,v=cubes[2].data,model='foo')


---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-44-eb1625a11aca> in <module>()
----> 1 myplot(cubes[0],u=cubes[1].data,v=cubes[2].data,model='foo')

/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/cube.pyc in data(self)
   1138         if self._data_manager is not None:
   1139             try:
-> 1140                 self._data = self._data_manager.load(self._data)
   1141 
   1142             except (MemoryError,

/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/manager.pyc in load(self, proxy_array)
    266         for index, proxy in np.ndenumerate(proxy_array):
    267             if proxy not in [None, 0]:  # 0 can come from slicing masked proxy; np.array(masked_constant).
--> 268                 payload = proxy.load(self._orig_data_shape, self.data_type, self.mdi, deferred_slice)
    269 
    270                 # Explicitly set the data fill value when no mdi value has been specified

/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.5.0_dev-py2.7.egg/iris/fileformats/netcdf.pyc in load(self, data_shape, data_type, mdi, deferred_slice)
    253         variable = dataset.variables[self.variable_name]
    254         # Get the NetCDF variable data and slice.
--> 255         payload = variable[deferred_slice]
    256         dataset.close()
    257 

/home/local/python27_epd/lib/python2.7/site-packages/netCDF4.so in netCDF4.Variable.__getitem__ (netCDF4.c:29289)()

/home/local/python27_epd/lib/python2.7/site-packages/netCDF4.so in netCDF4.Variable._get (netCDF4.c:34647)()

RuntimeError: NetCDF: Malformed or inaccessible DAP DATADDS

Save extracted Cube to GRIB2 file


In [12]:
# first add a Geographic Coord system  (required by GRIB2 writer)

# here we add a spherical earth with specified radius
slice.coord(axis='X').coord_system=iris.coord_systems.GeogCS(654321)
slice.coord(axis='Y').coord_system=iris.coord_systems.GeogCS(654321)

In [13]:
# add a forecast_period (required by GRIB2 writer)
slice.add_aux_coord(iris.coords.DimCoord(0, standard_name='forecast_period', units='hours'))

In [14]:
# add a vertical coordinate (required by GRIB2 writer)
slice.add_aux_coord(iris.coords.DimCoord(0, "height", units="m"))

In [15]:
# GRIB2 stores data in long values.  Do we have huge values that would create problems?
slice.data.max()


Out[15]:
nan

In [16]:
# ah, we have NaNs.  This causes problems for GRIB2 writer
# So convert to masked-array instead.
slice.data=ma.masked_invalid(slice.data)

In [17]:
# Finally... save slice as grib2
iris.save(slice,'hsig.grib2')


/home/local/python27_epd/lib/python2.7/site-packages/Iris-1.4.0_dev-py2.7.egg/iris/fileformats/grib_save_rules.py:186: UserWarning: Not yet translating standard name into grib param codes.
discipline, parameterCategory and parameterNumber have been zeroed.
  warnings.warn("Not yet translating standard name into grib param codes.\n"