notebook.community

Edit and run



In [1]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/UV-CDAT_logo.png')









    Out[1]:

Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT)



In [2]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/CDAT_modularity.jpg')









    Out[2]:

CDAT's major subsystems are:

cdms - Climate Data Management System (file I/O, variables, types, metadata, grids)
cdutil - Climate Data Specific Utilities (spatial and temporal averages, custom seasons, climatologies)
genutil - General Utilities (statistical and other convenience functions)
numPy - Numerical Python _ (large-array numerical operations)
vcs - Visualization and Control System (manages graphical window: picture template, graphical methods, data)
xmgrace - Python interface to Grace Line Plot Tool

UVCDAT is the graphical user interface for CDAT/UV-CDAT and helps users become familiar with CDAT by translating every button press and keystroke into Python scripts. UV-CDAT gui does not require learning Python and the CDAT software.

Visualization ‐ vcs and xmgrace

VCS Concepts and Terminology

Basic concepts:

VCS Canvas – where the plots are drawn

Graphic Methods – how the data is rendered (the plot type).

E.g. "boxfill", "isofill", "isoline", "vector", etc,. 
Multiple projections may be available for a given Graphic Method.

Templates – define the location where things are drawn on the canvas (data, legend, title, comments, units, etc..)
Primitives – additional secondary items such as lines, polygons, text, markers, etc.,

The VCS Canvas

VCS canvas needs to be initialized (created)

>>> import vcs

>>> x = vcs.init()  # without any arguments

Up to 8 Canvases at once.
Canvas as “magic board”, can clean or destroy it:
```
>>> x.clear()
>>> x.close()
```
Can have multiple plots on a single Canvas (not covered here)

VCS Help

Basic help on VCS can be obtain inline via:

>>> vcs.help()

This will list available function in VCS, notably functions to create/get/query VCS objects...

This also can be used to get help on a specific command:

>>> vcs.help('createboxfill')

Function: createboxfill
# Construct a new boxfill graphics method
Description of Function:
    Create a new boxfill graphics method given the name and the existing boxfill
    graphics method to copy the  attributes from. If no existing boxfill graphics 
    method name is given, then the default boxfill graphics method will be used 
    as the graphics method to which the attributes will be copied from.

    If the name provided already exists, then a error will be returned. Graphics 
    method names must be unique.

Graphic Methods Concepts

Essentially a graphic method represents HOW data are going to be plotted (the WHERE will be determined via templates, see later)

There are 13 type of graphic methods in VCS:
- 2D Graphic Methods
  - Boxfill, Isofill, Isoline, Meshfill, Vector, Outfill, Outline, Continents, (Taylordiagrams)
- 1D Graphic Methods
  - Yxvsx, Xyvsy, XvsY, Scatter

Specifying a Graphic Method

To specify a graphic method use a "create" function, then use plot:

For example, for a boxfill:

>>> gm = x.createboxfill('name')
>>> x.plot(data, gm)

If a graphic method already exists, use "get" functions:
```
>>> gm = x.getboxfill('name')
>>> x.plot(data, gm)
```
Replace 'boxfill' in the above for other methods.

2D ‐ "boxfill"

The boxfill graphic method takes a 2D array and represents it by filling a “box” (determined by the bounds on each axis values) with a color linked to the array’s value at this location:

>>> box = x.createboxfill('new')
>>> x.plot(data,box)

Example Plot



In [3]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/boxfill.png')









    Out[3]:



In [ ]:

    
import cdms2
DATAPATH = './data/Obs/mo/sst/HadISST/'
# open the nc file throgh cdms2 module
f = cdms2.open(DATAPATH + 'sst_HadISST_Climatology_1961-1990.nc')
# You can query the file
print f.listvariables() 
data = f('sst', time=('0-1-16 12:0'), latitude=(-90,90)) # retrieves the JAN data - a “slab”
data.shape



In [ ]:

    
# import vcs module
import vcs
# initialize vcs canvas
x = vcs.init()



In [ ]:

    
# create box fill object
box = x.createboxfill('new')
# plot the data with box fill template
x.plot(data, box)

Demo Live Plot



In [ ]:

    
# clear the canvas
x.clear()
# plot sliced data (Indian Region) with box fill template
x.plot(data(latitude=(0, 40), longitude=(60,100)), box)



In [ ]:

    
gentemp = x.createtemplate()
gentemp.scale(1.2, axis='y')
gentemp.scale(0.6, axis='x')
gentemp.move(-0.05, axis='y')

Demo Live Plot



In [ ]:

    
# clear the canvas
x.clear()
# plot sliced data (Indian Region) with box fill template
x.plot(data(latitude=(0, 40), longitude=(60,100)), box, gentemp)

2D ‐ "isofill"

Isofill graphic methods draws filled isocontour

They are extremely similar to boxfill "custom" type
Known Limitation:
- No control on labels position
- No control on isolines "Smoothness"
```
>>> isofill = x.createisofill('new')
>>> x.plot(data, isofill)
```

Example Plot



In [4]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/isofill.png')









    Out[4]:

Demo Live Plot



In [ ]:

    
# clear the canvas
x.clear()
# create new isofill template
isofill = x.createisofill()
# plot sliced data (Indian Region) with box fill template
x.plot(data(latitude=(0, 40), longitude=(60,100)), isofill, gentemp)

2D ‐ "isoline"

Isoline, draws isocontours, color, style, can be controlled.

Limitation:
- No control on the labels location
- No control of "smoothness"
```
>>> isoline = x.createisoline('new')
>>> x.plot(data, isoline)
```

Example Plot



In [5]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/isoline.png')









    Out[5]:

Demo Live Plot



In [ ]:

    
# clear the canvas
x.clear()
# create new isoline template
isoline = x.createisoline()
# set iso line label on
isoline.label = 'yes'
# plot sliced data (Indian Region) with box fill template
x.plot(data(latitude=(0, 40), longitude=(60,100)), isoline, gentemp)

2D ‐ "vector"

The "Vector" graphic method represents the combination of 2 arrays, via "vector" the first array representing the "X" axis component and the second array representing the "Y" axis component.

>>> vec = x.createvector('new')
>>> x.plot(u, v, vec)



In [ ]:

    
wndf = cdms2.open('data/Climatology/NCEP_NCAR_Climatology_ltm/Levels/wnd.xml')
wndf.listvariable()
u = wndf('uwnd', time=('1-1-1'), levels=850, latitude=(0, 40), longitude=(60,100))
v = wndf('vwnd', time=('1-1-1'), levels=850, latitude=(0, 40), longitude=(60,100))

Demo Live Plot



In [ ]:

    
# clear the canvas
x.clear()
# create new vector template
vec = x.createvector('new')
# plot sliced data (Indian Region) with box fill template
print "Default vector template will not suitable to all dataset"
x.plot(u, v, vec, gentemp)

Demo Live Plot



In [ ]:

    
# create lat, lon dictionary which refers str vs float
lat = {-30:'30S', -20:'20S',-10:'10S', 0:'Eq', 10: '10N', 20:'20N', 30:'30N', 40:'40N'}
lon = {60:'60E', 70:'70E', 80:'80E', 90:'90E', 100:'100E'}

# clear the canvas
x.clear()
# get new vector template
vec = x.getvector('new')
# control vector arrow scale
vec.scale = 2.0
# control reference arrow length
vec.reference = 5.0
# set lat, lon dictionary as labels
vec.yticlabels1 = lat
vec.xticlabels1 = lon
# plot sliced data (Indian Region) with box fill template
# updated vec template plot
print "So just adjust vector template scale, reference according to our need"
x.plot(u, v, vec, gentemp, continents=1)

1D ‐ "Y(x) vs x"

All 1D plots in VCS basically work the same way.
There are 4 types of 1D graphic method, we’ll start with the basic: Yxvsx, which stands for Y(x) vs x
This graphic method draws a 1D array (Y) as a function of its 1D axis (x)
Example zonal mean of the first time point of our data array

>>> zm = MV2.average(data[0],1) # Zm.shape is (46,)
>>> x.plot(zm) # knows to plot 1D with yxvsx
>>> yx = x.createyxvsx('new')
>>> x.plot(zm, yx)

Example Plot



In [6]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/line1.png')









    Out[6]:

1D ‐ VCS "Yxvsx" attributes

As in isoline, or vector, line, linecolor, linewidth, determine the line.
marker, markercolor, markersize, determine the markers to be drawn:

>>> yx.line = 'dot'
>>> yx.linecolor = 242
>>> yx.linewidth = 2
>>> yx.marker = 'star’
>>> yx.markercolor = 244

Example Plot



In [7]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/line2.png')









    Out[7]:

Graphic Methods Attributes



In [ ]:

    
b = x.createboxfill('new_one')
b.list()

2D ‐ World Coordinates

worldcoordinate attributes are present on all graphic methods.
can select a subset area.
instead of data selection for particular region, we can sub select/control world coordinate in template also.
for example to visualise Africa:

>>> b.datawc_x1 = -45
>>> b.datawc_x2 = 70
>>> b.datawc_y1 = -38
>>> b.datawc_y2 = 38
>>> x.plot(s,b)

Example Plot



In [8]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/africa.png')









    Out[8]:

Projections

P = x.createprojection()

Graphicmethod.projection=P
P.type = n
- N can be one of 30 possible
print P.doc
- Each type has specific parameters
P.list()



In [ ]:

    
P = x.createprojection()
P.list()
print P.__doc__

Demo Live Plot



In [ ]:

    
f = cdms2.open('data/sample_data/clt1.nc')
s = f('clt',longitude=(-210,50))
x.clear()
iso = x.createisofill()
p = x.createprojection()
p.type = 'orthographic'
iso.projection = p
x.plot(s, iso, ratio='1t')

Demo Live Plot



In [ ]:

    
x.clear()
p.type = 'lambert'
x.plot(s(latitude=(20,70),longitude=(-150,-50)), iso)

Each graphic method may have a number of projections, the full list is available from:



In [ ]:

    
x.show('projection')

Example Plot



In [9]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/projections.png')









    Out[9]:

Introducing Templates

Template tell VCS WHERE to draw objects on the canvas (whereas Graphic Methods specify HOW to draw them).
4 aspects of the plot controlled templates:
- Text location for things like title, comments, name, etc..
- Data area
- Tick marks and label locations
- Legend

VCS: template manipulation

Template ratio

  X.ratio=2 : y is twice as big as x
  X.ratio='auto'
  X.ratio='2t' : also moves tick marks

Template scaling (lower left data area unchanged)

  T.scale(.5) # half size
  T.scale(.5, axis='x') #half size in X, font unchanged
  T.scale(.5, axis='x', font=1) # also alter fonts

Template moving

  T.move(.2, .4) # move by 20% in x, 40% in y Positive values means up/right
  T.moveto(x,y) # move lower left corner of data to x,y

VCS: EzTemplate

example code. it will not execute for demo

from vcsaddons import EzTemplate
import vcs
x = vcs.init()
M = EzTemplate.Multi(rows=4,columns=3)
for i in range(12):
      t = M.get()
      x.plot(s,t,iso)

Example Template Plot



In [10]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/eztemplate.png')









    Out[10]:

Also available: EzTemplate.oneD,for 1D plots, uses the provided “base” template and move the legend according to the number of dataset

example code. it will not execute for demo

OD = EzTemplate.oneD(n=n,template=t)
for i in range(n):
      y = MV2.sin((i+1)*x)
      y.setAxis(0,ax)
      yx = X.createyxvsx()
      yx.linecolor=241+i
      yx.datawc_y1=‐1.
      yx.datawc_y2=1.
      t = OD.get()
      X.plot(y,t,yx,bg=bg)

vcs: text primitives



In [ ]:

    
text = x.createtext()
text.list()

Font_name = x.addfont(path_to_ttf_font)
Font = x.getfont(Font_name) # usable in template
Available fonts by default: ['Adelon', 'Arabic', 'AvantGarde', 'Chinese', 'Clarendon', 'Courier', 'Greek', 'Hebrew', 'Helvetica', 'Maths1', 'Maths2', 'Maths3', 'Maths4', 'Russian', 'Times', 'default']

vcs: other primitives

fa = x.createfillarea('new')
l = x.createline('new')
m = x.createmarker('new')
Each primitive has the 2 following attributes:
- Prim.viewport=[xv1,xv2,yv1,yv2] # default: [0,1,0,1]
  - In % of page, area of the primitive extends
- Prim.worldcoordinates = [x1,x2,y1,y2] # defalut [0,1,0,1]
  - Coordinates corresponding to xv1,xv2,yv1,yv2
  - Primitive units are in the worldcoordinate system
- Example
  - text.viewport = [.25,.75,.25,.75] # define smaller zone on page
  - text.worldcoordinate = [‐180, ‐90, 180, 90] # Define the coordinate system
  - text.x = [77.2300]
  - text.y = [28.6100]
  - text.string = ['New Delhi - India']
For overlay with an existing graphic method
- Prim.viewport # set to your template.data
- Prim.worldcoordinates # set to your graphic method.data.wc

vcs: Animating at the command line

Just keep write a frame to a canvas, to gif and clear, then do all over again...

import cdms2, vcs
f = cdms2.open('data/Obs/mo/sst/HadISST/sst_HadISST_Climatology_1961-1990.nc')
data = f('sst')
x = vcs.init()
isofill = x.createisofill()
templ = x.createtemplate()

for i in range(loop_num):
    x.clear()
    x.plot( f('sst', time=slice(i,i+1) ), isofill, templ)
    x.gif('img/sst_animate.gif', merge='a')

vcs: vcsaddons

GIS capability. You can read and plot gis/shapefiles.

Example Plot



In [11]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/gis.png')









    Out[11]:

example code. It will not work for demo, since we dont have sample shape file data

import vcs,vcsaddons
import cdms2,sys
x=vcs.init()
import vcs.test.support
bg=0
c=vcsaddons.createusercontinents(x=x)
lon1=‐125
lon2=‐75.
lat1=20.
lat2=55.
c.types = ['shapefile','shapefile']
c.sources = ['../Data/co1990p020','../Data/fe_2007_06_county',]
c.colors = [246,241,244,241]
c.widths=[1,1,1]
c.lines=['solid','solid','solid','dot']
f=cdms2.open(sys.prefix+’/sample_data/clt.nc’)
s=f("clt",latitude=(lat1,lat2),longitude=(lon1,lon2),time=slice(0,1))
t=x.createtemplate()
iso=x.createisofill()
x.plot(s,t,iso,continents=0,ratio='autot',bg=bg)
x.plot(s,c,t,ratio='autot',bg=bg)

Xmgrace

Xmgrace is a very nice plotting package for 1‐D data.
If you have xmgrace installed on your computer, you can script it via CDAT. Its interface has been mapped to be as close as possible from the "parameter" files from xmgrace.
example codes & plots



In [ ]:

    
# sample code for "xyboxplot" - Error Box Whiskers Plot
%load scripts/boxwisker_multi_xmgrace.py



In [ ]:

    
"""
Written By : Arulalan.T
Date : 31.07.2013
"""

import xmgrace
import numpy

x = xmgrace.init()

x.Graph[0].yaxis.min=-150. # ymin for graph 0
x.Graph[0].yaxis.max=300.
x.Graph[0].yaxis.tick.inc=50 # Main tick every unit
x.Graph[0].yaxis.tick.minor_ticks=0 # 4 sub in between , 1 every .25 units

x.Graph[0].xaxis.min=0.
x.Graph[0].xaxis.max=4.


# median, upperQuartile, lowerQuartile, max, min 
a = [14.9471, 125.928, -22.6248, 217.2, -52.5566]
#####
#### The order of the values to be passed in the yaxis
#### median - horizontal small line inside the box 
#### upperQuartile - box upper edge
#### lowerQuartile - box lower edge
#### max - maxmimum peak of the vertical line over the box 
#### min - lowerst value of the vertical line under the box 
#### 

yaxis = numpy.array(a)
### yaxis data should be in vertical stack.
yaxis = yaxis.reshape((5,1))

for i in range(2):
    x.Graph[0].Set[i].type = 'xyboxplot'
    # make the box color as black
    x.Graph[0].Set[i].symbol.color = 1
    if i == 0:
        # make the box fill color as white
        x.Graph[0].Set[i].symbol.fcolor = 0
        # adjust the box width size
        x.Graph[0].Set[i].symbol.size = 1
        # enable the upper and lower vertical line 
        x.Graph[0].Set[i].errorbar.status = 'on'
        # adjust the size of the horizontal lines of the edge 
        # of the upper & lower errorbar lines.
        x.Graph[0].Set[i].errorbar.size = 1
    else:
        # make the box fill color as white
        x.Graph[0].Set[i].symbol.fcolor = 10
        # adjust the box width size
        x.Graph[0].Set[i].symbol.size = 3
        # enable the upper and lower vertical line 
        x.Graph[0].Set[i].errorbar.status = 'on'
        # adjust the size of the horizontal lines of the edge 
        # of the upper & lower errorbar lines.
        x.Graph[0].Set[i].errorbar.size = 2
    # end of if i == 0:
    # add new set to the Graph '0'
    x.add_set(0)
    
x.plot([yaxis], xs=[[2]], G=0, S=0)
# pass different yaxis and x value for the another set 
x.plot([yaxis], xs=[[3]], G=0, S=1)
x.ps('box_xy_plot_multi')

print x.Graph[0].Set[0].list()

Outplut Plot of the above sample code for "xyboxplot" - Error Box Whiskers Plot

Example Plot



In [12]:

    
#hideme
from IPython.core.display import Image 
Image(filename='img/whiskers_multi.png')









    Out[12]:



In [ ]:

    
%load scripts/test_xmgrace.py



In [ ]:

    
import cdms2, cdutil, genutil
NINO3 = cdms2.selectors.Selector(cdutil.region.domain(latitude=(-5., 5., 'ccb'), longitude=(210., 270., 'ccb')))

INDIR = '../mo/sst/HadISST/' 
infile = INDIR + 'sst_HadISST_1870-1_2011-1.nc'
f= cdms2.open(infile)

nino3_data = f('sst', NINO3)
nino3_average = cdutil.averager(nino3_data, axis='xy')
nino3_slice = nino3_average(time=('1961-1-1', '1990-12-31'))
nino3_clim = cdutil.ANNUALCYCLE.climatology(nino3_slice)
nino3_anomaly = cdutil.ANNUALCYCLE.departures(nino3_average, nino3_clim)

#
# A quick and dirty plot
#
xm = genutil.xmgrace.init()
xm.plot(nino3_anomaly, G=0, S=0)
# by default only one Set (dataset) exists. So you add another one.
xm.add_set()
xm.plot(nino3_average, G=0, S=1)



#
# Same plot done in a more presentable way
#
xm2 = genutil.xmgrace.init()
# By default only one graph exists. Add another graph so...
xm2.add_graph()

#
# Set the viewport corners
#
xm2.Graph[0].vxmin = 0.1
xm2.Graph[0].vxmax = 0.9
xm2.Graph[0].vymin = 0.1
xm2.Graph[0].vymax = 0.48
# Second graph
xm2.Graph[1].vxmin = 0.1
xm2.Graph[1].vxmax = 0.9
xm2.Graph[1].vymin = 0.52
xm2.Graph[1].vymax = 0.9

xm2.Graph[0].frame.linewidth = 2
xm2.Graph[1].frame.linewidth = 2

#
# Set the y-axis minimum and maximum
#
xm2.Graph[0].ymin = -2.0
xm2.Graph[0].ymax = 4.0
# second graph
xm2.Graph[1].ymin = 22.
xm2.Graph[1].ymax = 30.


#
# Tick marks etc.
#
xm2.Graph[0].yaxis.tick.inc = 1.
xm2.Graph[0].yaxis.tick.orientation = 'out' # By default it is 'in'
# Second graph
xm2.Graph[1].yaxis.tick.inc = 2.
xm2.Graph[1].yaxis.tick.orientation = 'out' # By default it is 'in'
xm2.Graph[1].yaxis.tick.place = 'opposite' # By default it is 'normal'
xm2.Graph[1].yaxis.tick.label.place = 'opposite' # By default it is 'normal'

xm2.Graph[1].yaxis.lbl.place.side = 'opposite'

xm2.Graph[0].yaxis.label = 'NIN\\v{0.6}\\h{-0.5}~\\h{}\\v{}O3 Anomaly (\So\NC)'
xm2.Graph[1].yaxis.label = 'NIN\\v{0.6}\\h{-0.5}~\\h{}\\v{}O3 Average (\So\NC)'


#
# Let us put a zero axis on the lower(anomaly) plot.
#
xm2.Graph[0].altxaxis.status = 'on'
xm2.Graph[0].altxaxis.zero = 'True'
xm2.Graph[0].altxaxis.tick.status = 'off'
xm2.Graph[0].altxaxis.tick.label.status = 'off'


#
# Set the y-axis minimum and maximum
#
xm2.Graph[0].xmin = 16383360.
xm2.Graph[0].xmax = 17619340.
# second graph
xm2.Graph[1].xmin = 16383360.
xm2.Graph[1].xmax = 17619340.


#
# Now let us put meaningful x-axis tick labels.
#
lbl_dict = {}
taxis = nino3_average.getTime()
tc = taxis.asComponentTime()
for i in range(len(tc)):
    if divmod(tc[i].year, 10)[1] == 0 and tc[i].month == 6:
        lbl_dict[taxis[i]] = str(tc[i].year)
        #print taxis[i], tc[i].year
    elif divmod(tc[i].year, 5)[1] == 0 and tc[i].month == 6:
        lbl_dict[taxis[i]] = ''
    else:
        pass
    # end of if
# end of for i in range(len(tc)):

xm2.Graph[0].xaxis.tick.spec.loc = lbl_dict
xm2.Graph[1].xaxis.tick.spec.loc = lbl_dict



xm2.Graph[0].xaxis.tick.orientation = 'out'
xm2.Graph[1].xaxis.tick.orientation = 'out'
xm2.Graph[1].xaxis.tick.place = 'opposite' # By default it is 'normal'
xm2.Graph[1].xaxis.tick.label.place = 'opposite' # By default it is 'normal'

#
# Set the line colors that you want to plot.
#
# default color is black.
xm2.Graph[0].Set[0].line.color = 'blue'
xm2.Graph[1].Set[0].line.color = 'red'

xm2.plot(nino3_anomaly, G=0, S=0)
xm2.plot(nino3_average, G=1, S=0)

Other Plotting Packages

Matplotlib: http://matplotlib.sourceforge.net
Visus
IaGraphics

Acknowledgments

Dean Williams, Charles Doutriaux (PCMDI, LLNL).
Dr. Johnny Lin (Physics Department, North Park University, Chicago, Illinois).
Dr.Krishna AchtuaRao (Centre for Atmospheic Sciences, Indian Institute of Technology Delhi, India).

License

IPython Notebook Created by

  Arulalan.T 
  Date : 18.06.2014
  Project Associate,
  Centre for Atmospheic Sciences,
  Indian Institute of Technology Delhi, India.
  Blog : http://tuxcoder.wordpress.com 
  Repo : https://github.com/arulalant/UV-CDAT-IPython-Notebooks

This work (IPython Notebook & Html Slide) is licensed under a Creative Commons Attribution‐NonCommercial‐ShareAlike 3.0
Includes all python scripts
Freeware license (only for research purpose, not for commercial) for 'data' directory which contains '.nc' files (any one can download it from TIGGE http://tigge-portal.ecmwf.int/ website) and '.png', '.jpg' images