Patrick BROCKMANN - LSCE (Climate and Environment Sciences Laboratory)
Updated: 2019/11/13
I wish to acknowledge use of the ferret and pyferret programs for analysis and graphics in this notebook.
Ferret is a product of NOAA's Pacific Marine Environmental Laboratory.
Information is available at http://ferret.pmel.noaa.gov/Ferret/
Install ferret http://ferret.pmel.noaa.gov/Ferret/downloads/downloading_ferret
or
Install pyferret http://ferret.pmel.noaa.gov/Ferret/downloads/pyferret/
To use pyferret inside a notebook, please install the extension ferretmagic from https://github.com/PBrockmann/ipython_ferretmagic
pip install ferretmagic
Other extension can be found from https://github.com/ipython/ipython/wiki/Extensions-Index
Launch a ferret session by typing:
$ ferret
You get a ferret prompt starting by:
yes?
In a ferret sessions, the command history is available from up/down arrow keys.
Remember also that CTRL+a goes to the beginning of the line and CTR+e to the end.
Note that ferret is not case sensitive
This will let you call pyferret from your notebook and embed all output in it
In [1]:
%load_ext ferretmagic
Remember jupyter notebook shortcuts (https://www.cheatography.com/weidadeyue/cheat-sheets/jupyter-notebook/)
In [2]:
%%ferret
cancel data/all
use levitus_climatology
show data
In [3]:
%%ferret -q
shade temp[k=1] ; go land
In [4]:
%%ferret?
In [5]:
%%ferret -s 600,400
cancel mode logo
plot temp[x=@ave,y=@ave]
In [6]:
%%ferret
show transforms
In [7]:
%%ferret -q -s 600,500
shade/lev=20v temp[k=@var] ; go land
In addition to all variables read from open datasets, you can define your own variables with the let command by typing for example:
yes? let temp1 = temp - 273.15
It will defined a new variable temp1 but will only set the relation. No calcul is made at this time.
Define a recursive variable like
yes? let temp = temp + 20.
has no sens in ferret.
You can of course combine variables from different datasets by specifying its dataset number [d=...]
yes? let var1 = varA[d=1] + varB[d=2]You can define conditionnal variable with the following syntax:
yes? let var3= if varA gt varB then varA else varC
The locigal condition can be gt, ge, eq, lt, le and several expressions can be combined using boolean operators (and, or).
For example:
yes? let tempselect = if (temp ge 10) and (temp lt 20) then 1
yes? shade tempselect[k=1]
define a mask with the condition 10 <= temp < 20
or:
yes? let saltselect = if temp gt 20 then salt
yes? shade saltselect[k=1] ; go land
produces a map of salinity only for water warmer than 20 degrees.
In [8]:
%%ferret -q -s 1000,500
set viewport left; shade salt[k=1] ; go land
let saltselect = if temp gt 20 then salt
set viewport right; shade/lev=20v saltselect[k=1] ; go land
You may want to work only on a sub-domain of a variable. For this you can define limits on the different dimensions of the variable by using the syntax
or with indices notations
Using this:
yes? shade salt[X=30:120, Y=-40:30, K=1] ; go land
will plot a map centered on the Indian Ocean (longitudes from 30° to 120° degrees East and latitudes from 40° South to 30° North).
In [9]:
%%ferret -q -s 400,400
shade salt[X=30:120, Y=-40:30, K=1] ; go land
We reduce the X axis and let free the vertical axis to get a slice of the Indian Ocean.
In [10]:
%%ferret -q -s 600,500
shade temp[X=30:120@AVE, Y=-40:30]
You can apply transformation to any of the dimensions of the variable and reduce it to a single scalar.
In [11]:
%%ferret -s 400,400
plot/grat=(dash)/line/symbol salt[X=30:120@ave, Y=-40:30@ave]
list salt[X=30:120@ave, Y=-40:30@ave, Z=@max]
Selections on physical axis are expressed in the units of the axis (express time with DD-MMM-YYYY format). East longitudes and North latitudes are defined to be positive.
Note that transformations are sequentially applied from X to N axis except for the @AVE transformation that is applied simultaneously on X and Y.
salt[X=30:120@ave, Y=-40:30@ave, Z=@max]
represents well the maximum of the average and not the average of the maximum.
The following plot represents the wind module from 1985 to 1989 years. Overlaid the same field smoothed with a running mean over 12 time steps.
In [12]:
%%ferret -s 600,400
cancel data/all
use monthly_navy_winds.cdf
ppl color 6, 70, 70, 70
let/title="wind module"/units="m/s" module=(UWND^2+VWND^2)^0.5
let/title="North Atlantic Wind module"/units="m/s" module_AN=module[x=260:360@ave,y=30:60@ave]
plot/grat=(dash, color=6) module_AN[t="01-JAN-1985":"01-JAN-1989"]
plot/over/thick=3/title="Smooth on 12 months" module_AN[t="01-JAN-1985":"01-JAN-1989"@SBX:12]
The definitions of the variables should as generic as possible, ie without domain specification. It is only when the plot or calculation commands are used that the limits must be specified. Do not hesitate to multiply the number of variable definitions, from the most general to the most limited and also for the sake of clarity.
In the same idea, operations on variables can only be done if the variables are compatible in dimension. Adding a vector defined on a longitude (X) to a vector defined on time (T) will have no meaning.
Note that you can specify a domain as a qualifier of a command to avoid repetitions inside variables. So
yes? plot varA[i=20:40,d=1], varB[i=20:40,d=1], varC[i=20:40,d=2]
could be written
yes? plot/i=20:40 varA[d=1], varB[d=1], varC[d=2]The transformation @ITP provides the same functionality as MODE INTERPOLATE with a greater level of control.
In [13]:
%%ferret -q -s 1000,500
cancel data/all
use levitus_climatology
set viewport left; shade/lev=20v temp[z=2450] ; go land
! use of the @itp to interpolate between layer (by default ferret will select the nearest layer)
set viewport right; shade/lev=20v temp[z=2450@itp] ; go land
Grid is the structure that hosts the data values. The grid is composed of axis. You can access to the underlaying grid of a variable by typing:
yes? show grid temp
yes? show grid/z temp
In [14]:
%%ferret
show grid/z temp
You can do dynamic regridding to pass from one axis to another one. This is a very powerfull feature of ferret and you should read the official documentation to better understand those capacities (http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/Grids-Regions/GRIDS#Chapter4-Regridding).
Let's make a very simple example. Imagine you would like to regrid the previons temp variable over a new vertical axis, regulary spaced.
In [15]:
%%ferret
def axis/z=0:5000:200/edges/units="meters" myNewZaxis
show axis/z myNewZaxis
let newTemp = temp[gz=myNewZaxis] ! @lin is applied by default
show grid newTemp
Now let make a time regredding to pass from a monthly variable to a yearly variable (read http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/Grids-Regions/GRIDS#_VPID_199)
The @AVE—averaging should be used here by typing:
let uwndYearly = uwnd[gt=yearsAxis@ave]
Computes the length-weighted average of all points on the source grid that lie partly or completely within each grid cell of the destination grid. If any portion of a source grid cell containing data overlaps a given destination grid cell, then data from that source cell contributes to the destination cell, weighted by the fraction of the destination cell overlapped by the source cell. The source data are treated as continuous, extending to the edges of the grid cells.
See the difference with other regriddings based on @lin or @nrst
In [16]:
%%ferret
ppl color 6, 70,70,70
cancel data/all
use monthly_navy_winds
show grid uwnd
def axis/t="01-JAN-1982":"31-DEC-1992":1/edges/units=year yearsAxis
show axis yearsAxis
let uwndYearly = uwnd[gt=yearsAxis@ave]
let uwndYearly2 = uwnd[gt=yearsAxis@lin]
let uwndYearly3 = uwnd[gt=yearsAxis@nrst]
plot/grat=(dash, color=6)/line/symb uwnd[i=@ave,j=@ave], uwndYearly[i=@ave,j=@ave], uwndYearly2[i=@ave,j=@ave], uwndYearly3[i=@ave,j=@ave]
Let's go further and compute the 12-month climatology from the initial monthly variable
In [17]:
%%ferret -q -s 800,800
ppl color 6, 70,70,70
cancel data/all
use monthly_navy_winds
let uwnd_clim = uwnd[gt=month_reg@mod]
def viewport/x=0:1/y=0.66:1 third1; set viewport third1
go margins 0.4 0.6 1 1 ! reduce the margins
plot/nolab/grat=(dash, color=6)/line/symb uwnd[i=@ave,j=@ave]
def viewport/x=0:1/y=0.33:0.66 third2; set viewport third2
go margins 0.4 0.6 1 1
plot/nolab/grat=(dash, color=6)/line/symb uwnd_clim[i=@ave,j=@ave]
def viewport/x=0:1/y=0:0.33 third3; set viewport third3
go margins 0.4 0.6 1 1
plot/nolab/grat=(dash, color=6)/T="01-JAN-1982":"31-DEC-1992" uwnd[i=@ave,j=@ave], uwnd_clim[i=@ave,j=@ave]
Now plot the difference between climatology and the variable by filling in red when it is over 0 and in blue when lower.
In [18]:
%%ferret -q -s 800,400
ppl color 6, 70,70,70
cancel data/all
use monthly_navy_winds
let uwnd_clim = uwnd[gt=month_reg@mod]
set viewport upper
go margins 0.4 0.6 1 1
plot/nolab/grat=(dash, color=6)/T="01-JAN-1982":"31-DEC-1992" uwnd[i=@ave,j=@ave], uwnd_clim[i=@ave,j=@ave]
set viewport lower
go margins 0.4 0.6 1 1
let a0 = uwnd[i=@ave,j=@ave]-uwnd_clim[i=@ave,j=@ave,T="01-JAN-1982":"31-DEC-1992"]
! regrid on a fine grid to have fills nicely done
def axis/t="01-JAN-1982":"31-DEC-1992":1/edges/units=days timeDays
let a = a0[gt=timeDays]
let a_pos = if a ge 0 then a else 0
let a_neg = if a le 0 then a else 0
plot/nolab/grat=(dash, color=6) a
go fill_between poly/nolab/over/palette=red a_pos timeDays
go fill_between poly/nolab/over/palette=blue a_neg timeDays
plot/nolab/over/line/color=1/thick=2 a
plot/nolab/over/line/color=1/dash=(0.1,0.1,0.1,0.1)/thick=2 a*0
!frame/file=fill_between.pdf
ferret takes care of missing values.
This can be seen with the stat command
In [19]:
%%ferret
stat uwnd
You can replace missing values from the nearest point by using the @FNR (pour fill nearest) or more simply replace them by a choosen value.
In [20]:
%%ferret
let a={23,45,,12,90}
list a, missing(a,0)
You can save the display as png or pdf by typing the frame command
yes? shade temp[k=@min] ; go land
yes? frame/file=myImage.png
yes? frame/file=myImage.pdf
Note that png format is available only with pyferret (and with ferretmagic), use gif format when using ferret.
Please choose carefully your levels and palette to focus on what you want to highlight.
You can locate a predefined palette by typing:
$ Fpalette red
An interesting proposition have been made to give up the rainbow palette (the default in ferret) by remplacing it by more appropriate ones to better represent your data, easier to read by those with colorblindness, and print well in grey scale. Please read https://cran.r-project.org/web/packages/viridis/vignettes/intro-to-viridis.html
$ Fpalette div
to look to divergente palette.
Visiting http://matplotlib.org/examples/color/colormaps_reference.html can help to choose the right one.
In [21]:
%%ferret -q -s 1000,500
cancel data/all
use monthly_navy_winds
let/title="wind module"/units="m/s" module=(UWND^2+VWND^2)^0.5
set viewport left
contour/fill/pal=mpl_PSU_plasma/lev=(-INF)(0,12,0.5)(INF) module[l=1] ; go land
set viewport right
! levels are specified using 20C = 20 levels centered
contour/fill/lev=20C/pal=mpl_Div_BrBG vwnd[l=1] ; go land
In [22]:
%%ferret -s 800,800
cancel mode logo
! increase the default number of colors (6)
set mode linecolors:20
cancel data/all
use monthly_navy_winds
! keep color black as 1
ppl color 2, 0, 50, 100, 50
ppl color 3, 100, 50, 0, 50
ppl color 4, 80, 0, 80, 50
ppl color 20, 70, 70, 70
! Define Wind Speed
let WND = (UWND^2 + VWND^2)^0.5
let/title="Wind Speed (global)" var1 = WND[i=@ave,j=@ave,d=1]
let/title="Wind Speed (North Hemisphere)" var2 = WND[i=@ave,y=0:90@ave,d=1]
let/title="Wind Speed (South Hemisphere)" var3 = WND[i=@ave,y=-90:0@ave,d=1]
! plot line(s) will be drawn starting with color 2
plot/color/thick=3/grat=(dash,color=20)/key=title var1, var2, var3
ferret is a complete software that can perfeclty be used to build your own library of scripts.
Remember that with a classic ferret session, a journal is recorded with all your commands. This file named ferret.jnl, once cleaned from your different tries can be run by a simple:
yes? go monscript.jnl
You can also generalize the script by adding arguments. Read http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/introduction/GO-FILES#Chapter1-Writing_GO_tools
Then pyferret could be used to run your script:
$ pyferret -nodisplay -script monscript.jnl arg1 arg2
In [ ]: