In [1]:
name = '2015-11-13-meeting-summary'
title = '13 November Meeting Summary'
tags = 'anaconda, hpc, netcdf, xray'
author = 'Denis Sergeev'
In [3]:
from nb_tools import connect_notebook_to_post
from IPython.core.display import HTML
html = connect_notebook_to_post(name, title, tags, author)
Today was quite an informal meeting and since all the attendants were beginners to Python, we talked about basics.
<div class="alert alert-warning", style="font-size: 120%"> The instructions below have been summarised in a script which is available here: grace-python. </div>
After you log in to Grace, you should load the Anaconda module by typing the following command:
module load python/anaconda/2.3.0
Note that I'm using Python 2 here, but Anaconda with Python 3 is also available on Grace (you can find that module using the command module avail)
To be able to manage your Python distribution without calling IT, it's better to create an Anaconda environment in your home directory:
conda create -n myenv python anaconda
where myenv is just a name of the environment (could be anything, such as my_awesome_environment).
Now, typing
conda list
will print out the environments available to you. In my case the result was
myenv /gpfs/home/abc12xyz/.conda/envs/myenv
root * /gpfs/grace/anaconda/2.3.0
source activate myenv
This command appended the correct path to the system path.
(!) One thing that you should do is to clear standard environment variables:
unset PYTHONHOME PYTHONPATH
If you decided to stick with Anaconda, put the previous line in your .bashrc file.
Your Anaconda ecosystem is ready to use. Among many cool commands you can use conda install package_name to install a package.
To install xray: conda install xray
To install iris from SciTools channel (because iris is not in the default conda channel): conda install -c scitools iris
To test it, launch python (or better, ipython) and type import package_name.
In [3]:
import matplotlib.pyplot as plt
import numpy as np
We had a brief look at two packages that are widely used to work with data in NetCDF format: netcdf4 and xray.
Import the module (note the upper case)
In [4]:
import netCDF4 as nc
Open the file as a Dataset:
In [5]:
ds = nc.Dataset('../data/data.nc')
Print all variables:
In [6]:
ds.variables.keys()
Out[6]:
Interrogate the dimensions of the dataset:
In [7]:
ds.dimensions.keys()
Out[7]:
Get values of u variable:
In [8]:
u = ds.variables['u']
print('Name of u: {}'.format(u.long_name))
print('Units of u: {}'.format(u.units))
print('Shape of u: {}'.format(u.shape))
print('First element of u array: {}'.format(u[0,0,0,0]))
In [9]:
import xray
Open the same file as a xray Dataset:
In [10]:
xds = xray.open_dataset('../data/data.nc')
Neatly print out all the dataset info:
In [11]:
print(xds)
Select a variable and get its metadata:
In [12]:
u = xds.u
print(u)
During today's meeting a question was asked about interpolating gridded data (2D) to a finer grid. Luckily, there is a scipy function with a simple example:
In [13]:
HTML(html)
Out[13]: