In [2]:
from notebook.services.config import ConfigManager
cfm = ConfigManager()
cfm.update('livereveal', {
              'theme': 'simple',
              'transition': 'convex',
              'start_slideshow_at': 'selected'
})
from IPython.core.display import HTML
HTML("""
<style>
.answers { 
    visibility: hidden;
}
</style>
""")


Out[2]:

Anaconda Installation

includes: python, ipython, spyder, jupyter, and more...

Author: Barron Henderson

Better instructions are elsewhere

  1. These were written for distribution for a specific workshop
  2. These might be out of date
  3. Anaconda website (https://www.continuum.io/downloads) may be more throrough
  4. If you're still reading, go ahead and use these.

Windows Installation

How-to

  1. Select an installer from below
  2. Download the installer
  3. unzip first, only if necessary
  4. Double click installer
  5. Follow instructions to install
    • if you have admin rights, for system
    • if you don't, just your user - install directly in user directory.

Windows not zipped

OS X graphical

  1. Download http://repo.continuum.io/archive/Anaconda3-4.0.0-MacOSX-x86_64.pkg
  2. Double click
  3. Follow instructions to install
    • if you have admin rights, for system
    • if you don't, just your user

OS X Command line

  1. Download: http://repo.continuum.io/archive/Anaconda3-4.0.0-MacOSX-x86_64.sh
  2. Open a terminal (command+space, type terminal, enter)
  3. cd ~/Downloads
  4. bash Anaconda3-4.0.0-MacOSX-x86_64.sh
  5. Follow instructions to install
    • if you have admin rights, for system
    • if you don't, just your user

In [ ]:
%%bash
cd ~/Downloads
curl -LO http://repo.continuum.io/archive/Anaconda3-4.0.0-MacOSX-x86_64.sh
bash Anaconda3-4.0.0-MacOSX-x86_64.sh

Linux

  1. Download one of the installers
  2. Open a terminal
  3. Navigate to your download.
  4. Run installer
    • bash Anaconda3-4.0.0-Linux-x86_64.sh
    • bash Anaconda3-4.0.0-Linux-x86.sh
  5. Follow instructions to install
    • if you have admin rights, for system
    • if you don't, just your user

Confirm Installation

  1. Find "Navigator"
  2. Open "Navigator"
  3. Launch I[py] qtconsol
  4. Compare forematter

     Jupyter QtConsole 4.2.0
     Python 3.5.1 |Anaconda custom (x86_64)| (default, Dec  7 2015, 11:24:55)  
     Type "copyright", "credits" or "license" for more information.
    
     IPython 4.1.2 -- An enhanced Interactive Python.  
     ?         -> Introduction and overview of IPython's features.  
     %quickref -> Quick reference.  
     help      -> Python's own help system.  
     object?   -> Details about 'object', use 'object??' for extra details.

Alternative Confirmation

  1. Open a terminal or dos prompt
  2. type ipython -ic "exit()"
  • On windows, ipython might require ipython.exe

In [1]:
!ipython -ic "exit()"


Python 3.5.1 |Anaconda custom (x86_64)| (default, Dec  7 2015, 11:24:55) 
Type "copyright", "credits" or "license" for more information.

IPython 4.1.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

Extra Libraries

Most libraries we use, come with Anaconda. Some, however, need a little version syncronizing.


In [ ]:
!conda install "python>=3.3" "hdf5>=1.8.15.1" "h5py>=2.5.0" "libnetcdf>=4.3.3.1" "netcdf4>=1.2.2" "numpy=1.10.4" "scipy>=0.17.0" "matplotlib>=1.5.1" "pandas>=0.18.0" "proj4>=4.9.1" "pyproj>=1.9.4" "pyyaml>=3.11" "yaml>=0.1.6" "zlib>=1.2.8" "jupyter"

In [ ]:
!pip install Shapely-1.5.16-cp34-cp34m-win_amd64.whl
!pip install basemap-1.0.8-cp34-none-win_amd64.whl

Mac


In [ ]:
!conda install "python>=3.3" "hdf5>=1.8.15.1" "h5py>=2.5.0" "libnetcdf>=4.3.3.1" "netcdf4>=1.2.2" "numpy>=1.8" "scipy>=0.17.0" "matplotlib>=1.5.1" "pandas>=0.18.0" "proj.4>=4.9.1" "pyproj>=1.9.4" "pyyaml>=3.11" "yaml>=0.1.6" "zlib>=1.2.8" "basemap>=1.0.7" "shapely>=1.5"

Install PNC Library

PseudoNetCDF is an important library for processing Atmospheric data formats (bpch, uamiv, etc).

ANSWER HIDDEN

``` !pip install http://github.com/barronh/pseudonetcdf/archive/master.zip ```

Run pncdump.py

Just as a test.

  1. Run with help option
  2. Run with dummy path

In [3]:
!pncdump.py --help


usage: pncdump.py [-h] [--pnc PNC]
                  [-f {see --list-formats for choices}]
                  [--list-format] [--help-format HELPFORMAT]
                  [--sep] [--inherit] [--mangle]
                  [--rename RENAME]
                  [--remove-singleton REMOVESINGLETON] [-H]
                  [-t] [--full-indices [c|f]] [-l LEN]
                  [--float-precision FDIG]
                  [--double-precision PDIG]
                  [--dump-name CDLNAME]
                  [--coordkeys key1,key2]
                  [-v varname1[,varname2[,...,varnameN]]
                  [-a att_nm,var_nm,mode,att_typ,att_val]
                  [-m MASKS] [--from-convention FROMCONV]
                  [--to-convention TOCONV] [--stack STACK]
                  [--merge] [-s dim,start[,stop[,step]]]
                  [-r dim,function[,weight]]
                  [--mesh dim,weight,function]
                  [-c dim,mode,wgt1,wgt2,...wgtN] [-e EXTRACT]
                  [--extract-file EXTRACTFILE]
                  [--extractmethod {nn,linear,cubic,quintic,KDTree}]
                  [--op-typ OPERATORS] [--expr EXPRESSIONS]
                  [--exprscript EXPRESSIONSCRIPTS]
                  [ifile [ifile ...]]

PseudoNetCDF Argument Parsing

positional arguments:
  ifile                 path to a file formatted as type -f

optional arguments:
  -h, --help            show this help message and exit
  --pnc PNC             Set of pseudonetcdf commands to be
                        process separately
  -f {see --list-formats for choices}, --format {see --list-formats for choices}
                        File format (default netcdf), can be
                        one of the choices listed, or an
                        expression that evaluates to a reader.
                        Keyword arguments are passed via
                        ,kwd=value.
  --list-format         Show format options for -f
  --help-format HELPFORMAT
                        Show help for file format (must be one
                        of the options for -f)
  --sep                 Used to separate groups of arguments
                        for parsing (e.g., pncgen --
                        [options1] file(s)1 [--sep [options2]
                        file(s)2 [... [--sep [optionsN]
                        file(s)N]]
  --inherit             Allow subparsed sections (separated
                        with -- and --sep) to inherit from
                        global options (-f, --format is always
                        inherited).
  --mangle              Remove non-standard ascii from names
  --rename RENAME       Provide pairs of strings to be
                        substituted
                        --rename=type,oldkey,newkey (type: v =
                        variable; d = dimension;)
  --remove-singleton REMOVESINGLETON
                        Remove singleton (length 1) dimensions
  -H, --header
  -t, --timestring
  --full-indices [c|f]  Provide indices in CDL using either C
                        or Fortran style indexes. C style is
                        0-based and ordered from slowest
                        iterating dimension to fastest.
                        Fortran style is 1-based and ordered
                        from fastest to slowest iterating
                        dimension
  -l LEN, --length LEN  CDL line length (pncdump only)
  --float-precision FDIG
                        single precision digitis (default 8;
                        pncdump only)
  --double-precision PDIG
                        pdig double precision digits (default
                        16; pncdump only)
  --dump-name CDLNAME   Name for display in ncdump
  --coordkeys key1,key2
                        Variables to be ignored in pncbo.
  -v varname1[,varname2[,...,varnameN], --variables varname1[,varname2[,...,varnameN]
                        Variable names or regular expressions
                        (using match) separated by ','. If a
                        group(s) has been specified, only
                        variables in that (those) group(s)
                        will be selected.
  -a att_nm,var_nm,mode,att_typ,att_val, --attribute att_nm,var_nm,mode,att_typ,att_val
                        Variables have attributes that can be
                        added following nco syntax
                        (--attribute
                        att_nm,var_nm,mode,att_typ,att_val);
                        mode = a,c,d,m,o and att_typ =
                        f,d,l,s,c,b; att_typ is any valid
                        numpy type.
  -m MASKS, --mask MASKS
                        Masks to apply (e.g., greater,0 or
                        less,0 or values,0, or where,(time[:]%
                        24<12)[:,None].repeat(10,1))
  --from-convention FROMCONV
                        From convention currently only support
                        ioapi
  --to-convention TOCONV
                        To convention currently only supports
                        cf
  --stack STACK         Concatentate (stack) files on the
                        dimension.
  --merge               Combine variables into one file
  -s dim,start[,stop[,step]], --slice dim,start[,stop[,step]]
                        Variables have dimensions (time,
                        layer, lat, lon), which can be subset
                        using dim,start,stop,stride (e.g.,
                        --slice=layer,0,47,5 would sample
                        every fifth layer starting at 0)
  -r dim,function[,weight], --reduce dim,function[,weight]
                        Variable dimensions can be reduced
                        using dim,function,weight syntax
                        (e.g., --reduce=layer,mean,weight).
                        Weighting is not fully functional.
  --mesh dim,weight,function
                        Variable dimensions can be meshed
                        using dim,function,weight syntax
                        (e.g., --mesh=time,0.5,mean).
  -c dim,mode,wgt1,wgt2,...wgtN, --convolve dim,mode,wgt1,wgt2,...wgtN
                        Variable dimension is reduced by
                        convolve function
                        (dim,mode,wgt1,wgt2,...wgtN)
  -e EXTRACT, --extract EXTRACT
                        lon/lat coordinates to extract lon1,la
                        t1/lon2,lat2/lon3,lat3/.../lonN,latN
  --extract-file EXTRACTFILE
                        pncparse options for file
  --extractmethod {nn,linear,cubic,quintic,KDTree}
                        Method for extraction
  --op-typ OPERATORS    Operator for binary file operations.
                        Binary file operations use the first
                        two files, then the result and the
                        next file, etc. Use // or <= or % or
                        is not or >> or & or == or != or + or
                        * or - or / or < or >= or ** or > or
                        << or | or is or ^
  --expr EXPRESSIONS    Generic expressions to execute in the
                        context of the file.
  --exprscript EXPRESSIONSCRIPTS
                        Generic expressions to execute in the
                        context of the file.

Detailed Steps
--------------

PseudoNetCDF has many operations and the order often matters. The order is consistent with the order of options in the formatted help. The default order is summarized as:

1. Open with specified reader (-f)
2. Select subset of variables (-v)
2. Add attributes (-a)
4. Apply masks (--mask)
5. Add conventions to support later operations (--to-convention, --from-convention)
6. Combine files via stacking on dimensions (--stack)
7. Slice dimensions (-s --slice)
8. Reduce dimensions (-r --reduce)
9. Convolve dimensions (-c)
10. Extract specific coordinates (--extract)
11. Remove singleton dimensions (--remove-singleton)
12. Apply expressions (--expr then --exprscripts)
13. Apply binary operators (--op_typ)

To impose your own order, use standard options (global options) and then use -- to force positional interpretation of remaining options. In remaining options, use --sep to separate groups of files and options to be evaluated before any global operations.

In [4]:
!pncdump.py sdf


Traceback (most recent call last):
  File "/Users/barronh/Development/pseudonetcdf/src/PseudoNetCDF/pncparse.py", line 408, in getfiles
    f = allreaders[file_format](ipath, **format_options)
  File "netCDF4/_netCDF4.pyx", line 1795, in netCDF4._netCDF4.Dataset.__init__ (netCDF4/_netCDF4.c:12271)
RuntimeError: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/barronh/Development/pseudonetcdf/scripts/pncdump.py", line 3, in <module>
    main()
  File "/Users/barronh/Development/pseudonetcdf/src/PseudoNetCDF/pncdump.py", line 286, in main
    ifiles, options = pncparse(has_ofile = False)
  File "/Users/barronh/Development/pseudonetcdf/src/PseudoNetCDF/pncparse.py", line 303, in pncparse
    ifiles.extend(pncprep(subarg)[0])
  File "/Users/barronh/Development/pseudonetcdf/src/PseudoNetCDF/pncparse.py", line 359, in pncprep
    fs = getfiles(ipaths, args)
  File "/Users/barronh/Development/pseudonetcdf/src/PseudoNetCDF/pncparse.py", line 413, in getfiles
    raise oute from e
OSError: Unable to open path with netcdf(path, **{})
	path="sdf"
	error="No such file or directory"

Windows file association

  1. Do not make your own file associations.
  2. If you do, you may need to delete it (http://websistent.com/delete-file-association-windows-7/)
  3. If that still doesn't work, open a dos prompt.

In [ ]:
!assoc .py=Python.File
!ftype Python.File="C:\Users\barronh\Anaconda3\python.exe" "%1" %*