Read data from MPALA ERDDAP

Exploring use of Python to formulate ERDDAP data requests and process the responses.


In [1]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

If you go to the ERDDAP TableDap page, you can select which variables you want, the time ranges, etc, and then select how you want to download the data. You can either then download the data, or just copy the URL that would download the data. That URL can therefore be used as the basis of a custom data query, as shown below. We simply generated a URL, then replaced the time and data requested with python variables.

Note: If you just see a blank box below, you might have to tell your browser to "allow unsafe script". In Chrome it's a little shield that shows up on the right hand side of the address bar.


In [2]:
from IPython.display import IFrame
IFrame('http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.html', width='100%', height=450)


Out[2]:

In [3]:
#select the variables you want
vars='Tsoil10cmGrass_Avg,Tsoil20cmGass_Avg'
# Use ERDDAP's built-in relative time functionality to get last 48 hours:
start='now-7days'
stop='now'
# or specify a specific period:
start = '2013-05-06T00:00:00Z'
stop =  '2013-06-07T00:00:00Z'

In [15]:
#construct the ERDDAP URL
url='http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.csvp?\
time,%s&time>=%s&time<=%s' % (vars,start,stop)
df = pd.read_csv(url,index_col=0,parse_dates=True)

In [16]:
df.plot(figsize=(12,4));



In [17]:
# List last ten records
df.tail(10)


Out[17]:
Tsoil10cmGrass_Avg (Deg C) Tsoil20cmGass_Avg (Deg C)
time (UTC)
2013-06-06 22:30:00 22.70 23.17
2013-06-06 22:40:00 22.67 23.16
2013-06-06 22:50:00 22.64 23.14
2013-06-06 23:00:00 22.62 23.12
2013-06-06 23:10:00 22.59 23.11
2013-06-06 23:20:00 22.56 23.09
2013-06-06 23:30:00 22.53 23.07
2013-06-06 23:40:00 22.51 23.06
2013-06-06 23:50:00 22.48 23.04
2013-06-07 00:00:00 22.45 23.02

In [18]:
df.describe()


Out[18]:
Tsoil10cmGrass_Avg (Deg C) Tsoil20cmGass_Avg (Deg C)
count 4609.000000 4609.000000
mean 22.559937 22.801701
std 1.066437 0.765300
min 20.070000 20.990000
25% 21.740000 22.230000
50% 22.520000 22.800000
75% 23.360000 23.340000
max 24.860000 24.420000

Use ERDDAP to make a plot


In [8]:
from IPython.display import Image
url='http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.png?time,TsoilOpen_Avg&time%3E=2014-08-27T00:00:00Z&time%3C=2014-09-03T00:00:00Z&.draw=lines&.color=0x000000'
Image(url=url,format=u'png')


Out[8]:

In [14]:
!git push


ssh: /home/usgs/anaconda/lib/libcrypto.so.1.0.0: no version information available (required by ssh)
Counting objects: 7, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 70.42 KiB, done.
Total 5 (delta 2), reused 0 (delta 0)
To git@github.com:rsignell-usgs/notebook.git
   3ac3a13..4837208  master -> master

In [10]:
pwd


Out[10]:
u'/home/usgs'

In [11]:
cd /usgs/data2/notebook/ERDDAP


/usgs/data2/notebook/ERDDAP

In [ ]: