Read data from MPALA ERDDAP

Exploring use of Python to formulate ERDDAP data requests and process the responses.


In [25]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

If you go to the ERDDAP TableDap page, you can select which variables you want, the time ranges, etc, and then select how you want to download the data. You can either then download the data, or just copy the URL that would download the data. That URL can therefore be used as the basis of a custom data query, as shown below. We simply generated a URL, then replaced the time and data requested with python variables.

Note: If you just see a blank box below, you might have to tell your browser to "allow unsafe script". In Chrome it's a little shield that shows up on the right hand side of the address bar.


In [26]:
from IPython.display import IFrame
IFrame('http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.html', width='100%', height=450)


Out[26]:

In [27]:
#select the variables you want
vars='Tsoil10cmGrass_Avg,Tsoil20cmGass_Avg'
# Use ERDDAP's built-in relative time functionality to get last 48 hours:
start='now-7days'
stop='now'
# or specify a specific period:
start = '2013-05-06T00:00:00Z'
stop =  '2013-06-07T00:00:00Z'

In [34]:
#construct ERDDAP URL to get station info
url='http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.csv?station_id,longitude,latitude,altitude&time=2012-12-21T16:30:00Z'
df_sta = pd.read_csv(url,skiprows=[1])

In [35]:
df_sta.head()


Out[35]:
station_id longitude latitude altitude
0 MPALA Tower 36.9 0.5 0

In [37]:
#construct ERDDAP URL to get time series data
url='http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.csvp?\
time,%s&time>=%s&time<=%s' % (vars,start,stop)
df = pd.read_csv(url,index_col=0,parse_dates=True)

In [38]:
df.plot(figsize=(12,4),title=df_sta['station_id'][0]);



In [17]:
# List last ten records
df.tail(10)


Out[17]:
Tsoil10cmGrass_Avg (Deg C) Tsoil20cmGass_Avg (Deg C)
time (UTC)
2013-06-06 22:30:00 22.70 23.17
2013-06-06 22:40:00 22.67 23.16
2013-06-06 22:50:00 22.64 23.14
2013-06-06 23:00:00 22.62 23.12
2013-06-06 23:10:00 22.59 23.11
2013-06-06 23:20:00 22.56 23.09
2013-06-06 23:30:00 22.53 23.07
2013-06-06 23:40:00 22.51 23.06
2013-06-06 23:50:00 22.48 23.04
2013-06-07 00:00:00 22.45 23.02

In [18]:
df.describe()


Out[18]:
Tsoil10cmGrass_Avg (Deg C) Tsoil20cmGass_Avg (Deg C)
count 4609.000000 4609.000000
mean 22.559937 22.801701
std 1.066437 0.765300
min 20.070000 20.990000
25% 21.740000 22.230000
50% 22.520000 22.800000
75% 23.360000 23.340000
max 24.860000 24.420000

The following example is just to illustate that you can have ERDDAP make plots also via a URL, but that's a bit silly here since we have the full power of python at our fingertips. But can be useful for web pages and other applications


In [8]:
from IPython.display import Image
url='http://geoport.whoi.edu/erddap/tabledap/tower_65ce_ba2b_9a66.png?time,TsoilOpen_Avg&time%3E=2014-08-27T00:00:00Z&time%3C=2014-09-03T00:00:00Z&.draw=lines&.color=0x000000'
Image(url=url,format=u'png')


Out[8]: