Notebook to analyze downloaded gridded climate time-series data

Case study: the Sauk-Suiattle Watershed

Use this Jupyter Notebook to:

1. HydroShare setup and preparation
2. Re-establish the paths to the mapping file
3. Compute daily, monthly, and annual temperature and precipitation statistics
4. Visualize precipitation results relative to the forcing data
5. Visualize the time-series trends
6. Save results back into HydroShare






This data is compiled to digitally observe the watersheds, powered by HydroShare.
Provided by the Watershed Dynamics Group, Dept. of Civil and Environmental Engineering, University of Washington

1. HydroShare Setup and Preparation

To run this notebook, we must import several libaries. These are listed in order of 1) Python standard libraries, 2) hs_utils library provides functions for interacting with HydroShare, including resource querying, dowloading and creation, and 3) the observatory_gridded_hydromet library that is downloaded with this notebook.


In [1]:
# data processing
import os
import pandas as pd, numpy as np, dask, json
import ogh

# data migration library
from utilities import hydroshare

# plotting and shape libraries
%matplotlib inline

# silencing warning
import warnings
warnings.filterwarnings("ignore")

In [2]:
# initialize ogh_meta
meta_file = dict(ogh.ogh_meta())
sorted(meta_file.keys())


Out[2]:
['dailymet_bclivneh2013',
 'dailymet_livneh2013',
 'dailymet_livneh2015',
 'dailyvic_livneh2013',
 'dailyvic_livneh2015',
 'dailywrf_bcsalathe2014',
 'dailywrf_salathe2014']

In [3]:
sorted(meta_file['dailymet_livneh2013'].keys())


Out[3]:
['decision_steps',
 'delimiter',
 'domain',
 'end_date',
 'file_format',
 'filename_structure',
 'reference',
 'spatial_resolution',
 'start_date',
 'subdomain',
 'temporal_resolution',
 'variable_info',
 'variable_list',
 'web_protocol']

Establish a secure connection with HydroShare by instantiating the hydroshare class that is defined within hs_utils. In addition to connecting with HydroShare, this command also sets and prints environment variables for several parameters that will be useful for saving work back to HydroShare.


In [4]:
notebookdir = os.getcwd()

hs=hydroshare.hydroshare()
homedir = hs.getContentPath(os.environ["HS_RES_ID"])
os.chdir(homedir)
print('Data will be loaded from and save to:'+homedir)


Adding the following system variables:
   HS_USR_NAME = jphuong
   HS_RES_ID = 87dc5742cf164126a11ff45c3307fd9d
   HS_RES_TYPE = compositeresource
   JUPYTER_HUB_IP = jupyter.cuahsi.org

These can be accessed using the following command: 
   os.environ[key]

   (e.g.)
   os.environ["HS_USR_NAME"]  => jphuong
Successfully established a connection with HydroShare
Data will be loaded from and save to:/home/jovyan/work/notebooks/data/87dc5742cf164126a11ff45c3307fd9d/87dc5742cf164126a11ff45c3307fd9d/data/contents

If you are curious about where the data is being downloaded, click on the Jupyter Notebook dashboard icon to return to the File System view. The homedir directory location printed above is where you can find the data and contents you will download to a HydroShare JupyterHub server. At the end of this work session, you can migrate this data to the HydroShare iRods server as a Generic Resource.

2. Get list of gridded climate points for the watershed

For visualization purposes, we will also remap the study site shapefile, which is stored in HydroShare at the following url: https://www.hydroshare.org/resource/c532e0578e974201a0bc40a37ef2d284/. Since the shapefile was previously migrated, we can select 'N' for no overwriting.

In the usecase1 notebook, the treatgeoself function identified the gridded cell centroid coordinates that overlap with our study site. These coordinates were documented within the mapping file, which will be remapped here. In the usecase2 notebook, the downloaded files were cataloged within the mapping file, so we will use the mappingfileSummary function to characterize the files available for Sauk-Suiattle for each gridded data product.


In [5]:
"""
Sauk
"""
# Watershed extent
hs.getResourceFromHydroShare('c532e0578e974201a0bc40a37ef2d284')
sauk = hs.content['wbdhub12_17110006_WGS84_Basin.shp']

# reproject the shapefile into WGS84
ogh.reprojShapefile(sourcepath=sauk)


This resource already exists in your userspace.
c532e0578e974201a0bc40a37ef2d284/
|-- c532e0578e974201a0bc40a37ef2d284/
|   |-- bagit.txt
|   |-- manifest-md5.txt
|   |-- readme.txt
|   |-- tagmanifest-md5.txt
|   |-- data/
|   |   |-- resourcemap.xml
|   |   |-- resourcemetadata.xml
|   |   |-- contents/
|   |   |   |-- wbdhub12_17110006_WGS84_Basin.cpg
|   |   |   |-- wbdhub12_17110006_WGS84_Basin.shp
|   |   |   |-- wbdhub12_17110006_WGS84_Basin.shx
|   |   |   |-- wbdhub12_17110006_WGS84_Basin.dbf
|   |   |   |-- wbdhub12_17110006_WGS84_Basin.prj
|   |   |   |-- wbdhub12_17110006_WGS84_Basin_2.shp
|   |   |   |-- wbdhub12_17110006_WGS84_Basin_2.shx
|   |   |   |-- wbdhub12_17110006_WGS84_Basin_2.dbf
|   |   |   |-- wbdhub12_17110006_WGS84_Basin_2.cpg
|   |   |   |-- wbdhub12_17110006_WGS84_Basin_2.prj

Do you want to overwrite these data [Y/n]? n
Found the following file(s) associated with this HydroShare resource.
wbdhub12_17110006_WGS84_Basin.cpg
wbdhub12_17110006_WGS84_Basin.shp
wbdhub12_17110006_WGS84_Basin.shx
wbdhub12_17110006_WGS84_Basin.dbf
wbdhub12_17110006_WGS84_Basin.prj
wbdhub12_17110006_WGS84_Basin_2.shp
wbdhub12_17110006_WGS84_Basin_2.shx
wbdhub12_17110006_WGS84_Basin_2.dbf
wbdhub12_17110006_WGS84_Basin_2.cpg
wbdhub12_17110006_WGS84_Basin_2.prj
These files are stored in a dictionary called hs.content for your convenience. To access a file, simply issue the following command where MY_FILE is one of the files listed above:
hs.content["MY_FILE"] 

Summarize the file availability from each watershed mapping file


In [6]:
# map the mappingfiles from usecase1
mappingfile1 = os.path.join(homedir,'Sauk_mappingfile.csv')

t1 = ogh.mappingfileSummary(listofmappingfiles = [mappingfile1], 
                            listofwatershednames = ['Sauk-Suiattle river'],
                            meta_file=meta_file)
t1


Out[6]:
Watershed Sauk-Suiattle river
Median elevation in meters [range](Number of gridded cells) 1171[164-2216] (n=99)
dailymet_livneh2013 1171[164-2216] (n=99)
dailymet_bclivneh2013 1171[164-2216] (n=99)
dailymet_livneh2015 1171[164-2216] (n=99)
dailyvic_livneh2013 1171[164-2216] (n=99)
dailyvic_livneh2015 1171[164-2216] (n=99)
dailywrf_salathe2014 1171[164-2216] (n=99)
dailywrf_bcsalathe2014 1171[164-2216] (n=99)

3. Compute Hydrometeorology statistics

This section performs computations and generates plots of the Livneh 2013 and Salathe 2014 mean temperature and mean total monthly precipitation in order to compare them with each other. The generated plots are automatically downloaded and saved as .png files within the "homedir" directory.

Let's compare the Livneh 2013 and Salathe 2014 using the period of overlapping history.


In [7]:
# Livneh et al., 2013
dr1 = meta_file['dailymet_livneh2013']

# Salathe et al., 2014
dr2 = meta_file['dailywrf_salathe2014']

# define overlapping time window
dr = ogh.overlappingDates(date_set1=tuple([dr1['start_date'], dr1['end_date']]), 
                          date_set2=tuple([dr2['start_date'], dr2['end_date']]))
dr


Out[7]:
('1950-01-01', '2010-12-31')

Create a dictionary of climate variables for the long-term mean (ltm).

INPUT: gridded meteorology ASCII files located from the Sauk-Suiattle Mapping file. The inputs to gridclim_dict() include the folder location and name of the hydrometeorology data, the file start and end, the analysis start and end, and the elevation band to be included in the analsyis (max and min elevation).
OUTPUT: dictionary of dataframes where rows are temporal summaries and columns are spatial summaries


In [8]:
%%time
ltm = ogh.gridclim_dict(mappingfile=mappingfile1,
                        metadata=meta_file,
                        dataset='dailymet_livneh2013',
                        subset_start_date=dr[0],
                        subset_end_date=dr[1])


Number of data files within elevation range (164.0-2216.0 m): 99
PRECIP dataframe reading complete:0:00:01.804637
TMAX dataframe reading complete:0:00:03.152684
TMIN dataframe reading complete:0:00:04.504587
WINDSPD dataframe reading complete:0:00:05.856180
PRECIP_dailymet_livneh2013 calculations completed in 0:00:00.078095
TMAX_dailymet_livneh2013 calculations completed in 0:00:00.056387
TMIN_dailymet_livneh2013 calculations completed in 0:00:00.054475
WINDSPD_dailymet_livneh2013 calculations completed in 0:00:00.059460
CPU times: user 14.3 s, sys: 3.07 s, total: 17.4 s
Wall time: 6.12 s

Compute the total monthly and yearly precipitation, as well as the mean values across time and across stations

INPUT: daily precipitation for each station from the long-term mean dictionary (ltm)
OUTPUT: Append the computed dataframes and values into the ltm dictionary


In [9]:
# compute sums and mean monthly an yearly sums
ltm = ogh.aggregate_space_time_sum(df_dict=ltm,
                                   suffix='PRECIP_dailymet_livneh2013',
                                   start_date=dr[0],
                                   end_date=dr[1])


PRECIP_dailymet_livneh2013 calculations completed in 0:00:00.208475

Repeat spatial-temporal computations with Salathe et al 2014 WRF data


In [10]:
%%time
ltm = ogh.gridclim_dict(mappingfile=mappingfile1,
                        metadata=meta_file,
                        dataset='dailywrf_salathe2014',
                        subset_start_date=dr[0],
                        subset_end_date=dr[1],
                        df_dict=ltm)

# compute sums and mean monthly an yearly sums
ltm = ogh.aggregate_space_time_sum(df_dict=ltm,
                                   suffix='PRECIP_dailywrf_salathe2014',
                                   start_date=dr[0],
                                   end_date=dr[1])


Number of data files within elevation range (164.0-2216.0 m): 99
PRECIP dataframe reading complete:0:00:01.671365
TMAX dataframe reading complete:0:00:02.980384
TMIN dataframe reading complete:0:00:04.298338
WINDSPD dataframe reading complete:0:00:05.719839
PRECIP_dailywrf_salathe2014 calculations completed in 0:00:00.067408
TMAX_dailywrf_salathe2014 calculations completed in 0:00:00.065687
TMIN_dailywrf_salathe2014 calculations completed in 0:00:00.065989
WINDSPD_dailywrf_salathe2014 calculations completed in 0:00:00.071531
PRECIP_dailywrf_salathe2014 calculations completed in 0:00:00.204810
CPU times: user 9.95 s, sys: 1.49 s, total: 11.4 s
Wall time: 6.21 s

In [11]:
# print the name of the Livneh et al. 2013 raw MET and Salathe et al. 2014 raw WRF dataframes and values within ltm
sorted(ltm.keys())


Out[11]:
['PRECIP_dailymet_livneh2013',
 'PRECIP_dailywrf_salathe2014',
 'TMAX_dailymet_livneh2013',
 'TMAX_dailywrf_salathe2014',
 'TMIN_dailymet_livneh2013',
 'TMIN_dailywrf_salathe2014',
 'WINDSPD_dailymet_livneh2013',
 'WINDSPD_dailywrf_salathe2014',
 'anomyear_PRECIP_dailymet_livneh2013',
 'anomyear_PRECIP_dailywrf_salathe2014',
 'anomyear_TMAX_dailymet_livneh2013',
 'anomyear_TMAX_dailywrf_salathe2014',
 'anomyear_TMIN_dailymet_livneh2013',
 'anomyear_TMIN_dailywrf_salathe2014',
 'anomyear_WINDSPD_dailymet_livneh2013',
 'anomyear_WINDSPD_dailywrf_salathe2014',
 'meanalldailysum_PRECIP_dailymet_livneh2013',
 'meanalldailysum_PRECIP_dailywrf_salathe2014',
 'meanallmonthsum_PRECIP_dailymet_livneh2013',
 'meanallmonthsum_PRECIP_dailywrf_salathe2014',
 'meanallyear_PRECIP_dailymet_livneh2013',
 'meanallyear_PRECIP_dailywrf_salathe2014',
 'meanallyear_TMAX_dailymet_livneh2013',
 'meanallyear_TMAX_dailywrf_salathe2014',
 'meanallyear_TMIN_dailymet_livneh2013',
 'meanallyear_TMIN_dailywrf_salathe2014',
 'meanallyear_WINDSPD_dailymet_livneh2013',
 'meanallyear_WINDSPD_dailywrf_salathe2014',
 'meanallyearsum_PRECIP_dailymet_livneh2013',
 'meanallyearsum_PRECIP_dailywrf_salathe2014',
 'meanbydaily_PRECIP_dailymet_livneh2013',
 'meanbydaily_PRECIP_dailywrf_salathe2014',
 'meanbydaily_TMAX_dailymet_livneh2013',
 'meanbydaily_TMAX_dailywrf_salathe2014',
 'meanbydaily_TMIN_dailymet_livneh2013',
 'meanbydaily_TMIN_dailywrf_salathe2014',
 'meanbydaily_WINDSPD_dailymet_livneh2013',
 'meanbydaily_WINDSPD_dailywrf_salathe2014',
 'meanbymonth_PRECIP_dailymet_livneh2013',
 'meanbymonth_PRECIP_dailywrf_salathe2014',
 'meanbymonth_TMAX_dailymet_livneh2013',
 'meanbymonth_TMAX_dailywrf_salathe2014',
 'meanbymonth_TMIN_dailymet_livneh2013',
 'meanbymonth_TMIN_dailywrf_salathe2014',
 'meanbymonth_WINDSPD_dailymet_livneh2013',
 'meanbymonth_WINDSPD_dailywrf_salathe2014',
 'meanbymonthsum_PRECIP_dailymet_livneh2013',
 'meanbymonthsum_PRECIP_dailywrf_salathe2014',
 'meanbyyear_PRECIP_dailymet_livneh2013',
 'meanbyyear_PRECIP_dailywrf_salathe2014',
 'meanbyyear_TMAX_dailymet_livneh2013',
 'meanbyyear_TMAX_dailywrf_salathe2014',
 'meanbyyear_TMIN_dailymet_livneh2013',
 'meanbyyear_TMIN_dailywrf_salathe2014',
 'meanbyyear_WINDSPD_dailymet_livneh2013',
 'meanbyyear_WINDSPD_dailywrf_salathe2014',
 'meanbyyearsum_PRECIP_dailymet_livneh2013',
 'meanbyyearsum_PRECIP_dailywrf_salathe2014',
 'meandaily_PRECIP_dailymet_livneh2013',
 'meandaily_PRECIP_dailywrf_salathe2014',
 'meandaily_TMAX_dailymet_livneh2013',
 'meandaily_TMAX_dailywrf_salathe2014',
 'meandaily_TMIN_dailymet_livneh2013',
 'meandaily_TMIN_dailywrf_salathe2014',
 'meandaily_WINDSPD_dailymet_livneh2013',
 'meandaily_WINDSPD_dailywrf_salathe2014',
 'meanmonth_PRECIP_dailymet_livneh2013',
 'meanmonth_PRECIP_dailywrf_salathe2014',
 'meanmonth_TMAX_dailymet_livneh2013',
 'meanmonth_TMAX_dailywrf_salathe2014',
 'meanmonth_TMIN_dailymet_livneh2013',
 'meanmonth_TMIN_dailywrf_salathe2014',
 'meanmonth_WINDSPD_dailymet_livneh2013',
 'meanmonth_WINDSPD_dailywrf_salathe2014',
 'meanmonthsum_PRECIP_dailymet_livneh2013',
 'meanmonthsum_PRECIP_dailywrf_salathe2014',
 'meanyear_PRECIP_dailymet_livneh2013',
 'meanyear_PRECIP_dailywrf_salathe2014',
 'meanyear_TMAX_dailymet_livneh2013',
 'meanyear_TMAX_dailywrf_salathe2014',
 'meanyear_TMIN_dailymet_livneh2013',
 'meanyear_TMIN_dailywrf_salathe2014',
 'meanyear_WINDSPD_dailymet_livneh2013',
 'meanyear_WINDSPD_dailywrf_salathe2014',
 'meanyearsum_PRECIP_dailymet_livneh2013',
 'meanyearsum_PRECIP_dailywrf_salathe2014',
 'monthsum_PRECIP_dailymet_livneh2013',
 'monthsum_PRECIP_dailywrf_salathe2014',
 'yearsum_PRECIP_dailymet_livneh2013',
 'yearsum_PRECIP_dailywrf_salathe2014']

In [11]:
# initialize list of outputs
files=[]

# create the destination path for the dictionary of dataframes
ltm_sauk=os.path.join(homedir, 'ltm_sauk.json')
ogh.saveDictOfDf(dictionaryObject=ltm, outfilepath=ltm_sauk)
files.append(ltm_sauk)

# append the mapping file for Sauk-Suiattle gridded cell centroids
files.append(mappingfile1)

Visualize the Average Monthly Total Precipitations

INPUT: dataframe with each month as a row and each station as a column.
OUTPUT: A png file that represents the distribution across stations (in Wateryear order)


In [12]:
# # two lowest elevation locations
lowE_ref = ogh.findCentroidCode(mappingfile=mappingfile1, colvar='ELEV', colvalue=164)

# one highest elevation location
highE_ref = ogh.findCentroidCode(mappingfile=mappingfile1, colvar='ELEV', colvalue=2216)

# combine references together
reference_lines = highE_ref + lowE_ref
reference_lines


Out[12]:
[(69, 48.09375, -121.09375),
 (3, 48.46875, -121.59375),
 (24, 48.34375, -121.53125)]

In [13]:
# consider value range when comparing Livneh to Salathe 
vr = ogh.valueRange([ltm['meanbymonthsum_PRECIP_dailymet_livneh2013'], 
                     ltm['meanbymonthsum_PRECIP_dailywrf_salathe2014']])

print(vr.min(), vr.max())


35.0352458852 579.924590705

In [19]:
def in_to_mm(inch):
    mm = inch*25.4
    return(mm)


def F_to_C(F):
    C = (32*F - 32) * (5/9)
    return(C)

In [20]:
label = 'Darrington ranger station'
PRECIP_obs_in = {1:11.89, 2:8.84, 3:8.25, 4:5.09, 5:3.50, 6:2.84, 7:1.53, 8:1.71, 9:3.64, 10:7.42, 11:11.65, 12:13.12}
TMAX_obs_F = {1:40.7, 2:46.1, 3:52.0, 4:59.5, 5:66.5, 6:70.6, 7:77.3, 8:77.2, 9:71.1, 10:60.5, 11:48.0, 12:42.2}
TMIN_obs_F = {1:27.7,2:29.9,3:32.9,4:36.7,5:42.6,6:47.7,7:50.4,8:50.8,9:45.6,10:39.3,11:33.3,12:30.2}

# convert the units and rearrange to wateryear month order
wateryear=[10,11,12,1,2,3,4,5,6,7,8,9]
PRECIP_obs_mm = pd.DataFrame([in_to_mm(PRECIP_obs_in[ind]) for ind in wateryear]).rename(columns={0:label})
TMAX_obs_C = pd.DataFrame([F_to_C(TMAX_obs_F[ind]) for ind in wateryear]).rename(columns={0:label})
TMIN_obs_C = pd.DataFrame([F_to_C(TMIN_obs_F[ind]) for ind in wateryear]).rename(columns={0:label})

In [21]:
# plot the distribution of total monthly precipitation by month across stations
outfile=os.path.join(homedir, 'meanbymonthsumprecip_liv_bx.png')
files.append(outfile)
ax1 = ogh.renderValueInBoxplot(vardf=ltm['meanbymonthsum_PRECIP_dailymet_livneh2013'],
                               vmin=vr.min()-1,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='Sauk-Suiattle average total precipitation by month',
                               time_steps='month',
                               wateryear=True,
                               value_name='Average monthly precipitation (mm)',
                               cmap='seismic_r',
                               figsize=(10,8),
                               reference_lines = reference_lines,
                               obs_datavector=[PRECIP_obs_mm.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'])

# plot the distribution of total monthly precipitation by month across stations
outfile=os.path.join(homedir, 'meanbymonthsumprecip_wrf_bx.png')
files.append(outfile)
ax1 = ogh.renderValueInBoxplot(vardf=ltm['meanbymonthsum_PRECIP_dailywrf_salathe2014'],
                               vmin=0,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='',
                               time_steps='month',
                               wateryear=True,
                               value_name='',
                               cmap='seismic_r',
                               figsize=(10,8),
                               reference_lines = reference_lines,
                               obs_datavector=[PRECIP_obs_mm.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'],
                               ref_legend=False)


Visualize the "average monthly total precipitation"


In [26]:
%%time

# loop through even months in the water year with Livneh et al. (2013)
for ind, eachmonth in enumerate([10, 1, 4]):
    
    monthlabel = pd.datetime.strptime(str(eachmonth), '%m')
    outfile=os.path.join(homedir, 'SaukLivPrecip_{0}.png'.format(monthlabel.strftime('%b')))
    files.append(outfile)
    ogh.renderValuesInPoints(vardf=ltm['meanbymonthsum_PRECIP_dailymet_livneh2013'], 
                             vardf_dateindex=eachmonth,
                             shapefile=sauk,
                             outfilepath=outfile,
                             plottitle='Sauk-Suiattle watershed\nAverage total monthly Precipitation in '+ monthlabel.strftime('%B'),
                             colorbar_label='Average monthly precipitation (mm)',
                             spatial_resolution=1/16, margin=0.25, epsg=3857,
                             basemap_image='ESRI_Imagery_World_2D',
                             cmap='seismic_r',
                             figsize=(2,2))


CPU times: user 47.1 s, sys: 3.7 s, total: 50.8 s
Wall time: 48.5 s

Visualize the spatial distribution with the Salathe et al., 2014 WRF data


In [27]:
%%time
"""loop through even months in the water year for Salathe et al. (2014)"""
for ind, eachmonth in enumerate([10, 1, 4]):
    
    monthlabel = pd.datetime.strptime(str(eachmonth), '%m')
    outfile=os.path.join(homedir, 'SaukWRFPrecip_{0}.png'.format(monthlabel.strftime('%b')))
    files.append(outfile)
    ogh.renderValuesInPoints(vardf=ltm['meanbymonthsum_PRECIP_dailywrf_salathe2014'],
                             vardf_dateindex=eachmonth,
                             shapefile=sauk,
                             outfilepath=outfile,
                             plottitle='',
                             colorbar_label='Average monthly precipitation (mm)',
                             spatial_resolution=1/16, margin=0.25, epsg=3857,
                             basemap_image='ESRI_Imagery_World_2D',
                             cmap='seismic_r',
                             figsize=(2,2))


CPU times: user 46.7 s, sys: 3.62 s, total: 50.4 s
Wall time: 48.7 s

Compare Temperature


In [19]:
%%time

"""Creating a 2x2 comparison figure of daily TMAX and TMIN by Livneh et al. (2013) versus Salathe et al.(2014)"""

vr = ogh.valueRange([ltm['meanbymonth_TMAX_dailymet_livneh2013'], ltm['meanbymonth_TMIN_dailymet_livneh2013'],
                     ltm['meanbymonth_TMAX_dailywrf_salathe2014'], ltm['meanbymonth_TMIN_dailywrf_salathe2014']])


# Generate top-left figure with Livneh et al. (2013) TMAX (with legend and yaxis label)
eachdf = 'meanbymonth_TMAX_dailymet_livneh2013'
outfile=os.path.join(homedir, '{0}.png'.format(eachdf))
files.append(outfile)
ax1 = ogh.renderValueInBoxplot(vardf=ltm[eachdf],
                               vmin=vr.min()-1,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='',
                               time_steps='month',
                               wateryear=True,
                               value_name='Average daily maximum temperature (C)',
                               cmap='gnuplot2',
                               figsize=(8,8),
                               reference_lines=reference_lines,
                               ref_legend_loc=2,
                               obs_legend_loc=3,
                               obs_datavector=[TMAX_obs_C.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'])

# Generate bottom-left figure with Livneh et al. (2013) TMIN (with yaxis label)
eachdf = 'meanbymonth_TMIN_dailymet_livneh2013'
outfile=os.path.join(homedir, '{0}.png'.format(eachdf))
files.append(outfile)
ax1 = ogh.renderValueInBoxplot(vardf=ltm[eachdf],
                               vmin=vr.min()-1,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='',
                               time_steps='month',
                               wateryear=True,
                               value_name='Average daily minimum temperature (C)',
                               cmap='gnuplot2',
                               figsize=(8,8),
                               reference_lines=reference_lines,
                               ref_legend=False,
                               obs_datavector=[TMIN_obs_C.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'],
                               obs_legend=False)

# Generate top-and-bottom-right figures with Salathe WRF TMAX and TMIN, respectively (no legend or yaxis labels)
eachdf = 'meanbymonth_TMAX_dailywrf_salathe2014'
outfile=os.path.join(homedir, '{0}.png'.format(eachdf))
files.append(outfile)    
ax1 = ogh.renderValueInBoxplot(vardf=ltm[eachdf],
                               vmin=vr.min()-1,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='',
                               time_steps='month',
                               wateryear=True,
                               value_name='',
                               cmap='gnuplot2',
                               figsize=(8,8),
                               reference_lines=reference_lines,
                               ref_legend=False,
                               obs_datavector=[TMAX_obs_C.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'],
                               obs_legend=False)
    
eachdf = 'meanbymonth_TMIN_dailywrf_salathe2014'
outfile=os.path.join(homedir, '{0}.png'.format(eachdf))
files.append(outfile)    
ax1 = ogh.renderValueInBoxplot(vardf=ltm[eachdf],
                               vmin=vr.min()-1,
                               vmax=vr.max()+1,
                               outfilepath=outfile,
                               plottitle='',
                               time_steps='month',
                               wateryear=True,
                               value_name='',
                               cmap='gnuplot2',
                               figsize=(8,8),
                               reference_lines=reference_lines,
                               ref_legend=False,
                               obs_datavector=[TMIN_obs_C.iloc[:,0]],
                               obs_datalabel=['Darrington ranger stn'],
                               obs_legend=False)


CPU times: user 4.75 s, sys: 2.97 s, total: 7.71 s
Wall time: 3.97 s

In [20]:
%%time
"""loop through even months in the water year for Livneh et al. (2013) and Salathe et al. (2014)"""

vr = ogh.valueRange([ltm['meanbymonth_TMAX_dailymet_livneh2013'], ltm['meanbymonth_TMIN_dailymet_livneh2013'],
                     ltm['meanbymonth_TMAX_dailywrf_salathe2014'], ltm['meanbymonth_TMIN_dailywrf_salathe2014']])


for ind, eachmonth in enumerate([11]):
    monthlabel = pd.datetime.strptime(str(eachmonth), '%m')
    outfile=os.path.join(homedir, 'SaukLIVTMAX_{0}.png'.format(monthlabel.strftime('%b')))
    files.append(outfile)
    ogh.renderValuesInPoints(vardf=ltm['meanbymonth_TMAX_dailymet_livneh2013'],
                             vmin=vr.min()-1,
                             vmax=vr.max()+1,
                             vardf_dateindex=eachmonth,
                             shapefile=sauk,
                             outfilepath=outfile,
                             plottitle='',
                             colorbar_label='Air temperature (C)',
                             spatial_resolution=1/16, margin=1, epsg=3857, gridcell_alpha=0.8,
                             basemap_image='Canvas/World_Light_Gray_Base',
                             cmap='gnuplot2',
                             figsize=(1.5,1.5))
    
    monthlabel = pd.datetime.strptime(str(eachmonth), '%m')
    outfile=os.path.join(homedir, 'SaukWRFTMAX_{0}.png'.format(monthlabel.strftime('%b')))
    files.append(outfile)    
    ogh.renderValuesInPoints(vardf=ltm['meanbymonth_TMAX_dailywrf_salathe2014'],
                             vmin=vr.min()-1,
                             vmax=vr.max()+1,
                             vardf_dateindex=eachmonth,
                             shapefile=sauk,
                             outfilepath=outfile,
                             plottitle='',
                             colorbar_label='Air temperature (C)',
                             spatial_resolution=1/16, margin=1, epsg=3857, gridcell_alpha=0.8,
                             basemap_image='Canvas/World_Light_Gray_Base',
                             cmap='gnuplot2',
                             figsize=(1.5,1.5))


CPU times: user 30.3 s, sys: 2.27 s, total: 32.6 s
Wall time: 30.5 s

5. Save the results back into HydroShare

Using the hs_utils library, the results of the Geoprocessing steps above can be saved back into HydroShare. First, define all of the required metadata for resource creation, i.e. title, abstract, keywords, content files. In addition, we must define the type of resource that will be created, in this case genericresource.

Note: Make sure you save the notebook at this point, so that all notebook changes will be saved into the new HydroShare resource.

Total files and image to migrate


In [ ]:
len(files)

In [ ]:
# for each file downloaded onto the server folder, move to a new HydroShare Generic Resource
title = 'Computed spatial-temporal summaries of two gridded data product data sets for Sauk-Suiattle'
abstract = 'This resource contains the computed summaries for the Meteorology data from Livneh et al. 2013 and the WRF data from Salathe et al. 2014.'
keywords = ['Sauk-Suiattle', 'Livneh 2013', 'Salathe 2014','climate','hydromet','watershed', 'visualizations and summaries'] 
rtype = 'genericresource'

# create the new resource
resource_id = hs.createHydroShareResource(abstract, 
                                          title,
                                          keywords=keywords, 
                                          resource_type=rtype, 
                                          content_files=files, 
                                          public=False)