TROCAS, Reading and plotting underway data on interactive maps

Plotting TROCAS 6 underway data from 1-min-binned merged file created by Catherine Kuhn (TROCAS6_GPS_SONDE_CO2.csv).

Demos with Holoviz/GeoViews and Folium packages. Run with customized holoviz conda environment.

7/25/2019. Emilio


In [1]:
import os

import pandas as pd
import geopandas as gpd
import shapely.geometry as shpgeom
from cartopy import crs

import folium
import branca

import geoviews as gv
import geoviews.feature as gf

gv.extension('bokeh')



In [2]:
%matplotlib inline

In [3]:
pd.__version__, gpd.__version__, folium.__version__, gv.__version__


Out[3]:
('0.24.2', '0.5.1', '0.10.0', '1.6.2')

Read the merged data into a GeoDataframe


In [4]:
data_dpth = "/home/mayorga/Desktop/Richey-Amazon-Proposals/data/GD_TROCASData/fromKuhn/Data/T6"

In [5]:
t6_merged_fname = "TROCAS6_GPS_SONDE_CO2.csv"

In [6]:
t6_df = pd.read_csv(os.path.join(data_dpth, t6_merged_fname), 
                    parse_dates=['times'], infer_datetime_format=True)

In [7]:
t6_df.shape


Out[7]:
(8908, 36)

In [8]:
t6_gdf = gpd.GeoDataFrame(t6_df, 
                          geometry=t6_df[['longitude', 'latitude']].apply(shpgeom.Point, axis=1),
                          crs={'init': 'epsg:4326'})

In [9]:
t6_gdf.head()


Out[9]:
times latitude longitude CO2 ppm Cell Temp (c) Cell Press (kpa) CO2 atm CO2 mol/L CO2 umol/L BGA-PC RFU ... TSS mg/L Temp C Time (Fract. Sec) Turbidity FNU fDOM QSU fDOM RFU nLF Cond uS/cm pH pH mV geometry
0 2017-04-26 13:57:00 -0.052180 -51.140235 NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN POINT (-51.1402355 -0.052180449)
1 2017-04-26 13:58:00 -0.052297 -51.140371 4318.325 51.44 100.05 0.004318 0.000142 141.809172 NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN POINT (-51.14037108 -0.05229746)
2 2017-04-26 13:59:00 -0.052635 -51.140438 4363.840 51.44 100.05 0.004364 0.000143 143.303836 NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN POINT (-51.14043776 -0.05263458)
3 2017-04-26 14:00:00 -0.053500 -51.140462 4369.770 51.44 100.05 0.004370 0.000143 143.498571 -1.245 ... 0.0 29.643 0.0 84.065 58.790 7.96 59.45 6.73 4.85 POINT (-51.14046236 -0.053499593)
4 2017-04-26 14:01:00 -0.054486 -51.140192 4377.715 51.44 100.04 0.004378 0.000144 143.759476 -1.225 ... 0.0 29.605 0.0 82.170 58.625 7.90 59.70 6.71 6.05 POINT (-51.14019167 -0.054485723)

5 rows × 37 columns


In [10]:
t6_gdf.dtypes


Out[10]:
times                datetime64[ns]
latitude                    float64
longitude                   float64
CO2 ppm                     float64
Cell Temp (c)               float64
Cell Press (kpa)            float64
CO2 atm                     float64
CO2 mol/L                   float64
CO2 umol/L                  float64
BGA-PC RFU                  float64
BGA-PC ug/L                 float64
Battery V                   float64
Cable Pwr V                 float64
Chlorophyll RFU             float64
Chlorophyll ug/L            float64
Cond uS/cm                  float64
Depth m                     float64
Fault Code                  float64
ODO % sat                   float64
ODO mg/L                    float64
ODO mol/L                   float64
ODO ug/L                    float64
ODO umol/L                  float64
Press psi a                 float64
Sal psu                     float64
Site Name                   float64
SpCond uS/cm                float64
TSS mg/L                    float64
Temp C                      float64
Time (Fract. Sec)           float64
Turbidity FNU               float64
fDOM QSU                    float64
fDOM RFU                    float64
nLF Cond uS/cm              float64
pH                          float64
pH mV                       float64
geometry                     object
dtype: object

Remove some redundant or unused columns, to generate a slimmer dataframe


In [11]:
t6_gdf.drop(columns=['CO2 mol/L', 'ODO mol/L', 'ODO ug/L', 'Site Name'], inplace=True)

Quick GeoPandas CO2 plot


In [12]:
t6_gdf['CO2 ppm'].describe()


Out[12]:
count    3865.000000
mean     3851.032639
std       344.356039
min      2897.635000
25%      3622.680000
50%      3900.170000
75%      4098.480000
max      6249.770000
Name: CO2 ppm, dtype: float64

In [13]:
t6_gdf.plot(column='CO2 ppm', 
            vmin=t6_gdf['CO2 ppm'].min(), vmax=t6_gdf['CO2 ppm'].max(),
            legend=True, figsize=(12, 6));



In [14]:
# throw out rows with invalid point geometries
validgeo_gdf = t6_gdf[t6_gdf.is_valid]

Interactive maps of underway CO2 and chlorophyll data

Hover over data points to see the measured (binned) value and timestamp.

1. Using Holoviz


In [15]:
parameter = 'CO2 ppm'
filtered_gdf = validgeo_gdf[~validgeo_gdf[parameter].isna()]

In [16]:
uw_points = gv.Points(filtered_gdf, vdims=['CO2 ppm', 'times']) # , label='CO2 (ppm)'

In [17]:
esri_wtm_tiles = ('http://services.arcgisonline.com/arcgis/rest/services/'
                  'World_Topo_Map/MapServer/MapServer/tile/{Z}/{Y}/{X}')

In [18]:
geo_bg = gv.tile_sources.WMTS(esri_wtm_tiles).options(alpha=1)

In [19]:
title = "Underway Surface CO2, {} to {}".format(
    filtered_gdf.times.min().date(), 
    filtered_gdf.times.max().date())

customhover = 'hover'
geo_bg * uw_points.opts(title=title,
    width=900, height=600, 
    color='CO2 ppm', colorbar=True, cmap='viridis', size=4,
    toolbar='above', tools=[customhover]
)


Out[19]:

2. Using Folium


In [20]:
def underway_folium_map(source_gdf, parameter, units, tooltip_prec=1, cmap_name='YlOrRd_03'):
    # m = folium.Map(tiles="cartodbpositron")
    esri_wtm_tiles = ('http://services.arcgisonline.com/arcgis/rest/services/'
                      'World_Topo_Map/MapServer/MapServer/tile/{z}/{y}/{x}')

    filtered_gdf = source_gdf[~source_gdf[parameter].isna()]

    m = folium.Map(tiles=esri_wtm_tiles, attr='ESRI')
    
    cmap = branca.colormap.linear._colormaps[cmap_name].scale(
        filtered_gdf[parameter].min(), 
        filtered_gdf[parameter].max())
    cmap.caption = parameter
    cmap.add_to(m)

    # popup, tooltip
    # fillOpacity, fillColor, weight
    for idx, feature in filtered_gdf.iterrows():
        folium.CircleMarker(location=[feature.geometry.y, feature.geometry.x], 
                            radius=3,
                            color='black',
                            weight=0.1,
                            fill=True,
                            fill_color=cmap(feature[parameter]),
                            fill_opacity=1,
                            tooltip="{:.{prec}f} {} on {}".format(
                                feature[parameter], units, feature['times'], prec=tooltip_prec)
                     ).add_to(m)

    m.fit_bounds(m.get_bounds())

    return m

$CO_2$


In [21]:
m1 = underway_folium_map(validgeo_gdf, parameter='CO2 ppm', units='ppm', 
                         tooltip_prec=0, cmap_name='viridis')
m1


Out[21]:

Chlorophyll


In [22]:
m2 = underway_folium_map(validgeo_gdf, parameter='Chlorophyll ug/L', units='ug/L', 
                         tooltip_prec=1, cmap_name='YlOrRd_03')
m2


Out[22]:

In [ ]: