In [1]:
%matplotlib inline
import inspect, sys
In [2]:
# check pydov path
import pydov
In [3]:
from pydov.search.grondwatermonster import GrondwaterMonsterSearch
gwmonster = GrondwaterMonsterSearch()
A description is provided for the 'GrondwaterMonster' datatype:
In [4]:
print(gwmonster.get_description())
The different fields that are available for objects of the 'GrondwaterMonster' datatype can be requested with the get_fields() method:
In [5]:
fields = gwmonster.get_fields()
# print available fields
for f in fields.values():
print(f['name'])
You can get more information of a field by requesting it from the fields dictionary:
In [6]:
# print information for a certain field
fields['waarde']
Out[6]:
Optionally, if the values of the field have a specific domain the possible values are listed as values:
In [9]:
# if an attribute can have several values, these are listed under 'values', e.g. for 'parameter':
list(fields['parameter']['values'].items())[0:10]
Out[9]:
In [10]:
fields['parameter']['values']['NH4']
Out[10]:
Get data for all the groundwater samples that are geographically located within the bounds of the specified box.
The coordinates are in the Belgian Lambert72 (EPSG:31370) coordinate system and are given in the order of lower left x, lower left y, upper right x, upper right y.
In [11]:
from pydov.util.location import Within, Box
df = gwmonster.search(location=Within(Box(93378, 168009, 94246, 169873)))
df.head()
Out[11]:
Using the pkey attributes one can request the details of the corresponding grondwatermonster in a webbrowser (only showing the first unique records):
In [12]:
for pkey_grondwatermonster in df.pkey_grondwatermonster.unique()[0:5]:
print(pkey_grondwatermonster)
Next to querying groundwater samples based on their geographic location within a bounding box, we can also search for groundwater samples matching a specific set of properties. For this we can build a query using a combination of the 'GrondwaterMonster' fields and operators provided by the WFS protocol.
A list of possible operators can be found below:
In [13]:
[i for i,j in inspect.getmembers(sys.modules['owslib.fes'], inspect.isclass) if 'Property' in i]
Out[13]:
In this example we build a query using the PropertyIsEqualTo operator to find all groundwater samples that are within the community (gemeente) of 'Leuven':
In [14]:
from owslib.fes import PropertyIsEqualTo
query = PropertyIsEqualTo(
propertyname='gemeente',
literal='Leuven')
df = gwmonster.search(query=query)
df.head()
Out[14]:
Once again we can use the pkey_grondwatermonster as a permanent link to the information of the groundwater samples:
In [15]:
for pkey_grondwatermonster in df.pkey_grondwatermonster.unique()[0:5]:
print(pkey_grondwatermonster)
We can add the descriptions of the parameter values as an extra column 'parameter_label':
In [16]:
df['parameter_label'] = df['parameter'].map(fields['parameter']['values'])
df[['pkey_grondwatermonster', 'datum_monstername', 'parameter', 'parameter_label', 'waarde', 'eenheid']].head()
Out[16]:
Get all groundwater screens in Hamme that have measurements for cations (kationen). And filter to get only Sodium values after fetching all records.
In [17]:
from owslib.fes import Or, Not, PropertyIsNull, PropertyIsLessThanOrEqualTo, And, PropertyIsLike
query = And([PropertyIsEqualTo(propertyname='gemeente',
literal='Hamme'),
PropertyIsEqualTo(propertyname='kationen',
literal='true')
])
df_hamme = gwmonster.search(query=query,
return_fields=('pkey_grondwatermonster', 'parameter', 'parametergroep', 'waarde', 'eenheid','datum_monstername'))
df_hamme.head()
Out[17]:
You should note that this initial dataframe contains all parameters (not just the cations). The filter will only make sure that only samples where any cation was analysed are in the list. If we want to filter more, we should do so in the resulting dataframe.
In [18]:
df_hamme = df_hamme[df_hamme.parameter=='Na']
df_hamme.head()
Out[18]:
For further analysis and visualisation of the time series data, we can use the data analysis library pandas and visualisation library matplotlib.
In [19]:
import pandas as pd
import matplotlib.pyplot as plt
Query the data of a specific filter using its pkey:
In [20]:
query = PropertyIsEqualTo(
propertyname='pkey_filter',
literal='https://www.dov.vlaanderen.be/data/filter/1991-001040')
df = gwmonster.search(query=query)
df.head()
Out[20]:
The date is still stored as a string type. Transforming to a data type using the available pandas function to_datetime and using these dates as row index:
In [21]:
df['datum_monstername'] = pd.to_datetime(df['datum_monstername'])
For many usecases, it is useful to create a pivoted table, showing the value per parameter
In [22]:
pivot = df.pivot_table(columns=df.parameter, values='waarde', index='datum_monstername')
pivot
Out[22]:
The default plotting functionality of Pandas can be used:
In [35]:
parameters = ['NO3', 'NO2', 'NH4']
ax = pivot[parameters].plot.line(style='.-', figsize=(12, 5))
ax.set_xlabel('');
ax.set_ylabel('concentration (mg/l)');
ax.set_title('Concentration nitrite, nitrate and ammonium for filter id 1991-001040');
In [73]:
from pydov.search.grondwaterfilter import GrondwaterFilterSearch
from pydov.util.query import Join
gfs = GrondwaterFilterSearch()
gemeente = 'Kalmthout'
filter_query = And([PropertyIsLike(propertyname='meetnet',
literal='meetnet 1 %'),
PropertyIsEqualTo(propertyname='gemeente',
literal=gemeente)])
filters = gfs.search(query=filter_query, return_fields=['pkey_filter'])
monsters = gwmonster.search(query=Join(filters, 'pkey_filter'))
monsters.head()
Out[73]:
We will filter out some parameters, and show trends per location.
In [131]:
parameter = 'NH4'
trends_sel = monsters[(monsters.parameter==parameter) & (monsters.veld_labo=='LABO')]
trends_sel = trends_sel.set_index('datum_monstername')
trends_sel['label'] = trends_sel['gw_id'] + ' F' + trends_sel['filternummer']
# By pivoting, we get each location in a different column
trends_sel_pivot = trends_sel.pivot_table(columns='label', values='waarde', index='datum_monstername')
trends_sel_pivot.index = pd.to_datetime(trends_sel_pivot.index)
# resample to yearly values and plot data
ax = trends_sel_pivot.resample('A').median().plot.line(style='.-', figsize=(12, 5))
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))
ax.set_title(f'Long term evolution of {parameter} in {gemeente}');
ax.set_xlabel('year');
ax.set_ylabel('concentration (mg/l)');