In [1]:
%pylab inline
"Validation class" is a hybrid class which requires both a numerical-model-based object (e.g. "FVCOM" or "Station" object) and at least one measurement-based object (e.g. "ADCP", "TideGauge" or "Drifter" object or a list of their combination). Its aim is to provide validation benchmarks in order to access the accuracy of a FVCOM simulation compared to matching measurements.
As any other library in Python, PySeidon has to be first imported before to be used. Here we will use an alternative import statement compared to the one previoulsy presented:
In [2]:
from pyseidon import *
Star here means all. Usually this form of statements would import the entire library. In the case of PySeidon, this statement will import the following object classes: FVCOM, Station, Validation, ADCP, Tidegauge and Drifter. Only the Validation class will be tackle in this tutorial.
Python is by definition an object oriented language...and so is matlab. PySeidon is based on this notion of object, so let us define our first "Validation" object.
Exercise 1:
Answer:
In [3]:
Validation?
According to the documentation, in order to define a Validation object, the required inputs are:
Note that, flow option permits to impose flow comparison by surface flow ('sf'), depth-averaged flow ('daf') or at any depth (float). If this option is not provided, the option choice will be prompted to the user.
Exercise 2:
Answer:
In [4]:
drift = Drifter('./data4tutorial/drifter_GP_01aug2013.mat')
adcp = ADCP('./data4tutorial/adcp_GP_01aug2013.mat')
tg = TideGauge('./data4tutorial/tidegauge_GP_01aug2013.mat')
station = Station('http://ecoii.acadiau.ca/thredds/dodsC/ecoii/test/Station3D_dngrid_BF_20130730_20130809.nc')
fvcom = FVCOM('http://ecoii.acadiau.ca/thredds/dodsC/ecoii/test/FVCOM3D_dngrid_GP_20130801_20130802.nc')
val1 = Validation(adcp, station)
val2 = Validation([adcp, drift, tg], fvcom, flow='daf')
val3 = Validation(adcp, fvcom, flow=-5.0)
The TideGauge object possesses 2 attributes and 5 methods. They would appear by typing val1. Tab for instance.
An attribute is a quantity intrinsic to its object. A method is an intrinsic function which changes an attribute of its object. Contrarily a function will generate its own output:
The Station attributes are:
The Station methods & functions are:
Exercise 3:
Answer:
In [5]:
val1.validate_data(plot=True, save_csv=True, filename='test1')
val1.taylor_diagram()
val1.validate_harmonics(filename='test2')
Exercise 4:
Answer:
In [6]:
val2.validate_data(save_csv=True, filename='test3')
val2.taylor_diagram()
Exercise 5:
In [7]:
val3.validate_data(save_csv=True, filename='test4')
val3.benchmarks_map()
This collection and analysis of a set of statistics mostly adhere to the benchmarks defined as standards for hydrodynamic model validation by NOAA [1]. Additional statistics have been added to provide additional clarity on the skill of the model [2, 3, 4].
The present validation set is usually performed the following variables yet could be extended to other hydrodynamic quantities:
Benchmarks
Following is a list of the statistics used to evaluate model skil
The value of X is dependent on the variable and the statistic it's being used for. As defined here, X equals 10% of the data range. NOAA additionally defines limits on the values for determining the skill of the model. Accordingly, CF can be no lower than 90%, NOF/POF can be no higher than 1%, and MDPO/MDNO can be no longer than 24hr/1440min. ideally, the Willmott Skill and R2 should be 1, and the phase should be 0.
References
[1] K. W. Hess, T. F. Gross, R. A. Schmalz, J. G. Kelley, F. Aikman and E. Wei, "NOS standards for evaluating operational nowcast and forecst hydrodynamic model systems," National Oceanic and Atmospheric Administration, Silver Srping, Maryland, 2003.
[2] K. Gunn and C. Stock-Williams, "On validating numerical hydrodynamic models of complex tidal flow," International Journal of Marine Energy, Vols. 3-4, no. Special, pp. 82-97, 2013.
[3] N. Georgas and A. F. Blumberg, "Establishing Confidence in Marine Forecast Systems: The Design and Skill Assessment of the New York Harbor Observation and Prediction System, Version 3 (NYHOPS v3)," in 11th International Conference on Estuarine and Coastal Modeling, Seattle, Washington, United States, 2010.
[4] Y. Liu, P. MacCready, H. M. Barbara, E. P. Dever, M. Kosro and N. S. Banas, "Evaluation of a coastal ocean circulation model for the Columbia River plume in summer 2004," Journal of Geophysical Research, vol. 114, no. C2, p. 1978–2012, 2009.
As beta tester, your first assignement is to report bugs...yet not everything is a bug. The first thing to check before to report a bug is to verify that your version of PySeidon is up-to-date. The best way to keep up with the package evolution is to git to clone the repository, use pull to update it and re-install it if needed.
The second thing to check before to report a bug is to verify that the bug is reproducible. When running into a bug, double check that your inputs fit the description of the documentation then turn the debug flag on (e.g. output = tidegaugeobject.function(inputs, debug=True)) and submit the command again. If the error re-occurs then report it (i.e. copy entire error message + command and send it to package administrator)
Your second role as beta-tester is to submit suggestions and critics to the developpers regarding the functioning and functionality of the package. Beta testing phase is the best opportunity to steer a project towards the applications you would like to be tackled...