2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData-checkpoint


Go down for licence and other metadata about this presentation

\newpage

Preamble

Licence

Unless stated otherwise all content is released under a [CC0]+BY licence. I'd appreciate it if you reference this but it is not necessary.

\newpage

Using Ipython for presentations

A short video showing how to use Ipython for presentations


In [4]:
from IPython.display import YouTubeVideo
YouTubeVideo('F4rFuIb1Ie4')


Out[4]:

In [2]:
## PDF output using pandoc

import os


### Export this notebook as markdown
commandLineSyntax = 'ipython nbconvert --to markdown 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb'
print (commandLineSyntax)

os.system(commandLineSyntax)

### Export this notebook and the document header as PDF using Pandoc

commandLineSyntax = 'pandoc  -f markdown -t latex -N -V geometry:margin=1in DocumentHeader.md 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.md --filter pandoc-citeproc  --latex-engine=xelatex --toc -o interim.pdf '

os.system(commandLineSyntax)

### Remove cruft from the pdf

commandLineSyntax = 'pdftk interim.pdf cat 1-3 16-end output 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.pdf'

os.system(commandLineSyntax)

### Remove the interim pdf

commandLineSyntax = 'rm interim.pdf'

os.system(commandLineSyntax)


ipython nbconvert --to markdown 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb
Out[2]:
0

The environment

In order to replicate my environment you need to know what I have installed!

Set up watermark

This describes the versions of software used during the creation.

Please note that critical libraries can also be watermarked as follows:

%watermark -v -m -p numpy,scipy

In [2]:
%install_ext https://raw.githubusercontent.com/rasbt/python_reference/master/ipython_magic/watermark.py
%load_ext watermark


Installed watermark.py. To use it, type:
  %load_ext watermark

In [3]:
%watermark -a "Anthony Beck"  -d -v -m -g


Anthony Beck 29/06/2015 

CPython 2.7.10
IPython 3.2.0

compiler   : GCC 4.4.7 20120313 (Red Hat 4.4.7-1)
system     : Linux
release    : 3.13.0-37-generic
machine    : x86_64
processor  : x86_64
CPU cores  : 4
interpreter: 64bit
Git hash   : 

In [9]:
#List of installed conda packages
!conda list


# packages in environment at /home/arb/LocalPython/Anaconda27/anaconda:
#
_license                  1.1                      py27_0  
abstract-rendering        0.5.1                np19py27_0  
accelerate                1.11.0              np19py27_p0  
affine                    1.1.0                    py27_0  
alabaster                 0.7.3                    py27_0  
anaconda                  2.2.0                np19py27_0  
argcomplete               0.8.4                    py27_0  
arrow                     0.5.4                     <pip>
astroid                   1.3.6                     <pip>
astropy                   1.0.2                np19py27_0  
babel                     1.3                      py27_0  
backports.ssl-match-hostname 3.4.0.2                   <pip>
basemap                   1.0.7                np19py27_0  
bcolz                     0.8.1                np19py27_0  
beautiful-soup            4.3.2                    py27_0  
beautifulsoup4            4.3.2                     <pip>
binstar                   0.10.3                   py27_0  
bitarray                  0.8.1                    py27_0  
blaze                     0.8.0                     <pip>
blaze-core                0.8.0                np19py27_0  
blz                       0.6.2                np19py27_0  
bokeh                     0.8.2                np19py27_0  
boto                      2.38.0                   py27_0  
bottle                    0.12.7                   py27_0  
brewer2mpl                1.4                      py27_0  
cairo                     1.12.18                       4  
cartopy                   0.10.0               np18py27_0  
cdecimal                  2.3                      py27_0  
certifi                   14.05.14                 py27_0  
cffi                      0.9.2                    py27_0  
click                     4.0                      py27_0  
cligj                     0.1.0                    py27_0  
cloog                     0.18.0                        0  
clyent                    0.3.4                    py27_0  
colorama                  0.3.3                    py27_0  
conda                     3.14.0                   py27_0  
conda-build               1.12.1                   py27_0  
conda-env                 2.2.3                    py27_0  
configobj                 5.0.6                    py27_0  
cryptography              0.8.2                    py27_0  
cudatoolkit               6.0                          p0  
curl                      7.38.0                        0  
cython                    0.22                     py27_0  
cytoolz                   0.7.2                    py27_0  
datashape                 0.4.5                np19py27_0  
dateutil                  2.4.1                    py27_0  
decorator                 3.4.0                    py27_0  
descartes                 1.0.1                    py27_0  
docutils                  0.12                     py27_0  
dynd-python               0.6.5                np19py27_0  
enum                      0.4.4                     <pip>
enum34                    1.0.4                    py27_0  
fastcache                 1.0.2                    py27_0  
fiona                     1.5.1                np19py27_0  
flask                     0.10.1                   py27_1  
fontconfig                2.11.1                        4  
freetype                  2.5.2                         2  
funcsigs                  0.4                      py27_0  
futures                   2.2.0                    py27_0  
gdal                      1.11.2               np19py27_2  
geopandas                 0.1.1                    py27_0  
geopy                     1.10.0                    <pip>
geos                      3.3.3                         0  
gevent                    1.0.1                    py27_0  
gevent-websocket          0.9.3                    py27_0  
glib                      2.43.0                        2  
gmp                       5.1.2                         2  
greenlet                  0.4.6                    py27_0  
grin                      1.2.1                    py27_1  
h5py                      2.5.0                np19py27_0  
harfbuzz                  0.9.35                        6  
hdf5                      1.8.14                        0  
html5lib                  0.99999                   <pip>
ipython                   3.2.0                    py27_0  
ipython-notebook          3.2.0                    py27_0  
ipython-qtconsole         3.1.0                    py27_0  
isl                       0.12.2                        0  
itsdangerous              0.24                     py27_0  
jdcal                     1.0                      py27_0  
jedi                      0.8.1                    py27_0  
jinja2                    2.7.3                    py27_1  
jpeg                      8d                            0  
jsonschema                2.4.0                    py27_0  
libdynd                   0.6.5                         0  
libffi                    3.0.13                        0  
libgcc                    4.8.4                         1  
libgdal                   1.11.2                        0  
libnetcdf                 4.3.2                         1  
libpng                    1.6.17                        0  
libsodium                 0.4.5                         0  
libtiff                   4.0.2                         1  
libxml2                   2.9.0                         0  
libxslt                   1.1.28                        0  
llvmlite                  0.4.0                    py27_0  
logilab-common            0.63.2                    <pip>
lxml                      3.4.4                    py27_0  
mapnik                    0.1                       <pip>
markupsafe                0.23                     py27_0  
matplotlib                1.4.3                np19py27_2  
mistune                   0.6                      py27_0  
mkl                       11.1                np19py27_p3  
mkl-rt                    11.1                         p0  
mkl-service               1.0.0                   py27_p1  
mklfft                    2.0                 np19py27_p0  
mock                      1.0.1                    py27_0  
mpc                       1.0.1                         0  
mpfr                      3.1.2                         0  
multipledispatch          0.4.7                    py27_0  
ncurses                   5.9                           4  
networkx                  1.9.1                    py27_0  
nltk                      3.0.2                np19py27_0  
nose                      1.3.6                    py27_0  
notedown                  1.4.4                     <pip>
numba                     0.18.2               np19py27_1  
numbapro                  0.18.0              np19py27_p2  
numbapro_cudalib          0.2                           0  
numexpr                   2.3.1               np19py27_p0  [mkl]
numpy                     1.9.2                   py27_p0  [mkl]
odo                       0.3.2                np19py27_0  
openpyxl                  2.0.2                    py27_0  
openssl                   1.0.1k                        1  
pandana                   0.1.2                    py27_0  
pandas                    0.16.2               np19py27_0  
pandasql                  0.6.2                np19py27_0  
pandoc-attributes         0.1.7                     <pip>
pandocfilters             1.2.4                     <pip>
pango                     1.36.8                        3  
patchelf                  0.6                           0  
patsy                     0.3.0                np19py27_0  
pcre                      8.31                          0  
pep8                      1.6.2                    py27_0  
pillow                    2.7.0                    py27_1  
pip                       7.0.3                    py27_0  
pixman                    0.26.2                        0  
plotly                    1.6.17                    <pip>
ply                       3.6                      py27_0  
prettyplotlib             0.1.7                     <pip>
prettytable               0.7.2                    py27_0  
proj4                     4.8.0                         0  
psutil                    2.2.1                    py27_0  
ptyprocess                0.4                      py27_0  
py                        1.4.26                   py27_0  
py2cairo                  1.10.0                   py27_2  
pyasn1                    0.1.7                    py27_0  
pycosat                   0.6.1                    py27_0  
pycparser                 2.12                     py27_0  
pycrypto                  2.6.1                    py27_0  
pycurl                    7.19.5.1                 py27_0  
pyflakes                  0.8.1                    py27_0  
pygments                  2.0.2                    py27_0  
pylint                    1.4.3                     <pip>
pymc                      2.3.4               np19py27_p0  [mkl]
pyopenssl                 0.15.1                   py27_0  
pyparsing                 2.0.3                    py27_0  
pyproj                    1.9.3                    py27_0  
pyqt                      4.11.3                   py27_1  
pysal                     1.6.0                np19py27_1  
pyshp                     1.2.1                     <pip>
pytables                  3.1.1                np19py27_2  
pytest                    2.7.0                    py27_0  
python                    2.7.10                        0  
python-dateutil           2.4.2                    py27_0  
pytz                      2015.4                   py27_0  
pyyaml                    3.11                     py27_1  
pyzmq                     14.7.0                   py27_0  
qt                        4.8.6                         3  
r                         3.1.3                         0  
r-base                    3.1.3                         2  
r-boot                    1.3_16                 r3.1.3_0  
r-class                   7.3_12                 r3.1.3_0  
r-cluster                 1.15.3                        0  
r-codetools               0.2_11                 r3.1.3_0  
r-foreign                 0.8_63                 r3.1.3_0  
r-kernsmooth              2.23_14                r3.1.3_0  
r-lattice                 0.20_31                r3.1.3_0  
r-mass                    7.3.37                        0  
r-matrix                  1.2_1                  r3.1.3_0  
r-mgcv                    1.8_6                  r3.1.3_0  
r-nlme                    3.1.118                       0  
r-nlme                    3.1_120                r3.1.3_0  
r-nnet                    7.3_9                  r3.1.3_0  
r-recommended             3.1.3                         0  
r-rpart                   4.1_9                  r3.1.3_0  
r-spatial                 7.3_9                  r3.1.3_0  
r-survival                2.38_2                 r3.1.3_0  
rasterio                  0.15.1                   py27_0  
readline                  6.2                           2  
redis                     2.6.9                         0  
redis-py                  2.10.3                   py27_0  
requests                  2.7.0                    py27_0  
rope                      0.9.4                    py27_1  
rpy2                      2.5.6                    py27_0  
runipy                    0.1.3                    py27_0  
scikit-image              0.11.3               np19py27_0  
scikit-learn              0.16.1              np19py27_p0  [mkl]
scipy                     0.15.1              np19py27_p0  [mkl]
seaborn                   0.5.1                     <pip>
setuptools                17.1.1                   py27_0  
shapely                   1.5.9                     <pip>
simplejson                3.6.3                    py27_0  
singledispatch            3.4.0.3                  py27_1  
sip                       4.16.5                   py27_0  
six                       1.9.0                    py27_0  
snowballstemmer           1.2.0                    py27_0  
snuggs                    1.3.1                     <pip>
sockjs-tornado            1.0.1                    py27_0  
sphinx                    1.3.1                    py27_0  
sphinx-rtd-theme          0.1.7                     <pip>
sphinx_rtd_theme          0.1.7                    py27_0  
spyder                    2.3.4                    py27_1  
spyder-app                2.3.4                    py27_0  
sqlalchemy                1.0.3                    py27_0  
sqlite                    3.8.4.1                       1  
sqlparse                  0.1.14                   py27_0  
ssl_match_hostname        3.4.0.2                  py27_0  
statsmodels               0.6.1                np19py27_0  
sympy                     0.7.6                    py27_0  
system                    5.8                           2  
tables                    3.1.1                     <pip>
terminado                 0.5                      py27_0  
theano                    0.7.0                np19py27_0  
tk                        8.5.18                        0  
toolz                     0.7.2                    py27_0  
tornado                   4.2                      py27_0  
tweepy                    2.3                      py27_0  
twitter                   1.17.0                    <pip>
ujson                     1.33                     py27_0  
unicodecsv                0.9.4                    py27_0  
urbansim                  2.0.1                     <pip>
util-linux                2.21                          0  
werkzeug                  0.10.4                   py27_0  
xlrd                      0.9.3                    py27_0  
xlsxwriter                0.7.2                    py27_0  
xlwt                      1.0.0                    py27_0  
yaml                      0.1.6                         0  
zeromq                    4.0.5                         0  
zlib                      1.2.8                         0  

In [10]:
#List of installed pip packages
!pip list


abstract-rendering (0.5.1)
affine (1.1.0)
alabaster (0.7.3)
argcomplete (0.8.4)
arrow (0.5.4)
astroid (1.3.6)
astropy (1.0.2)
Babel (1.3)
backports.ssl-match-hostname (3.4.0.2)
basemap (1.0.7)
bcolz (0.8.1)
beautifulsoup4 (4.3.2)
binstar (0.10.3)
bitarray (0.8.1)
blaze (0.8.0)
blz (0.6.2)
bokeh (0.8.2)
boto (2.38.0)
bottle (0.12.7)
brewer2mpl (1.4.1)
Cartopy (0.10.0)
cdecimal (2.3)
certifi (14.5.14)
cffi (0.9.2)
click (4.0)
cligj (0.1.0)
clyent (0.3.4)
colorama (0.3.3)
conda (3.14.0)
conda-build (1.12.1)
conda-env (2.2.3)
configobj (5.0.6)
cryptography (0.8.2)
Cython (0.22)
cytoolz (0.7.2)
datashape (0.4.5)
decorator (3.4.0)
descartes (1.0.1)
docutils (0.12)
enum (0.4.4)
enum34 (1.0.4)
fastcache (1.0.2)
Fiona (1.5.1)
Flask (0.10.1)
funcsigs (0.4)
futures (2.2.0)
GDAL (1.11.2)
geopandas (0.1.1)
geopy (1.10.0)
gevent (1.0.1)
gevent-websocket (0.9.3)
greenlet (0.4.6)
grin (1.2.1)
h5py (2.5.0)
html5lib (0.99999)
ipython (3.2.0)
itsdangerous (0.24)
jdcal (1.0)
jedi (0.8.1)
Jinja2 (2.7.3)
jsonschema (2.4.0)
llvmlite (0.4.0)
logilab-common (0.63.2)
lxml (3.4.4)
mapnik (0.1)
MarkupSafe (0.23)
matplotlib (1.4.3)
mistune (0.6)
mklfft (0.0.0)
mock (1.0.1)
multipledispatch (0.4.7)
networkx (1.9.1)
nltk (3.0.2)
nose (1.3.6)
notedown (1.4.4)
numba (0.18.2)
numbapro (0.18.0)
numexpr (2.3.1)
numpy (1.9.2)
odo (0.3.2)
openpyxl (2.0.2)
pandana (0.1.2)
pandas (0.16.2)
pandasql (0.6.2)
pandoc-attributes (0.1.7)
pandocfilters (1.2.4)
patsy (0.3.0)
pep8 (1.6.2)
Pillow (2.7.0)
pip (7.0.3)
plotly (1.6.17)
ply (3.6)
prettyplotlib (0.1.7)
prettytable (0.7.2)
psutil (2.2.1)
ptyprocess (0.4)
py (1.4.26)
pyasn1 (0.1.7)
pycosat (0.6.1)
pycparser (2.12)
pycrypto (2.6.1)
pycurl (7.19.5.1)
pyflakes (0.8.1)
Pygments (2.0.2)
pylint (1.4.3)
pymc (2.3.4)
pyOpenSSL (0.15.1)
pyparsing (2.0.3)
pyproj (1.9.4)
PySAL (1.6.0)
pyshp (1.2.1)
pytest (2.7.0)
python-dateutil (2.4.2)
pytz (2015.4)
PyYAML (3.11)
pyzmq (14.7.0)
rasterio (0.23.0)
redis (2.10.3)
requests (2.7.0)
rope (0.9.4)
rpy2 (2.5.6)
runipy (0.1.3)
scikit-image (0.11.3)
scikit-learn (0.16.1)
scipy (0.15.1)
seaborn (0.5.1)
setuptools (17.1.1)
Shapely (1.5.9)
simplejson (3.6.3)
singledispatch (3.4.0.3)
six (1.9.0)
snowballstemmer (1.2.0)
snuggs (1.3.1)
sockjs-tornado (1.0.1)
Sphinx (1.2.3)
sphinx-rtd-theme (0.1.7)
spyder (2.3.4)
SQLAlchemy (1.0.3)
sqlparse (0.1.14)
statsmodels (0.6.1)
sympy (0.7.6)
tables (3.1.1)
terminado (0.5)
Theano (0.7.0)
toolz (0.7.2)
tornado (4.2)
tweepy (2.3.0)
twitter (1.17.0)
ujson (1.33)
unicodecsv (0.9.4)
urbansim (2.0.1)
Werkzeug (0.10.4)
xlrd (0.9.3)
XlsxWriter (0.7.2)
xlwt (1.0.0)

Running dynamic presentations

You need to install the RISE Ipython Library from Damián Avila for dynamic presentations

To convert and run this as a static presentation run the following command:


In [ ]:
# I can't see notes in this - Not sure what the issue is. Try this in a solely python 2 environment

#PRIGINAL !ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides --post serve



!ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides  --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2 --post serve


/home/arb/LocalPython/Anaconda27/anaconda/lib/python2.7/site-packages/IPython/nbconvert.py:13: ShimWarning: The `IPython.nbconvert` package has been deprecated. You should import from ipython_nbconvert instead.
  "You should import from ipython_nbconvert instead.", ShimWarning)
[NbConvertApp] Converting notebook 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb to slides
[NbConvertApp] Writing 293147 bytes to 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.slides.html
[NbConvertApp] Redirecting reveal.js requests to https://cdn.jsdelivr.net/reveal.js/2.6.2
Serving your slides at http://127.0.0.1:8000/2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.slides.html
Use Control-C to stop this server
Created new window in existing browser session.
WARNING:tornado.access:404 GET /custom.css (127.0.0.1) 2.89ms
WARNING:tornado.access:404 GET /favicon.ico (127.0.0.1) 0.92ms

In [1]:
#ipython nbconvert --to slides Analysis-scheme1.ipynb --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2

!ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides  --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2


[NbConvertApp] Converting notebook 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb to slides
[NbConvertApp] Writing 279707 bytes to 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.slides.html

To close this instances press control 'c' in the ipython notebook terminal console

Static presentations allow the presenter to see speakers notes (use the 's' key)

If running dynamically run the scripts below

Pre load some useful libraries


In [13]:
#Future proof python 2
from __future__ import print_function #For python3 print syntax
from __future__ import division

# def
import IPython.core.display

# A function to collect user input - ipynb_input(varname='username', prompt='What is your username')

def ipynb_input(varname, prompt=''):
        """Prompt user for input and assign string val to given variable name."""
        js_code = ("""
            var value = prompt("{prompt}","");
            var py_code = "{varname} = '" + value + "'";
            IPython.notebook.kernel.execute(py_code);
        """).format(prompt=prompt, varname=varname)
        return IPython.core.display.Javascript(js_code)
    
# inline

%pylab inline


Populating the interactive namespace from numpy and matplotlib

\newpage

About me

  • Honorary Research Fellow, University of Nottingham: orcid
  • Director, Geolytics Limited - A spatial data analytics consultancy

About this presentation

\newpage

Contribution to GIScience learning outcomes

This presentation contributes to the following learning outcomes for this course.

  1. Knowledge and Understanding:
    • Appreciate the importance of standards for Geographic Information and the role of the Open Geospatial Consortium.
    • Understand the term 'interoperability'.
    • Appreciate the different models for database design.
    • Understand the basis of Linked Data.
    • Find UK government open data and understand some of the complexities in the use of this data.
    • Appreciate the data issues involved in managing large distributed databases, Location-Based Services and the emergence of real-time data gathering through the 'Sensor-Web'.
    • Understand the different models for creating international Spatial Data Infrastructures.
  2. Intellectual Skills:
    • Evaluate the role of standards and professional bodies in GIS.
    • Articulate the meaning and importance of interoperability, semantics and ontologies.
    • Assess the technical and organisational issues which come into play when attempting to design large distributed geographic databases aimed at supporting 'real-world' problems.

Suggested Questions:

  1. What do you think are the challenges for National Mapping Agencies and major Volunteer Geographic Data initiatives (OSM) are over the next 5 to 10 years?
    • This questions aims to get the student to think about how technology, policy and practice frameworks will impact upon the GI landscape. Will, for example, OSM make the Ordnance Survey irrelevant? What would then happen to organisations like the Land Registry that require credible data products to support their activites. How will things like the semantic web impact on GI. Can the aspirations of Open Data be realised and bring a wealth of nwe products and services. What impact will the Internet of Things have on GI.
  2. Describe the issues surrounding interoperability. How have the OGC, OSGeo and INSPIRE helped and what challenges remain.
    • This should be a simple description of data silos, technical interoperability (syntactic, semantic, schematic), standards, standards bodies and inspire. Full dscussions will consider non-technical interoperability.
  3. Is cartography relevant to modern GI? Discuss this in relation to map representations, the range of available open data and the needs of different users.
    • This is a somewhat open ended question with the purpose of getting the student to consider the differences between mapping as a broadly static data representation and GI as a dynamic data rich discipline. Obviously cartography or visualization is important - this is where issues like clutter, colour, visualization, symbology, generalisation can all be discussed. The open data point is to get the students to consider the range of available geo and attribute data and to think of GI as a spatial data stack. This should lead into the student being able to discuss generic products (providing similar service to the masses (i.e. maps)) and bespoke services (providing specific services to niche communities) which both use the same underlying data sources. Good answers will bring in some of the non-technical interoperability issues - particularly licencing.

SORT OUT DROPBOX and then let Ana know my details.


In [ ]:
# THINGS TO DISCUSS

## Data needs to be credible, accessible, properly licenceed. Bring in Internet of Things

A potted history of mapping

In the beginning was the geoword

and the word was cartography

\newpage

  • Cartography was king.
  • Static representations of spatial knowledge with the cartographer deciding what to represent.
    • Hence, maps are domain specific knowledge repositories for spatial data

\newpage

And then there was data .........

\newpage

Restrictive data

\newpage

Disconnected data with different:

  • Standards
  • Quality
  • Databases
  • Semantics

\newpage

Why is this an issue: INSPIRE

describe inspire

\newpage

Making data interoperable and open

\newpage

Interoperability

Put in interoperability quote here (plus references)

Defense interoperability

As defined by DoD policy, interoperability is the ability of systems, units, or forces to provide data, information, material, and services to, and accept the same from, other systems, units, or forces; and to use the data, information, materiel, and services so exchanged to enable them to operate effectively together. IT and NSS interoperability includes both the technical exchange of information and the end-to-end operational effectiveness of that exchanged information as required for mission accomplishment. Interoperability is more than just information exchange; it includes systems, processes, procedures, organizations, and missions over the life cycle and must be balanced with information assurance.

\newpage

Technical interoperability - levelling the field

\newpage

Discuss each interoperability and its resolution.

Bring in:

  • OGC - standards body (remember the standards must be open (read data/write data)
  • GML (interoperable foramt) - interchange

\newpage

What did technical interoperability facilitate

From Map to Model The changing paradigm of map creation from cartography to data driven visualization

\newpage

\newpage

\newpage

\newpage

The world was a happy place.......

Our data was interoperable!

Our data became open data

\newpage

Background - open..... nesss

in communities

in government

in academia

\newpage

\newpage

Open..... nesss -- in government

The Shakespeare review [-@shakespeare_shakespeare_2013] indicate that the amount of government Open Data, at least in the UK, is only going to grow. Open data has the potential to trigger a revolution in how governments think about providing services to citizens and how they measure their success: this produces societal impact. This will require an understanding of citizen needs, behaviours, and mental models, and how to use data to improve services.

\newpage

A McKinsey Global Institute report examines the economic impact of Open Data [@mckinsey_open_2013] and estimates that globally open data could be worth a minimum of $3 trillion annually.

\newpage

Background - open..... nesss

in academia

Open inquiry is at the heart of the scientific enterprise..... Science’s powerful capacity for self-correction comes from this openness to scrutiny and challenge.

Science as an open enterprise [@royal_society_science_2012 p. 7].

The Royal Society’s report Science as an open enterprise [-@royal_society_science_2012] identifies how 21^st^ century communication technologies are changing the ways in which scientists conduct, and society engages with, science. The report recognises that ‘open’ enquiry is pivotal for the success of science, both in research and in society.

This goes beyond open access to publications (Open Access), to include access to data and other research outputs (Open Data), and the process by which data is turned into knowledge (Open Science).

The next generation open data in academia

Zenodo is a DATA REPOSITORY which offers:

  • accreditation
  • different licences
  • different exposure (private (closed), public (open) and embargoed (timestamped))
  • DOIs
  • is free at the point of use
  • is likely to be around for a long time
    • supported by Horizon 2020 and delivered by CREN

\newpage

The underlying rationale of Open Data is:

  • unfettered access to large amounts of ‘raw’ data
    • enables patterns of re-use and knowledge creation that were previously impossible.
    • improves transparency and efficiency
    • encourages innovative service delivery
  • introduces a range of data-mining and visualisation challenges,
    • which require multi-disciplinary collaboration across domains
    • catalyst to research and industry
  • supports the generation of new products, services and markets
  • the prize for succeeding is improved knowledge-led policy and practice that transforms
    • communities,
    • practitioners,
    • science and
    • society

the Open Data Landscape

Formal and informal data.OS versus VGI (Open Street Map)

Conclude with Linked Data + Open Data = Linked Open Data

So where are these new data products?

Data, data everywhere - but where are the new derivatives and services?

\newpage

Non-technical interoperability issues?

Issues surrounding non-technical interoperability include:

  • Policy interoperabilty
  • Licence interoperability
  • Legal interoperability
  • Social interoperability

We will focus on licence interoperability

\newpage

Policy Interoperability

Access control

Are you allowed to use data from this country?

From international journal of digital curation

Social Interoperability

Social interoperability is concerned about the environment and the human processes involved in the information exchange.

see here

It's a bit wooley

{The purpose of the Case Studies is to:}(https://rd-alliance.org/groups/rdacodata-legal-interoperability-ig.html)

Provide more specific information on best practices, as well as barriers and constraints in a number of different scientific domains and communities of practice (lessons learned); Illustrate the variety of legal frameworks that govern research data, different approaches to intellectual property and copyright across jurisdictions, different disciplinary expectations and norms, and alternative mechanisms to address the legal interoperability of data that have been tried in practice; and Identify opportunities for cross-disciplinary, cross-domain fertilization and collaboration and for new initiatives to address key barriers and constraints.

Also look here

Chatham house paper

Licence Interoperability

A specific form of legal interoperability

Example of applying the semantic web to licence interoperability

There is a multitude of formal and informal data.

\newpage

What is a licence?

Wikipedia state:

A license may be granted by a party ("licensor") to another party ("licensee") as an element of an agreement between those parties.

A shorthand definition of a license is "an authorization (by the licensor) to use the licensed material (by the licensee)."

Each of these data objects can be licenced in a different way. This shows some of the licences described by the RDFLicence ontology

\newpage

Concepts (derived from Formal Concept Analysis) surrounding licences

\newpage

Two lead organisations have developed legal frameworks for content licensing:

Until the release of CC version 4, published in November 2013, the CC licence did not cover data. Between them, CC and ODC licences can cover all forms of digital work.

  • There are many other licence types
  • Many are bespoke
    • Bespoke licences are difficult to manage
    • Many legacy datasets have bespoke licences

I'll describe CC in more detail

\newpage

Creative Commons Zero

Creative Commons Zero (CC0) is essentially public domain which allows:

  • Reproduction
  • Distribution
  • Derivations

Constraints on CC0

The following clauses constrain CC0:

  • Permissions
    • ND – No derivatives: the licensee can not derive new content from the resource.
  • Requirements
    • BY – By attribution: the licensee must attribute the source.
    • SA – Share-alike: if the licensee adapts the resource, it must be released under the same licence.
  • Prohibitions
    • NC – Non commercial: the licensee must not use the work commercially without prior approval.

CC license combinations

License Reproduction Distribution Derivation BY SA NC
CC0 X X X
CC-BY-ND X X X
CC-BY-NC-ND X X X X
CC-BY X X X X
CC-BY-SA X X X X X
CC-BY-NC X X X X X
CC-BY-NC-SA X X X X X X

Table: Creative Commons license combinations

\newpage

Why are licenses important?

  • They tell you what you can and can't do with 'stuff'
  • Very significant when multiple datasets are combined
    • It then becomes an issue of license compatibility

\newpage

Which is important when we mash up data

Certain licences when combined:

  • Are incompatible
    • Creating data islands
  • Inhibit commercial exploitation (NC)
  • Force the adoption of certain licences
    • If you want people to commercially exploit your stuff don't incorporate CC-BY-NC-SA data!
  • Stops the derivation of new works

A conceptual licence processing workflow. The licence processing service analyses the incoming licence metadata and determines if the data can be legally integrated and any resulting licence implications for the derived product.

\newpage

A rudimentry logic example

Data1 hasDerivedContentIn NewThing.

Data1 hasLicence a cc-by-sa.

What hasLicence a cc-by-sa? #reason here

If X hasDerivedContentIn Y and hasLicence Z then Y hasLicence Z. #reason here

Data2 hasDerivedContentIn NewThing.

Data2 hasLicence a cc-by-nc-sa.

What hasLicence a cc-by-nc-sa? #reason here

Nothing hasLicence a cc-by-nc-sa and hasLicence a cc-by-sa. #reason here

And processing this within the Protege reasoning environment


In [4]:
from IPython.display import YouTubeVideo
YouTubeVideo('jUzGF401vLc')


Out[4]:

\newpage

Here's something I prepared earlier

A live presentation (for those who weren't at the event).....


In [12]:
from IPython.display import YouTubeVideo
YouTubeVideo('tkRB5Rp1_W4')


Out[12]:

\newpage

A more robust logic

  • Would need to decouple licence incompatibility from licence name into licence clause (see table below)
  • Deal with all licence type
  • Provide recommendations based on desired derivative licence type
  • Link this through to the type of process in a workflow:
    • data derivation is, from a licence position, very different to contextual display
License Reproduction Distribution Derivation BY SA NC
CC0 X X X
CC-BY-ND X X X
CC-BY-NC-ND X X X X
CC-BY X X X X
CC-BY-SA X X X X X
CC-BY-NC X X X X X
CC-BY-NC-SA X X X X X X
ODC-PDDL X X X
ODC-BY X X X X
ODC-ODbL X X X X X
OGL 2.0 X X X X
OS OpenData X X X X ?

Table: Creative Commons license combinations

\newpage

OGC and Licence interoperability

  • The geo business landscape is increasingly based on integrating heterogeneous data to develop new products
  • Licence heterogeneity is a barrier to data integration and interoperability
  • A licence calculus can help resolve and identify heterogenties leading to
    • legal compliance
    • confidence
  • Use of standards and collaboration with organisations is crucial
  • Failure to do this could lead to breaches in data licenses
    • and we all know where that puts us........

\newpage

Ontologies, semantics and linked data

We've already been introduced to ontologies in the licence calculus example.

SLides to describe ontologies, semantics and linked data

Inlcude geo-logic from Brandon

Geo example:

Leeds is a city.

Yorkshire is a county.

Sheffield is a city.

Lancaster is a city.

Lancashire is a county.

Lancaster has a port.

What is Leeds?

Leeds isIn Yorkshire.

Sheffield isIn Yorkshire.

Lancaster isIn Lancashire.

What isIn Yorkshire?

If X isIn Y then Y contains X.

What contains Leeds?

Yorkshire borders Lancashire.

If X borders Y then Y borders X.

What borders Lancashire?

Yorkshire isIn UnitedKingdom.

Lancashire isIn UnitedKingdom.

#Transitivity

If X isIn Y and Y isIn Z then X isIn Z.

If X contains Y and Y contains Z then X contains Z

using proper isIn

Leeds is a city.

Yorkshire is a county.

Sheffield is a city.

Lancaster is a city.

Lancashire is a county.

Lancaster has a port.

What is Leeds?

Leeds is spatiallyWithin Yorkshire.

Sheffield is spatiallyWithin Yorkshire.

Lancaster is spatiallyWithin Lancashire.

What is spatiallyWithin Yorkshire?

If X is spatiallyWithin Y then Y spatiallyContains X.

What spatiallyContains Leeds?

Yorkshire borders Lancashire.

If X borders Y then Y borders X.

What borders Lancashire?

Yorkshire is spatiallyWithin UnitedKingdom.

Lancashire is spatiallyWithin UnitedKingdom.

#Transitivity

If X is spatiallyWithin Y and Y is spatiallyWithin Z then X is spatiallyWithin Z.

If X spatiallyContains Y and Y spatiallyContains Z then X spatiallyContains Z

What is spatiallyWithin UnitedKingdom?

Adding more......

Pudsey is spatiallyWithin Leeds.

Kirkstall is spatiallyWithin Leeds.

Meanwood is spatiallyWithin Leeds.

Roundhay is spatiallyWithin Leeds.

Scarcroft is spatiallyWithin Leeds.

and more



UnitedKingdom isPartOf Europe.

UnitedKingdom is a country.

If X isPartOf Y and X spatiallyContains Z then Z isPartOf Y.

What isPartOf Europe?

In [ ]:
and more

```

If X spatiallyContains Y and X is a city then Y is a place and Y is a cityPart.


Every city is a place.

What is a place.

```

In [ ]:
and more

```
UK isPartOf Europe.

UK is sameAs UnitedKingdom.

If X has a port then X borders Water.

What borders Water?
```

Questions

In terms of discussion I'm interested in how these issues affect you.........

\newpage

References