PROV Templates in Python

 Author: Stephan Kindermann
 Affiliation: DKRZ
 Community: ENES (Earth System Sciences)
 Version: 0.1 (July 2018)

Motivation:

PROV template expansion currently is only supported by the provconvert java tool.
initial generation and integration of PROV descriptions as part of ENES community efforts is oftenly done by adopting interactive languages like python.
The sharing PROV adoption narratives is well supported by using jupyter notebooks and python.
Core infrastructure services in ENES are implemented using python.
provenvert output can not be imported using the python prov package (only supporting PROV-json serialization format for import)
The java based provconvert tool thus is difficult to exploit in ENES use cases
What is needed in the short term are simple python based wrappers integrated in our community workflow, generating prov descriptions best on the basis of prov template instantiations.
The idea of using PROV templates yet is helpful, the only problem is the usage of provconvert in a python based working environment

Approach taken Thus a simple approach is taken which on the one hand side allows to use PROV templates and which on the other hand side allows for pure python based template instantiations. Drawback is that the elaborated prov expansion algorithm can not be used - yet to make the expansion explicit on the basis of python can also seen as an advantage as the community PROV adopters don't need to dig into the expansion algorithm implemented as part of provconvert (and eventual errors therein).

The approach taken is illustrated in the following:

PROV templates (being standard PROV documents) are generated in Python based on the prov library alongside PROV template instances.
A very simple instantiation algorithm is used to instantiate templates based on dictionaries containing the variable settings .. this instantiation algorithm is stepwise expanded

Generate a PROV template

PROV templates are generated in functions with all RROV variables as parameters. This function is called with prov template variable names to generate prov templates. When called with instances the result is an prov document corresponding to the instantiated prov template.

In the following the approach is illustrated based on a concrete examplec (corresponding to the first example in the provconvert tutorial).



In [11]:

    
# Define the variable parts in the template as dictionary keys
# dictionary values are the prov template variable bindings in one case
# and correspond to the variable instance settings in the other case
import prov.model as prov
template_dict = {
    'var_author':'var:author',
    'var_value':'var:value',
    'var_name':'var:name',
    'var_quote':'var:quote'
}
instance_dict = {
    'var_author':'orcid:0000-0002-3494-120X',
    'var_value':'A Little Provenance Goes a Long Way',
    'var_name':'Luc Moreau',
    'var_quote':'ex:quote1'
}



In [24]:

    
def make_prov(var_value,var_name,var_quote,var_author): 

    document = prov.ProvDocument()
    # ----- namspace settings --------------------------------------------------
    document.add_namespace('prov','http://www.w3.org/ns/prov#')
    document.add_namespace('var','http://openprovenance.org/var#>')
    document.add_namespace('vargen','http://openprovenance.org/vargen#')
    document.add_namespace('tmpl','http://openprovenance.org/tmpl#')
    document.add_namespace('foaf','http://xmlns.com/foaf/0.1/')
    document.add_namespace('ex', 'http://example.org/')
    document.add_namespace('orcid','http://orcid.org/')

    #document.set_default_namespace('http://example.org/0/')
    #document.add_namespace('rdf','http://www.w3.org/1999/02/22-rdf-syntax-ns#')
    #document.add_namespace('rdfs','http://www.w3.org/2000/01/rdf-schema#')
    #document.add_namespace('xsd','http://www.w3.org/2001/XMLSchema#')
    #document.add_namespace('ex1', 'http://example.org/1/')
    #document.add_namespace('ex2', 'http://example.org/2/')
    # ----------------------------------------------------------------------------

    bundle = document.bundle('vargen:bundleid')
    #bundle.set_default_namespace('http://example.org/0/')
    quote = bundle.entity(var_quote,(
         ('prov:value',var_value),
    ))    

    author = bundle.entity(var_author,(
        (prov.PROV_TYPE, "prov:Person"),
        ('foaf:name',var_name)
    )) 

    bundle.wasAttributedTo(var_quote,var_author)
    
    return document

def save_and_show(filename):
    doc1 = make_prov(**template_dict)
    print(doc1.get_provn())

    with open(filename, 'w') as provn_file:
        provn_file.write(doc1.get_provn())
    
    print("------")
    print("saved in file:",filename)

save_and_show('/home/stephan/test/xxxx.provn')









    



document
  prefix var <http://openprovenance.org/var#>>
  prefix vargen <http://openprovenance.org/vargen#>
  prefix tmpl <http://openprovenance.org/tmpl#>
  prefix foaf <http://xmlns.com/foaf/0.1/>
  prefix ex <http://example.org/>
  prefix orcid <http://orcid.org/>
  
  bundle vargen:bundleid
    entity(var:quote, [prov:value="var:value"])
    entity(var:author, [prov:type="prov:Person", foaf:name="var:name"])
    wasAttributedTo(var:quote, var:author)
  endBundle
endDocument
------
saved in file: /home/stephan/test/xxxx.provn

Compare generated PROV template with original example:



In [ ]:

    
# %load Downloads/ProvToolbox-Tutorial4-0.7.0/src/main/resources/template1.provn

document

  prefix var <http://openprovenance.org/var#>
  prefix vargen <http://openprovenance.org/vargen#>
  prefix tmpl <http://openprovenance.org/tmpl#>

  prefix foaf <http://xmlns.com/foaf/0.1/>
  
  bundle vargen:bundleId
    entity(var:quote, [prov:value='var:value'])
    entity(var:author, [prov:type='prov:Person', foaf:name='var:name'])
    wasAttributedTo(var:quote,var:author)
  endBundle

endDocument

Instantiate PROV template



In [14]:

    
doc2 = make_prov(**instance_dict)
print(doc2.get_provn())









    



ex:quote1
document
  prefix var <http://openprovenance.org/var#>>
  prefix vargen <http://openprovenance.org/vargen#>
  prefix tmpl <http://openprovenance.org/tmpl#>
  prefix foaf <http://xmlns.com/foaf/0.1/>
  prefix ex <http://example.org/>
  
  bundle vargen:bundleid
    entity(ex:quote1, [prov:value="A Little Provenance Goes a Long Way"])
    entity(ex:0000-0002-3494-120X, [prov:type="prov:Person", foaf:name="Luc Moreau"])
    wasAttributedTo(ex:quote1, ex:0000-0002-3494-120X)
  endBundle
endDocument



In [25]:

    
%matplotlib  inline
doc1.plot()
doc2.plot()



In [ ]:

    
# take same instantiation as in the tutorial:
# %load Downloads/ProvToolbox-Tutorial4-0.7.0/src/main/resources/binding1.ttl
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix tmpl: <http://openprovenance.org/tmpl#> .
@prefix var: <http://openprovenance.org/var#> .
@prefix ex: <http://example.com/#> .

var:author a prov:Entity;
           tmpl:value_0 <http://orcid.org/0000-0002-3494-120X>.
var:name   a prov:Entity;
           tmpl:2dvalue_0_0 "Luc Moreau".
var:quote  a prov:Entity;
           tmpl:value_0 ex:quote1.
var:value  a prov:Entity;
           tmpl:2dvalue_0_0 "A Little Provenance Goes a Long Way".



In [ ]:

Instantiate PROV template using provconvert and compare results



In [15]:

    
!provconvert -infile test/template1.provn -bindings test/binding1.ttl -outfile test/doc1.provn









    



bindings version is 1



In [ ]:

    
# %load test/doc1.provn
document
bundle uuid:b07bc92f-f16a-443c-9e0a-3bda4063fc10
prefix foaf <http://xmlns.com/foaf/0.1/>
prefix pre_0 <http://orcid.org/>
prefix ex <http://example.com/#>
prefix uuid <urn:uuid:>

entity(ex:quote1,[prov:value = "A Little Provenance Goes a Long Way" %% xsd:string])
entity(pre_0:0000-0002-3494-120X,[prov:type = 'prov:Person', foaf:name = "Luc Moreau" %% xsd:string])
wasAttributedTo(ex:quote1, pre_0:0000-0002-3494-120X)
endBundle
endDocument



In [18]:

    
!provconvert -infile test/doc1.provn -outfile test/doc1.png



In [19]:

    
!provconvert -infile test/template1.provn -outfile test/template1.png



In [ ]:

    
# %load Downloads/ProvToolbox-Tutorial4-0.7.0/target/doc1.provn
document
bundle uuid:4c7236d5-6420-4a88-b192-6089e27aa88e
prefix foaf <http://xmlns.com/foaf/0.1/>
prefix pre_0 <http://orcid.org/>
prefix ex <http://example.com/#>
prefix uuid <urn:uuid:>

entity(ex:quote1,[prov:value = "A Little Provenance Goes a Long Way" %% xsd:string])
entity(pre_0:0000-0002-3494-120X,[prov:type = 'prov:Person', foaf:name = "Luc Moreau" %% xsd:string])
wasAttributedTo(ex:quote1, pre_0:0000-0002-3494-120X)
endBundle
endDocument



In [ ]:

    
#------------------------to be removeed ------------------------------------



In [ ]:

    
%matplotlib inline
document.plot()



In [ ]:

    
!cat Downloads/ProvToolbox-Tutorial4-0.7.0/Makefile

Show provconvert result to compare



In [ ]:

    
#def set_template_vars(var_dict):
#    for var, value in var_dict.items():
#        globals()[var] = value