DKRZ data ingest information handling

This demo notebook is for data managers only !

The submission_forms package provides a collection of components to support the management of information related to data ingest related activities (data transport, data checking, data publication and data archival):

data submission related information management
- who, when, what, for which project, data characteristics
data management related information management
- ingest, quality assurance, publication, archiving

Background: approaches at other sites

example workflows in other data centers:

DKRZ ingest workflow system

Data ingest request via

jupyter notebook on server (http://data-forms.dkrz.de:8080), or
jupyter notebook filled at home and sent to DKRZ (email, rt system)
- download form_template from: (tbd. github or redmine or recommend pypi installation only - which includes all templates .. )

Data ingest request workflow:

ingest request related information is stored in json format in git repo
all workflow steps are reflected in json subfields
workflow json format has W3C prov aligned schema and can be transformed to W3C prov format (and visualized as a prov graph)
for search git repo can be indexed into a (in-memory) key-value DB



In [1]:

    
# do this in case you want to change imported module code while working with this notebook
# -- (for development and testing puposes only)
%load_ext autoreload
%autoreload 2



In [16]:

    
# to generate empyty project form including all options for variables
# e.g.: 
ACTIVITY_STATUS = "0:open, 1:in-progress ,2:action-required, 3:paused,4:closed"          
ERROR_STATUS = "0:open,1:ok,2:error"
ENTITY_STATUS = "0:open,1:stored,2:submitted,3:re-opened,4:closed"
CHECK_STATUS = "0:open,1:warning, 2:error,3:ok"
import dkrz_forms
#from dkrz_forms import form_handler, utils
#sf_t = utils.generate_project_form('ESGF_replication')
#print(checks.get_options(sf_t.sub.activity.status))

demo examples - step by step

Data managers have two separate application scenarios for data ingest information management:

interactive information adaptation for specific individual data ingest activities
automatic information adaptation by integrating in own data management scripts (e.g. data qualitiy assurance or data publication)
- examples: successfull ESGF publication can update automatically the publication workflow step information

Alternative A)
- check out out git repo https://gitlab.dkrz.de/DKRZ-CMIP-Pool/data_forms_repo
- this repo contains all completed submission forms
- all data manager related changes are also committed there
- subdirectories in this repo relate to the individual projects (e.g. CMIP6, CORDEX, ESGF_replication, ..)
- each entry there contains the last name of the data submission originator
Alternative B) (not yet documented, only prototype)
- use search interface and API of search index on all submision forms



In [4]:

    
## To do: include different examples how to query for data ingest activities based on different properties

#info_file = "/home/stephan/tmp/Repos/form_repo/test/test_testsuite_123.json"
#info_file = "/home/stephan/Forms/local_repo/test/test_testsuite_1.json"
#info_file = "/home/stephan/Forms/local_repo/CORDEX/CORDEX_kindermann_2.json"
info_file = "/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json"

from dkrz_forms import form_handler, utils,checks,wflow_handler
from datetime import datetime

my_form = utils.load_workflow_form(info_file)



In [5]:

    
wflow_dict = wflow_handler.get_wflow_description(my_form)
list(wflow_dict.values())









    Out[5]:





['sub', 'rev', 'ing', 'qua', 'pub']



In [6]:

    
wflow_handler.rename_action('data_submission_review',my_form)









    Out[6]:





'rev'



In [7]:

    
my_form = wflow_handler.start_action('data_submission_review',my_form,"stephan kindermann")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 855cc93] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## stephan kindermann: rev started
 2 files changed, 184 insertions(+), 37 deletions(-)



In [8]:

    
myform = wflow_handler.update_action('data_submission_review',my_form,"stephan kindermann")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master c9b9be5] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## stephan kindermann: rev updated
 1 file changed, 3 insertions(+), 3 deletions(-)



In [9]:

    
review_report = {}
review_report['comment'] = 'needed to change and correct submission form'
review_report['additional_info'] = "mail exchange with a@b with respect to question ..."

myform = wflow_handler.finish_action('data_submission_review',my_form,"stephan kindermann",review_report)









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master efadd90] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## stephan kindermann: rev finished
 1 file changed, 8 insertions(+), 8 deletions(-)

interactive "help": use ?form.part and tab completion:



In [10]:

    
?my_form



In [11]:

    
?my_form.sub



In [ ]:

    
my_form.sub.entity_out.report.

Display status of report



In [12]:

    
sf = form_handler.save_form(my_form, "init")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master e8dd4ef] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## init
 1 file changed, 3 insertions(+), 3 deletions(-)



In [138]:

    
report = checks.check_report(my_form,"sub")
checks.display_report(report)









    




 Summary of check results 
Value Check Result Check Comment
collection_pid NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
scientific_context NOT OK Error: mandatory string not set
submission_type NOT OK Error: option parameter not set
update_notification NOT OK Error: option parameter not set



In [13]:

    
my_form.rev.entity_in.check_status









    Out[13]:





'0:open,1:warning, 2:error,3:ok'

Display status of form



In [14]:

    
my_form.sub.activity.ticket_url









    Out[14]:





'mandatory'



In [17]:

    
part = dkrz_forms.checks.check_step_form(my_form,"sub")
dkrz_forms.checks.display_check(part,"sub")









    



=============================================================================
Workflow step:  sub
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
email OK mandatory string not empty
first_name OK mandatory string not empty
keyword OK mandatory string not empty
last_name OK mandatory string not empty
responsible_person OK mandatory string not empty






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
form_dir OK mandatory string not empty
form_path OK mandatory string not empty
source_path NOT OK Error: mandatory string not set
tag OK Warning: optional parameter not set
version OK Warning: optional parameter not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status OK ok: Valid option setting
checks_done NOT OK Error: mandatory string not set
commit_message OK ok: parameter is optional
form_json OK mandatory string not empty
form_name OK mandatory string not empty
form_repo OK mandatory string not empty
form_repo_path OK mandatory string not empty
pwd OK mandatory string not empty
status OK ok: Valid option setting
submission_form NOT OK Error: mandatory string not set
submission_json NOT OK Error: mandatory string not set
submission_repo NOT OK Error: mandatory string not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
commit_hash OK Warning: optional parameter not set
end_time OK Warning: optional parameter not set
error_status NOT OK Error: option parameter not set
keyword OK mandatory string not empty
method NOT OK Error: mandatory string not set
pwd NOT OK Error: mandatory string not set
start_time OK mandatory string not empty
status OK ok: Valid option setting
ticket_id OK Warning: optional parameter not set
ticket_url NOT OK Error: mandatory string not set
timestamp OK ok: parameter is optional



In [18]:

    
## global check
res  = checks.check_generic_form(my_form)
checks.display_checks(my_form,res)









    



=============================================================================
Workflow step:  data_submission
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
email OK mandatory string not empty
first_name OK mandatory string not empty
keyword OK mandatory string not empty
last_name OK mandatory string not empty
responsible_person OK mandatory string not empty






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
form_dir OK mandatory string not empty
form_path OK mandatory string not empty
source_path NOT OK Error: mandatory string not set
tag OK Warning: optional parameter not set
version OK Warning: optional parameter not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status OK ok: Valid option setting
checks_done NOT OK Error: mandatory string not set
commit_message OK ok: parameter is optional
form_json OK mandatory string not empty
form_name OK mandatory string not empty
form_repo OK mandatory string not empty
form_repo_path OK mandatory string not empty
pwd OK mandatory string not empty
status OK ok: Valid option setting
submission_form NOT OK Error: mandatory string not set
submission_json NOT OK Error: mandatory string not set
submission_repo NOT OK Error: mandatory string not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
commit_hash OK Warning: optional parameter not set
end_time OK Warning: optional parameter not set
error_status NOT OK Error: option parameter not set
keyword OK mandatory string not empty
method NOT OK Error: mandatory string not set
pwd NOT OK Error: mandatory string not set
start_time OK mandatory string not empty
status OK ok: Valid option setting
ticket_id OK Warning: optional parameter not set
ticket_url NOT OK Error: mandatory string not set
timestamp OK ok: parameter is optional






    



=============================================================================
Workflow step:  data_submission_review
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
responsible_person OK mandatory string not empty






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
checks_done NOT OK Error: mandatory string not set
commit_message OK Warning: optional parameter not set
form_json NOT OK Error: mandatory string not set
form_name NOT OK Error: mandatory string not set
form_repo NOT OK Error: mandatory string not set
form_repo_path NOT OK Error: mandatory string not set
pwd NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
submission_form NOT OK Error: mandatory string not set
submission_json NOT OK Error: mandatory string not set
submission_repo NOT OK Error: mandatory string not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status OK ok: Valid option setting
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status OK ok: Valid option setting
tag OK Warning: optional parameter not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
end_time OK ok: parameter is optional
error_status OK ok: Valid option setting
start_time OK mandatory string not empty
status OK ok: Valid option setting
ticket_id NOT OK Error: mandatory string not set
ticket_url NOT OK Error: mandatory string not set
timestamp OK ok: parameter is optional






    



=============================================================================
Workflow step:  data_ingest
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
responsible_person NOT OK Error: mandatory string not set






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
end_time OK Warning: optional parameter not set
error_status NOT OK Error: option parameter not set
start_time NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
ticket_id NOT OK Error: mandatory string not set
timestamp OK Warning: optional parameter not set






    



=============================================================================
Workflow step:  data_quality_assurance
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
responsible_person NOT OK Error: mandatory string not set






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
end_time OK Warning: optional parameter not set
error_status NOT OK Error: option parameter not set
follow_up_ticket OK Warning: optional parameter not set
qua_tool_version NOT OK Error: mandatory string not set
start_time NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
ticket_id NOT OK Error: mandatory string not set






    



=============================================================================
Workflow step:  data_publication
PROV entity:  agent






    




 Summary of check results 
Value Check Result Check Comment
responsible_person NOT OK Error: mandatory string not set
trigger NOT OK Error: option parameter not set






    



PROV entity:  entity_in






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  entity_out






    




 Summary of check results 
Value Check Result Check Comment
check_status NOT OK Error: option parameter not set
comment OK Warning: optional parameter not set
date NOT OK Error: mandatory string not set
search_string OK Warning: optional parameter not set
status NOT OK Error: option parameter not set
tag OK Warning: optional parameter not set






    



PROV entity:  activity






    




 Summary of check results 
Value Check Result Check Comment
comment OK Warning: optional parameter not set
end_time OK Warning: optional parameter not set
error_status NOT OK Error: option parameter not set
follow_up_ticket OK Warning: optional parameter not set
start_time NOT OK Error: mandatory string not set
status NOT OK Error: option parameter not set
ticket_id NOT OK Error: mandatory string not set
timestamp OK Warning: optional parameter not set



In [19]:

    
print(my_form.sub.entity_out.status)
print(my_form.rev.entity_in.form_json)
print(my_form.sub.activity.ticket_id)









    



0:open
mandatory
optional

Step 2: complete information in specific workflow step

workflow steps of specific project are given in my_form.workflow
normally my_form.workflow is given as
- [[u'sub', u'data_submission'], [u'rev', u'data_submission_review'], [u'ing', u'data_ingest'], [u'qua', u'data_quality_assurance']]
thus my_form.sub would contain the data_submission related information, my_form.rev the review related one etc.
- the workflow step related information dictionaries are configured based on config/project_config.py



In [20]:

    
print(my_form.workflow)









    



[['sub', 'data_submission'], ['rev', 'data_submission_review'], ['ing', 'data_ingest'], ['qua', 'data_quality_assurance'], ['pub', 'data_publication']]

each workflow_step dictionary is structured consistently according to

activity: activity related information
agent: responsible person related info
entity_out: specific output information of this workflow step



In [21]:

    
review = my_form.rev
?review.activity

workflow step: submission review

ToDo: Split in start/end information update actions



In [22]:

    
my_form.rev.









    



  File "<ipython-input-22-c6b85c6f6d24>", line 1
    my_form.rev.
                ^
SyntaxError: invalid syntax



In [23]:

    
workflow_form = utils.load_workflow_form(info_file)
   
review = workflow_form.rev

# any additional information keys can be added,
# yet they are invisible to generic information management tools ..
workflow_form.status = "review"

review.activity.status = "1:in-review"
review.activity.start_time = str(datetime.now())
review.activity.review_comment = "data volume check to be done"
review.agent.responsible_person = "sk"

sf = form_handler.save_form(workflow_form, "sk: review started")

review.activity.status = "3:accepted"
review.activity.ticket_id = "25389"
review.activity.end_time = str(datetime.now())

review.entity_out.comment = "This submission is related to submission abc_cde"
review.entity_out.tag = "sub:abc_cde"  # tags are used to relate different forms to each other
review.entity_out.report = {'x':'y'}   # result of validation in a dict (self defined properties)

# ToDo: test and document save_form for data managers (config setting for repo)   
sf = form_handler.save_form(workflow_form, "kindermann: form_review()")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master d49ce54] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## sk: review started
 1 file changed, 6 insertions(+), 5 deletions(-)


Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 1c10c62] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kindermann: form_review()
 1 file changed, 11 insertions(+), 9 deletions(-)

Comment: alternatively in tools workflow_step related information could also be directly given and assigned via dictionaries, yet this is only recommended for data managers making sure the structure is consistent with the preconfigured one given in config/project_config.py

example validation.activity.__dict__ = data_manager_generated_dict



In [24]:

    
workflow_form = utils.load_workflow_form(info_file)
   
ingest = workflow_form.ing



In [25]:

    
?ingest.entity_out



In [26]:

    
# agent related info
workflow_form.status = "ingest"

ingest.activity.status = "started"
ingest.agent.responsible_person = "hdh"
ingest.activity.start_time=str(datetime.now())

# activity related info

ingest.activity.comment = "data pull: credentials needed for remote site"
sf = form_handler.save_form(workflow_form, "kindermann: form_review()")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 73c0689] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kindermann: form_review()
 1 file changed, 6 insertions(+), 6 deletions(-)



In [27]:

    
ingest.activity.status = "completed"
ingest.activity.end_time = str(datetime.now())

# report of the ingest process (entity_out of ingest workflow step)
ingest_report = ingest.entity_out
ingest_report.tag = "a:b:c"  # tag structure to be defined
ingest_report.status = "completed"
# free entries for detailed report information
ingest_report.report.remote_server = "gridftp.awi.de://export/data/CMIP6/test"
ingest_report.report.server_credentials = "in server_cred.krb keypass"
ingest_report.report.target_path = ".."
sf = form_handler.save_form(workflow_form, "kindermann: form_review()")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 4c1bddb] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kindermann: form_review()
 1 file changed, 12 insertions(+), 8 deletions(-)



In [ ]:

    
ingest_report.report.

workflow step: data quality assurance



In [28]:

    
from datetime import datetime
workflow_form = utils.load_workflow_form(info_file)
   
qua = workflow_form.qua



In [29]:

    
workflow_form.status = "quality assurance"
qua.agent.responsible_person = "hdh"

qua.activity.status = "starting" 
qua.activity.start_time = str(datetime.now())

sf = form_handler.save_form(workflow_form, "hdh: qa start")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master bbe07bf] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## hdh: qa start
 1 file changed, 5 insertions(+), 5 deletions(-)



In [30]:

    
qua.entity_out.status = "completed"
qua.entity_out.report = {
    "QA_conclusion": "PASS",
    "project": "CORDEX",
    "institute": "CLMcom",
    "model": "CLMcom-CCLM4-8-17-CLM3-5",
    "domain": "AUS-44",
    "driving_experiment":  [ "ICHEC-EC-EARTH"],
    "experiment": [ "history", "rcp45", "rcp85"],
    "ensemble_member": [ "r12i1p1" ],
    "frequency": [ "day", "mon", "sem" ],
    "annotation":
    [
        {
            "scope": ["mon", "sem"],
            "variable": [ "tasmax", "tasmin", "sfcWindmax" ],
            "caption": "attribute <variable>:cell_methods for climatologies requires <time>:climatology instead of time_bnds",
            "comment": "due to the format of the data, climatology is equivalent to time_bnds",
            "severity": "note"
        }
    ]
}
sf = form_handler.save_form(workflow_form, "hdh: qua complete")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master c187621] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## hdh: qua complete
 1 file changed, 43 insertions(+), 5 deletions(-)

workflow step: data publication



In [31]:

    
workflow_form = utils.load_workflow_form(info_file)

workflow_form.status = "publishing"

pub = workflow_form.pub
pub.agent.responsible_person = "katharina"
pub.activity.status = "starting"
pub.activity.start_time = str(datetime.now())

sf = form_handler.save_form(workflow_form, "kb: publishing")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 4ce028d] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kb: publishing
 1 file changed, 5 insertions(+), 5 deletions(-)



In [32]:

    
pub.activity.status = "completed"
pub.activity.comment = "..."
pub.activity.end_time = ".."
pub.activity.report = {'model':"MPI-M"}   # activity related report information

pub.entity_out.report = {'model':"MPI-M"} # the report of the publication action - all info characterizing the publication
sf = form_handler.save_form(workflow_form, "kb: published")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 3caec11] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kb: published
 1 file changed, 12 insertions(+), 8 deletions(-)



In [33]:

    
sf = form_handler.save_form(workflow_form, "kindermann: form demo run 1")









    




Form Handler - save form status message:
entity_in.form_path and entity_out.form_repo_path
/home/stephan/Forms/CORDEX/CORDEX_mm_mm.ipynb
/opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.ipynb
 --- form stored in transfer format in: /opt/jupyter/notebooks/form_directory/CORDEX/CORDEX_mm_mm.json
 
 --- commit message:[master 71e4e81] Form Handler: submission form for user mm saved using prefix CORDEX_mm_mm ## kindermann: form demo run 1
 1 file changed, 3 insertions(+), 3 deletions(-)



In [34]:

    
sf.sub.activity.commit_hash









    Out[34]:





'optional'



In [ ]:

Value	Check Result	Check Comment
collection_pid	NOT OK	Error: option parameter not set
comment	OK	Warning: optional parameter not set
scientific_context	NOT OK	Error: mandatory string not set
submission_type	NOT OK	Error: option parameter not set
update_notification	NOT OK	Error: option parameter not set