Metadata and change recording

Each pymrio core system object contains a field 'meta' which stores meta data as well as changes to the MRIO system. This data is stored as json file in the root of a saved MRIO data and accessible through the attribute '.meta':


In [31]:
import pymrio
io = pymrio.load_test()

In [32]:
io.meta


Out[32]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20191004 14:39:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio
20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json

In [33]:
io.meta('Loaded the pymrio test sytem')


Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20191004 14:39:58 - NOTE -  Loaded the pymrio test sytem
20191004 14:39:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio
20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json

We can now do several steps to modify the system, for example:


In [34]:
io.calc_all()
io.aggregate(region_agg = 'global')


Out[34]:
<pymrio.core.mriosystem.IOSystem at 0x7fe99a0614d0>

In [35]:
io.meta


Out[35]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate population vector
20191004 14:39:58 - MODIFICATION -  Aggregate industry output x
20191004 14:39:58 - MODIFICATION -  Aggregate transaction matrix Z
20191004 14:39:58 - MODIFICATION -  Aggregate final demand y
20191004 14:39:58 - MODIFICATION -  Reset to absolute flows
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions
 ... (more lines in history)

Notes can added at any time:


In [36]:
io.meta.note('First round of calculations finished')

In [37]:
io.meta


Out[37]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20191004 14:39:58 - NOTE -  First round of calculations finished
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate population vector
20191004 14:39:58 - MODIFICATION -  Aggregate industry output x
20191004 14:39:58 - MODIFICATION -  Aggregate transaction matrix Z
20191004 14:39:58 - MODIFICATION -  Aggregate final demand y
20191004 14:39:58 - MODIFICATION -  Reset to absolute flows
 ... (more lines in history)

In addition, all file io operations are recorde in the meta data:


In [38]:
io.save_all('/tmp/foo')


Out[38]:
<pymrio.core.mriosystem.IOSystem at 0x7fe99a0614d0>

In [39]:
io_new = pymrio.load_all('/tmp/foo')

In [40]:
io_new.meta


Out[40]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /tmp/foo/metadata.json
History:
20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs
20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/emissions
20191004 14:39:58 - FILEIO -  Loaded IO system from /tmp/foo
20191004 14:39:58 - FILEIO -  Saved testmrio to /tmp/foo
20191004 14:39:58 - NOTE -  First round of calculations finished
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate population vector
 ... (more lines in history)

The top level meta data can be changed as well. These changes will also be recorded in the history:


In [41]:
io_new.meta.change_meta('Version', 'v2')

In [42]:
io_new.meta


Out[42]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v2
File: /tmp/foo/metadata.json
History:
20191004 14:39:58 - METADATA_CHANGE -  Changed parameter "version" from "v1" to "v2"
20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs
20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/emissions
20191004 14:39:58 - FILEIO -  Loaded IO system from /tmp/foo
20191004 14:39:58 - FILEIO -  Saved testmrio to /tmp/foo
20191004 14:39:58 - NOTE -  First round of calculations finished
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions
20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
20191004 14:39:58 - MODIFICATION -  Aggregate extensions...
 ... (more lines in history)

To get the full history list, use:


In [43]:
io_new.meta.history


Out[43]:
['20191004 14:39:58 - METADATA_CHANGE -  Changed parameter "version" from "v1" to "v2"',
 '20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs',
 '20191004 14:39:58 - FILEIO -  Added satellite account from /tmp/foo/emissions',
 '20191004 14:39:58 - FILEIO -  Loaded IO system from /tmp/foo',
 '20191004 14:39:58 - FILEIO -  Saved testmrio to /tmp/foo',
 '20191004 14:39:58 - NOTE -  First round of calculations finished',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20191004 14:39:58 - MODIFICATION -  Aggregate extensions...',
 '20191004 14:39:58 - MODIFICATION -  Aggregate extensions...',
 '20191004 14:39:58 - MODIFICATION -  Aggregate population vector',
 '20191004 14:39:58 - MODIFICATION -  Aggregate industry output x',
 '20191004 14:39:58 - MODIFICATION -  Aggregate transaction matrix Z',
 '20191004 14:39:58 - MODIFICATION -  Aggregate final demand y',
 '20191004 14:39:58 - MODIFICATION -  Reset to absolute flows',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20191004 14:39:58 - MODIFICATION -  Leontief matrix L calculated',
 '20191004 14:39:58 - MODIFICATION -  Coefficient matrix A calculated',
 '20191004 14:39:58 - MODIFICATION -  Industry output x calculated',
 '20191004 14:39:58 - NOTE -  Loaded the pymrio test sytem',
 '20191004 14:39:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio',
 '20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json']

This can be restricted to one of the history types by:


In [44]:
io_new.meta.modification_history


Out[44]:
['20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20191004 14:39:58 - MODIFICATION -  Aggregate extensions...',
 '20191004 14:39:58 - MODIFICATION -  Aggregate extensions...',
 '20191004 14:39:58 - MODIFICATION -  Aggregate population vector',
 '20191004 14:39:58 - MODIFICATION -  Aggregate industry output x',
 '20191004 14:39:58 - MODIFICATION -  Aggregate transaction matrix Z',
 '20191004 14:39:58 - MODIFICATION -  Aggregate final demand y',
 '20191004 14:39:58 - MODIFICATION -  Reset to absolute flows',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20191004 14:39:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20191004 14:39:58 - MODIFICATION -  Leontief matrix L calculated',
 '20191004 14:39:58 - MODIFICATION -  Coefficient matrix A calculated',
 '20191004 14:39:58 - MODIFICATION -  Industry output x calculated']

or


In [45]:
io_new.meta.note_history


Out[45]:
['20191004 14:39:58 - NOTE -  First round of calculations finished',
 '20191004 14:39:58 - NOTE -  Loaded the pymrio test sytem']