OMX Validator

The OMX validator is a Jupyter notebook hosted in Google Colaboratory. It is an interactive Python environment that validates OMX matrices using the openmatrix library. The validator has been tested with the example omx file. OMX files can also be inspected with the OMX Viewer.

Upload Files

Select local files for upload via the Table of Contents + Files + Upload. Run the code cell below to list the OMX files uploaded by clicking on the [ ] play button. While testing, you may need to reset the UI control, which you can do via Runtime + Restart runtime. The Files control can be accessed by clicking the File button.



In [31]:

    
!ls -d *omx









    



example.omx

Validator Functions

This step installs the openmatrix package from pypi.org, which includes the OMX validation functions and command line tool. Run the code cell to install the package.



In [25]:

    
!pip install openmatrix









    



Collecting openmatrix
  Downloading https://files.pythonhosted.org/packages/05/e8/6a8227e97127a2a1db98f5a22ca565bfe942e89de0644837627920101213/OpenMatrix-0.3.5.0-py3-none-any.whl
Requirement already satisfied: tables>=3.1.0 in /usr/local/lib/python3.6/dist-packages (from openmatrix) (3.4.4)
Requirement already satisfied: numpy>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from openmatrix) (1.18.4)
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/dist-packages (from tables>=3.1.0->openmatrix) (1.12.0)
Requirement already satisfied: numexpr>=2.5.2 in /usr/local/lib/python3.6/dist-packages (from tables>=3.1.0->openmatrix) (2.7.1)
Installing collected packages: openmatrix
Successfully installed openmatrix-0.3.5.0

Validate

This section validates the OMX file against the Specification by using the omx-validate command line tool. Specify the file to validate in the command line call below and then run the code cell.

The following checks are run and an overall Pass or Fail is returned at the end.

Has OMX_VERSION attribute set to 0.2
Has SHAPE array attribute set to two item integer array
Has data group for matrices
Matrix shape matches file shape
Uses common data types (float or int) for matrices
Matrices chunked for faster I/O
Uses zlib compression and level 1 if compression used
Has NA attribute if desired (but not required)
Has lookup group for labels/indexes if desired (but not required)
Lookup length matches shape
Uses common data types (int or str) for lookups
Has Lookup DIM attribute of 0 (row) or 1 (column) if desired (but not required)



In [33]:

    
!omx-validate example.omx









    



File contents: example.omx
example.omx (File) ''
Last modif.: 'Sun May 17 14:24:38 2020'
Object Tree: 
/ (RootGroup) ''
/data (Group) ''
/data/FARE (CArray(485, 485), shuffle, zlib(1)) 'Fare Transit'
/data/IVT (CArray(485, 485), shuffle, zlib(1)) 'In-vehicle time Transit'
/data/IVTBUS (CArray(485, 485), shuffle, zlib(1)) 'In-vehicle time-TSys(Bus) Transit'
/data/IVTRAIL (CArray(485, 485), shuffle, zlib(1)) 'In-vehicle time-TSys(Rail) Transit'
/data/IVTTRAM (CArray(485, 485), shuffle, zlib(1)) 'In-vehicle time-TSys(Tram) Transit'
/data/OWT (CArray(485, 485), shuffle, zlib(1)) 'Origin wait time Transit'
/data/TRANSFERS (CArray(485, 485), shuffle, zlib(1)) 'Number of transfers Transit'
/data/TWT (CArray(485, 485), shuffle, zlib(1)) 'Transfer wait time Transit'
/lookup (Group) ''
/lookup/NO (Array(485,)) ''


Check 1: Has OMX_VERSION attribute set to 0.2
  File version is 0.2: Pass

Check 2: Has SHAPE array attribute set to two item integer array
  Length is 2: Pass
  First item is integer: Pass
  Second item is integer: Pass
  Shape: (485, 485)

Check 3: Has data group for matrices
  Group: Pass
  Number of Matrices: 8
  Matrix names: ['FARE', 'IVT', 'IVTBUS', 'IVTRAIL', 'IVTTRAM', 'OWT', 'TRANSFERS', 'TWT']

Check 4: Matrix shape matches file shape
  Matrix shape:  FARE : (485, 485) : Pass
  Matrix shape:  IVT : (485, 485) : Pass
  Matrix shape:  IVTBUS : (485, 485) : Pass
  Matrix shape:  IVTRAIL : (485, 485) : Pass
  Matrix shape:  IVTTRAM : (485, 485) : Pass
  Matrix shape:  OWT : (485, 485) : Pass
  Matrix shape:  TRANSFERS : (485, 485) : Pass
  Matrix shape:  TWT : (485, 485) : Pass

Check 5: Uses common data types (float or int) for matrices
  Matrix:  FARE : float64 : Pass
  Matrix:  IVT : float64 : Pass
  Matrix:  IVTBUS : float64 : Pass
  Matrix:  IVTRAIL : float64 : Pass
  Matrix:  IVTTRAM : float64 : Pass
  Matrix:  OWT : float64 : Pass
  Matrix:  TRANSFERS : float64 : Pass
  Matrix:  TWT : float64 : Pass

Check 6: Matrices chunked for faster I/O
  Matrix chunkshape:  FARE : (16, 485) : Pass
  Matrix chunkshape:  IVT : (16, 485) : Pass
  Matrix chunkshape:  IVTBUS : (16, 485) : Pass
  Matrix chunkshape:  IVTRAIL : (16, 485) : Pass
  Matrix chunkshape:  IVTTRAM : (16, 485) : Pass
  Matrix chunkshape:  OWT : (16, 485) : Pass
  Matrix chunkshape:  TRANSFERS : (16, 485) : Pass
  Matrix chunkshape:  TWT : (16, 485) : Pass

Check 7: Uses zlib compression if compression used
  Matrix compression library and level:  FARE : zlib : 1 : Pass
  Matrix compression library and level:  IVT : zlib : 1 : Pass
  Matrix compression library and level:  IVTBUS : zlib : 1 : Pass
  Matrix compression library and level:  IVTRAIL : zlib : 1 : Pass
  Matrix compression library and level:  IVTTRAM : zlib : 1 : Pass
  Matrix compression library and level:  OWT : zlib : 1 : Pass
  Matrix compression library and level:  TRANSFERS : zlib : 1 : Pass
  Matrix compression library and level:  TWT : zlib : 1 : Pass

Check 8: Has NA attribute if desired (but not required)
  Matrix NA attribute:  FARE : Fail
  Matrix NA attribute:  IVT : Fail
  Matrix NA attribute:  IVTBUS : Fail
  Matrix NA attribute:  IVTRAIL : Fail
  Matrix NA attribute:  IVTTRAM : Fail
  Matrix NA attribute:  OWT : Fail
  Matrix NA attribute:  TRANSFERS : Fail
  Matrix NA attribute:  TWT : Fail

Check 9: Has lookup group for labels/indexes if desired (but not required)
  Group: Pass
  Number of Lookups: 1
  Lookups names: ['NO']

Check 10: Lookup shapes are 1-d and match file shape
  Lookup:  NO : (485,) : Pass

Check 11: Uses common data types (int or str) for lookups
  Lookup:  NO : uint32 : Pass

Check 12: Has Lookup DIM attribute of 0 (row) or 1 (column) if desired (but not required)
  Not supported at this time by the Python openmatrix package

Overall result 
  Check 1 : Required : Pass
  Check 2 : Required : Pass
  Check 3 : Required : Pass
  Check 4 : Required : Pass
  Check 5 : Required : Pass
  Check 6 : Required : Pass
  Check 7 : Not required : Pass
  Check 8 : Not required : Fail
  Check 9 : Not required : Pass
  Check 10 : Not required : Pass
  Check 11 : Not required : Pass
  Check 12 : Not required : Fail
  Overall :  Pass