Consensus Storage

In this notebook, I detail the structure of the HDF5 when storing consensuses.

Input data format

The input data are stored in crowdastro.h5 and crowdastro.csv (v0.1.0):

• /
  • atlas
    • cdfs
      • images_2x2
      • images_5x5
      • classification_positions
      • classification_combinations
      • positions
      • training_indices
      • validation_indices
      • testing_indices
  • swire
    • cdfs
      • images_2x2
      • images_5x5
      • catalogue

Output data format

• /
  • atlas
    • cdfs
      • consensus_positions
      • consensus_combinations
      • consensus_agreements

All valid consensuses are stored in /atlas/cdfs/{consensus_positions,consensus_combinations} with the position of the classification and the corresponding radio combination respectively. There is also a dataset for storing the agreements, both radio and location. All of these datasets also contain the ATLAS index they are associated with.