ROIseries "Wine Tasting"

This example demonstrates:

  • the feature_sommelier

aim:

Identify clouds in a series of Sentinel 2 (S2) Satellite images

strategy

 1. Feature Generation
 For each combination of:
     image-object, S2 scene, S2 band (10m) and NDVI
         calculate MEAN, STDDEV, MIN and MAX

 2. Feature Tasting
 Feed the results to the feauture_sommelier together with
 ground truth data to taste the usefullness of these features.
 NOTE: The feature_sommelier is implemented in Python. 
       In the future it will be possible to call it directly from IDL
       via the IDL -> Python bridge. For now the feature_sommelier
       has to be used directly within python.

1. Feature Generation

locate shapefile


In [0]:
ref = GET_RELDIR('ROIseries_3D__define',1,['data','sentinel_2a'])
shp = ref+"vector\"+"studyarea.shp"
id_col_name = "Id"

create an array of pathes to Sentinel 2A scenes with known ground truth (cloudy: 'True' or 'False')


In [1]:
ground_truth = READ_CSV(ref+"table\scene_properties.csv")
filenames_csv = (ground_truth.FIELD1)[1:*]
cloudy_csv = (ground_truth.FIELD2)[1:*]
cloudy_knowledge_indices = WHERE((cloudy_csv EQ 'False') OR (cloudy_csv EQ 'True'))
cloudy_knowledge = cloudy_csv[cloudy_knowledge_indices]
rasterseries = ref + "rasters\"+filenames_csv[cloudy_knowledge_indices]

Instantiate ROIseries_3D-objects for individual bands and NDVI and Calculate MEAN, STDDEV, MIN, MAX for each image_object for each time step.


In [2]:
ids = LIST("R","G","B","NIR","NDVI")
indexer_formula = LIST("R[0]","R[1]","R[2]","R[3]","(R[3]-R[0])/(R[3]+R[0])")
bands = LIST()
csv_written = LIST()
FOR i=0,4 DO BEGIN &$
    print,"==================================================================" &$
    print,i &$
    current_object_3D = ROIseries_3D() &$
    temp = current_object_3D.COOKIE_CUTTER(ids[i],FILEPATH(ids[i]),shp,id_col_name,rasterseries,SPECTRAL_INDEXER_FORMULA=indexer_formula[i]) &$
    temp = current_object_3D.TIME_FROM_FILENAMES(rasterseries,[15,4],[19,2],[21,2], POSHOUR=[24,2],POSMINUTE=[26,2],POSSECOND=[28,2]) &$
    current_object_1D = current_object_3D.SPATIAL_MIXER("MEAN") &$
    csv_written.add, current_object_1D.FEATURES_TO_CSV(['RAW']) &$
    current_object_1D = current_object_3D.SPATIAL_MIXER("STDDEV") &$
    csv_written.add, current_object_1D.FEATURES_TO_CSV(['RAW']) &$
    current_object_1D = current_object_3D.SPATIAL_MIXER("MIN") &$
    csv_written.add, current_object_1D.FEATURES_TO_CSV(['RAW']) &$
    current_object_1D = current_object_3D.SPATIAL_MIXER("MAX") &$
    csv_written.add, current_object_1D.FEATURES_TO_CSV(['RAW']) &$
ENDFOR


==================================================================
           0
images loaded
Pumping up
array indices generated
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\R\features\R_features_2017-04-21T15-26-10.12803554534927Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\R\features\R_features_2017-04-21T15-26-10.23702710866943Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\R\features\R_features_2017-04-21T15-26-10.3300461173059Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\R\features\R_features_2017-04-21T15-26-10.40805816650405Z.csv
==================================================================
           1
images loaded
Pumping up
array indices generated
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\G\features\G_features_2017-04-21T15-26-10.59602737426772Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\G\features\G_features_2017-04-21T15-26-10.70505917072311Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\G\features\G_features_2017-04-21T15-26-10.78303098678603Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\G\features\G_features_2017-04-21T15-26-10.87604999542251Z.csv
==================================================================
           2
images loaded
Pumping up
array indices generated
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\B\features\B_features_2017-04-21T15-26-11.07906639575973Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\B\features\B_features_2017-04-21T15-26-11.17305099964156Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\B\features\B_features_2017-04-21T15-26-11.26602977514282Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\B\features\B_features_2017-04-21T15-26-11.36005461215988Z.csv
==================================================================
           3
images loaded
Pumping up
array indices generated
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NIR\features\NIR_features_2017-04-21T15-26-11.54705822467819Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NIR\features\NIR_features_2017-04-21T15-26-11.64104282856002Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NIR\features\NIR_features_2017-04-21T15-26-11.7340618371965Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NIR\features\NIR_features_2017-04-21T15-26-11.82804644107833Z.csv
==================================================================
           4
images loaded
Pumping up
array indices generated
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NDVI\features\NDVI_features_2017-04-21T15-26-12.01505005359664Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NDVI\features\NDVI_features_2017-04-21T15-26-12.10903465747848Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NDVI\features\NDVI_features_2017-04-21T15-26-12.20205366611495Z.csv
Features will be written to: C:\Program Files\Harris\ENVI54\IDL86\NDVI\features\NDVI_features_2017-04-21T15-26-12.29603826999679Z.csv

The list csv_written holds all paths to the CSVs containing the features. Execute the next line and then copy/paste the console output to your python script.


In [3]:
csv_written


[
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\R\\features\\R_features_2017-04-21T15-26-10.12803554534927Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\R\\features\\R_features_2017-04-21T15-26-10.23702710866943Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\R\\features\\R_features_2017-04-21T15-26-10.3300461173059Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\R\\features\\R_features_2017-04-21T15-26-10.40805816650405Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\G\\features\\G_features_2017-04-21T15-26-10.59602737426772Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\G\\features\\G_features_2017-04-21T15-26-10.70505917072311Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\G\\features\\G_features_2017-04-21T15-26-10.78303098678603Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\G\\features\\G_features_2017-04-21T15-26-10.87604999542251Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\B\\features\\B_features_2017-04-21T15-26-11.07906639575973Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\B\\features\\B_features_2017-04-21T15-26-11.17305099964156Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\B\\features\\B_features_2017-04-21T15-26-11.26602977514282Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\B\\features\\B_features_2017-04-21T15-26-11.36005461215988Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NIR\\features\\NIR_features_2017-04-21T15-26-11.54705822467819Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NIR\\features\\NIR_features_2017-04-21T15-26-11.64104282856002Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NIR\\features\\NIR_features_2017-04-21T15-26-11.7340618371965Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NIR\\features\\NIR_features_2017-04-21T15-26-11.82804644107833Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NDVI\\features\\NDVI_features_2017-04-21T15-26-12.01505005359664Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NDVI\\features\\NDVI_features_2017-04-21T15-26-12.10903465747848Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NDVI\\features\\NDVI_features_2017-04-21T15-26-12.20205366611495Z.csv",
    "C:\\Program Files\\Harris\\ENVI54\\IDL86\\NDVI\\features\\NDVI_features_2017-04-21T15-26-12.29603826999679Z.csv"
]