See instructions from Google here: https://docs.google.com/document/d/1tvkSGb-49YlSqW3AGknr7T_xoRB1KngCD3f2uiwOS3Q/edit
In [1]:
import sys
import os
import ee
# This script assumes your authentification credentials are stored as operatoring system
# environment variables.
__MY_SERVICE_ACCOUNT = os.environ.get('MY_SERVICE_ACCOUNT')
__MY_PRIVATE_KEY_FILE = os.environ.get('MY_PRIVATE_KEY_FILE')
# Initialize the Earth Engine object, using your authentication credentials.
ee.Initialize()
In [2]:
# Make sure that Python can find the CMT source files
CMT_INSTALL_FOLDER = '/home/smcmich1/repo/earthEngine/CrisisMappingToolkit/'
sys.path.append(CMT_INSTALL_FOLDER)
import cmt.util.evaluation
from cmt.mapclient_qt import centerMap, addToMap
A domain is a geographic location associated with certain sensor images, global data sets, and other supporting files. A domain is described by a custom XML file and can easily be loaded in Python. Once the XML file is loaded all of the associated data can be easily accessed. Note that none of the images are stored locally; instead they have been uploaded to web storage locations where Earth Engine can access them.
In [3]:
import cmt.domain
domainPath = os.path.join(CMT_INSTALL_FOLDER, 'config/domains/modis/kashmore_2010_8.xml')
kashmore_domain = cmt.domain.Domain(domainPath)
In [4]:
import cmt.util.gui_util
cmt.util.gui_util.visualizeDomain(kashmore_domain)
A GUI should appear in a seperate window displaying the domain location. If the GUI does not appear, try restarting the IPython kernel and trying again. This is the default GUI used by the CMT. It is an enhanced version of the GUI provided with the Earth Engine Python API and behaves similarly to the Earth Engine online "playground" interface.
In [5]:
from cmt.modis.flood_algorithms import *
# Select the algorithm to use and then call it
algorithm = DIFFERENCE
(alg, result) = detect_flood(kashmore_domain, algorithm)
# Get a color pre-associated with the algorithm, then draw it on the map
color = get_algorithm_color(algorithm)
addToMap(result.mask(result), {'min': 0, 'max': 1, 'opacity': 0.5, 'palette': '000000, ' + color}, alg, False)
In [6]:
precision, recall, eval_count, quality = cmt.util.evaluation.evaluate_approach(result, kashmore_domain.ground_truth, kashmore_domain.bounds, is_algorithm_fractional(algorithm))
print('For algorithm "%s", precision = %f and recall = %f' % (alg, precision, recall) )
The two main scores for evaluating an algorithm are "precision" and "recall".
In order for these measurements to be computed the domain must have a ground truth file associated with it which labels each pixel as flooded or dry.
The documentation so far covers most of the code used to write a file such as the tool detect_flood_modis.py. The rest of the documentation covers different aspects of the CMT in more detail.
The Crisis Mapping Toolkit has so far been used with the following types of data:
MODIS and LANDSAT data are the easiest types to work with because Earth Engine already has all of that data loaded and easily accessible. SAR data on the other hand can be difficult or expensive to get ahold of.
Most of the processing algorithms currently in CMT are for processing MODIS or SAR data and are split between the modis and radar folders. Some of the algorithms, such as the active contour, can also operate on other types of data.
Instructions for how to load your own data are located in the "Domains" section of this documentation.
The algorithms currently implemented by the CMT fall into these categories:
MODIS
RADAR
Skybox
In addition to the default GUI, the Crisis Mapping Toolkit has another GUI customized to perform a few useful operations. It is accessible by running the "flood_detection_wizard.py" tool. The main map portion of the production GUI is the same as in the default GUI but there are additional controls above and below the map window.
A Domain consists of a region, training information, and a list of descriptions of avialable sensor data. They can be easily loaded from XML files and the existing algorithms are all designed to take domain objects as input. MODIS and DEM data are almost always available in any domain. Instructions for creating a custom domain XML file are in the next section.
To use a custom domain generally requires three files:
For more detailed descriptions of all the possible contents of a domain file, check out the domain_example and sensor_example XML files and all of the real config files that are included with the Crisis Mapping Toolkit.
Here are some examples of code working with the Domain class in Python:
In [ ]:
# Access a specific parameter listed in the domain file
kashmore_domain.algorithm_parameters['modis_diff_threshold']
# Call this function to get whatever digital elevation map is available.
dem = kashmore_domain.get_dem()
# All the sensors included in the domain are stored as a list
first_sensor = kashmore_domain.sensor_list[0]
# If you know the name of a sensor you can access it like this
modis_sensor = kashmore_domain.modis
# Then you can access individual sensor bands like this
one_band = modis_sensor.sur_refl_b03
# To get the EE image object containing all the bands, do this
all_bands = modis.image
# The sensor contains some other information,
# but only if the information is present in the XML files
first_band_name = band_names[0]
first_band_resolution = modis.band_resolutions[first_band_name]
# Related domains have the same structure as the main domain
# and can be accessed like this
kashmore_domain.training_domain
kashmore_domain.unflooded_domain
Earth Engine is very powerful but it is not well suited for all tasks. In these cases you can use the LocalEEImage class to easily download image data from Earth Engine and work with it locally using whatever Python image processing method you prefer. You can see an example of doing this in the Active Contour algorithm.
In [ ]: