python-vegindex package command line toolsThe python-vegindex package can be used to extract greeness-index values from PhenoCam Network images. In this notebook we'll download some images and a region-of-interest (ROI) mask list and extract the greeness-index timeseries.
To generate the standard PhenoCam timeseries you'll need images from one of the sites and you'll also need to specify the ROI for those images.
On the PhenoCam Network website you'll find a link to the "Data" page. The page contains a form for downloading image data. To access this page you must first register with the site using a valid e-mail. After registration you can log in to the site and access the Data Page. Fill out the form:
If the data request was successfully processed, a .zip file download will start. Extract the images from the zipfile
into a directory of your choice. For this notebook I've just placed the data in ./phenocamdata/:
phenocamdata/bartlettir/
├── 2016
│ ├── 01
│ │ ├── bartlettir_2016_01_01_113137.jpg
│ │ ├── bartlettir_2016_01_01_114107.jpg
│ │ ├── bartlettir_2016_01_01_115106.jpg
│ │ ├── bartlettir_2016_01_01_120107.jpg
│ │ ├── bartlettir_2016_01_01_121110.jpg
│ │ ├── bartlettir_2016_01_01_122107.jpg
│ │ ├── bartlettir_2016_01_01_123108.jpg
.
.
.
│ ├── bartlettir_2016_12_19_120107.jpg
│ ├── bartlettir_2016_12_19_121107.jpg
│ ├── bartlettir_2016_12_19_122107.jpg
│ ├── bartlettir_2016_12_19_123109.jpg
│ ├── bartlettir_2016_12_19_124108.jpg
│ └── bartlettir_2016_12_19_125106.jpg
├── bartlettir_meta.json
└── bartlettir_meta.txt
The python-vegindex package will expect the data to be in this format.
The other input data we need to use the package is an ROI List file and the associated ROI masks. The list just specifies a date range and mask for the time period covered by the images. If the field-of-view (FOV) of the camera
is stable the list will only need a single entry which can cover the entire time range of the images.
Each ROI page on the PhenoCam Network website (e.g. bartlettir DB_0001 has links for the ROI List and ROI masks in the list. Use those links to download the list and masks. These need to be placed in a sub-directory
(a.k.a. folder) named ROI under the site directory:
phenocamdata/bartlettir/ROI/
├── bartlettir_DB_0001_01.tif
├── bartlettir_DB_0001_02.tif
├── bartlettir_DB_0001_03.tif
├── bartlettir_DB_0001_04.tif
└── bartlettir_DB_0001_roi.csv
In this example we only have data for a single year (2016) but the downloaded ROI list and ROI masks cover data for several years. As long as the ROI list covers the time period for the image data we can generate the time series data. Image data and ROI data for multiple sites can all be placed in the same phenocamdata directory.
Virtual environments attempt to deal with the problem of creating an isolated and reproduceable environment in which to run a software package. There are several ways to deal with virtual environments in python. I'll go over the basics here but a full discussion is beyond the scope of this document.
virtualenv and virtualenvwrapperFor python2 these tools are widely used and you should be able to find installation instructions for your operating system. Once these tools are installed you can do the following:
mkdir test_vegindex
cd test_vegindex/
mkvirtualenv test_vegindex
pip install -U pip
pip install vegindex
You should now have the python-vegindex package installed. Because the vegindex package depends on several other packages you may encounter errors duing installation involving missing dependencies. If this is case you can try installing the Pillow package before installing vegindex:
pip install Pillow
pip install vegindexIf all goes well you should have two new commands available, generate_roi_timeseries and generate_summary_timeseries. Both commands depend on a environment variable to specify the path to the input data.
Depending on your operating system one of the following commands should work:
export PHENOCAM_ARCHIVE_DIR=./phenocamdata
or
set PHENOCAM_ARCHIVE_DIR=./phenocamdatagenerate_roi_timeseriesThe first command will generate summary statistics for the pixels within the ROI mask region for each image in the site directory. The only filter applied to the image is on the total brightness (R + G + B). Images which are too dark or too light will not have statistics calculated.
In [4]:
%%bash
generate_roi_timeseries --help
To actually run the tool you would do something like:
export PHENOCAM_ARCHIVE_DIR='./phenocamdata'
generate_roi_timeseries bartlettir DB_0001
Using the -v option will send the resulting output to the screen as well as to the output file. The output file will be generated in the ROI directory and will have the name (in this case) of bartlettir_DB_0001_timeseries.csv.
In [7]:
%%bash
head -22 ./phenocamdata/bartlettir/ROI/bartlettir_DB_0001_timeseries.csv
generate_summary_timeseriesThe second command reads in the output from generate_roi_timeseries tool and produces 1-day or 3-day summaries of the images taken during this period. These summaries are used to reduce the noise in the greeness signal due to variations in illumination that occur duing the day. These variations are a result of varying sun angle, atmospheric conditions, exposure, etc.
In [8]:
%%bash
generate_summary_timeseries --help
To actually run the tool you would do something like:
export PHENOCAM_ARCHIVE_DIR='./phenocamdata'
generate_summary_timeseries -p 3 bartlettir DB_0001
where we've specified a 3-day time period for aggregation.
In [9]:
%%bash
export PHENOCAM_ARCHIVE_DIR='./phenocamdata'
generate_summary_timeseries -p 3 bartlettir DB_0001;
In [10]:
%%bash
head -25 phenocamdata/bartlettir/ROI/bartlettir_DB_0001_3day.csv
The resulting CSV files are in a format which can be easily plotted or used for further analysis. Of course the data generated here is also available for download from the PhenoCam Network website. So the real utility of the command-line tools is for generating your own ROI list and masks.
I've shown how to use Pandas to plot the data in a separate notebook.
In [15]:
%run Plotting_Output.ipynb
In [ ]:
In [ ]: