Overview:
This notebook describes how to merge annotations that are generated in piecewise when annotating a large structure, or that arise in an annotation study when one user adds annotations to another user's work as corrections. In these cases there is a collection of annotations that overlap and need to be merged without any regular or predictable interfaces.
The example presented below addresses this case using an R-tree algorithm that identifies merging candidates without exhuastive search. While this approach can also merge annotations generated by tiled analysis it is slower than the alternative.
This extends on some of the work described in Amgad et al, 2019:
Mohamed Amgad, Habiba Elfandy, Hagar Hussein, ..., Jonathan Beezley, Deepak R Chittajallu, David Manthey, David A Gutman, Lee A D Cooper, Structured crowdsourcing enables convolutional segmentation of histology images, Bioinformatics, 2019, btz083
Here is a sample result:
In [ ]:
Implementation summary
This algorithm merges annotations in coordinate space, which means it can merge very large structures without encountering memory issues. The algorithm works as follows:
Identify contours that that have the same label (e.g. tumor)
Add bounding boxes from these contours to an R-tree. The R-tree implementation used here is modified from here and uses k-means clustering to balance the tree.
Starting from the bottom of the tree, merge all contours from leafs that belong to the same nodes.
Move one level up the hierarchy, each time incorporating merged contours from nodes that share a common parent. This is repeated until there is one merged contour at the root node. The contours are first dilated slightly to make sure any small gaps are filled in the merged result, then are eroded by the same factor after merging.
Save the coordinates from each merged polygon in a new pandas DataFrame.
This process ensures that the number of comparisons is << n^2. This is very important since algorithm complexity plays a key role as whole slide images may contain tens of thousands of annotated structures.
Where to look?
|_ histomicstk/
|_annotations_and_masks/
|_polygon_merger_v2.py
|_tests/
|_ test_polygon_merger.py
In [1]:
import os
import sys
CWD = os.getcwd()
sys.path.append(os.path.join(CWD, '..', '..', 'histomicstk', 'annotations_and_masks'))
import girder_client
from histomicstk.annotations_and_masks.polygon_merger_v2 import Polygon_merger_v2
from histomicstk.annotations_and_masks.masks_to_annotations_handler import (
get_annotation_documents_from_contours, _discard_nonenclosed_background_group)
from histomicstk.annotations_and_masks.annotation_and_mask_utils import parse_slide_annotations_into_tables
In [2]:
## 1. Connect girder client and set parameters
In [3]:
APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/'
SOURCE_SLIDE_ID = '5d5d6910bd4404c6b1f3d893'
POST_SLIDE_ID = '5d586d76bd4404c6b1f286ae'
gc = girder_client.GirderClient(apiUrl=APIURL)
# gc.authenticate(interactive=True)
gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb')
# get and parse slide annotations into dataframe
slide_annotations = gc.get('/annotation/item/' + SOURCE_SLIDE_ID)
_, contours_df = parse_slide_annotations_into_tables(slide_annotations)
In [2]:
print(Polygon_merger_v2.__doc__)
In [5]:
print(Polygon_merger_v2.__init__.__doc__)
In [6]:
contours_df.head()
Out[6]:
In [7]:
# init & run polygon merger
pm = Polygon_merger_v2(contours_df, verbose=1)
pm.unique_groups.remove("roi")
pm.run()
NOTE:
The following steps are only "aesthetic", and just ensure the contours look nice when posted to Digital Slide Archive for viewing with GeoJS.
In [8]:
# add colors (aesthetic)
for group in pm.unique_groups:
cs = contours_df.loc[contours_df.loc[:, "group"] == group, "color"]
pm.new_contours.loc[
pm.new_contours.loc[:, "group"] == group, "color"] = cs.iloc[0]
# get rid of nonenclosed stroma (aesthetic)
pm.new_contours = _discard_nonenclosed_background_group(
pm.new_contours, background_group="mostly_stroma")
In [9]:
pm.new_contours.head()
Out[9]:
In [11]:
# deleting existing annotations in target slide (if any)
existing_annotations = gc.get('/annotation/item/' + POST_SLIDE_ID)
for ann in existing_annotations:
gc.delete('/annotation/%s' % ann['_id'])
# get list of annotation documents
annotation_docs = get_annotation_documents_from_contours(
pm.new_contours.copy(), separate_docs_by_group=True,
docnamePrefix='test',
verbose=False, monitorPrefix=POST_SLIDE_ID + ": annotation docs")
# post annotations to slide -- make sure it posts without errors
for annotation_doc in annotation_docs:
resp = gc.post(
"/annotation?itemId=" + POST_SLIDE_ID, json=annotation_doc)
Now you can go to HistomicsUI and confirm that the posted annotations make sense and correspond to tissue boundaries and expected labels.