Overview:
Annotation studies often focus on small regions of interest (ROIs) that are orders of magnitude smaller than whole slide images and sparsely distributed over many slides. Reviewing these annotations involves significant time spent navigating from one ROI to another within and across slides. To aid in review we developed tools to generate mosaic gallery images that condense these ROIs into a dense multiresolution images that can be viewed in HistomicsUI. These gallery images speed up the review process by minimizing navigation and the need for toggling annotations.
In this minimal example, we show how 29 ROIs from two slides are parsed into three gallery images for pathologist review (a typical project may contain 100s of ROIs). This video demonstrates the gallery image functionality, and the code below which shows the gallery image creation process.
Where to look:
|_histomicstk/
|_annotations_and_masks/
|_review_gallery.py
|_tests/
|_test_review_gallery.py
In [1]:
import os
import tempfile
import shutil
from imageio import imread
from pandas import read_csv
import girder_client
from histomicstk.annotations_and_masks.review_gallery import \
get_all_rois_from_folder_v2, get_all_rois_from_slide_v2, \
_plot_rapid_review_vis, create_review_galleries
import matplotlib.pylab as plt
%matplotlib inline
A csv file like the one in
histomicstk/annotations_and_masks/tests/test_files/sample_GTcodes_v2.csv is needed to define what group each pixel value corresponds to in the mask image, to define the overlay order of various annotation groups, and which groups are considered to be ROIs. Note that the term "group" here comes from the annotation model where each group represents a class like "tumor" or "necrosis" and is associated with a an annotation style.
Please refer to "Converting annotations to semantic segmentation mask images" for more details.
In [ ]:
URL = 'http://candygram.neurology.emory.edu:8080/'
APIURL = URL + 'api/v1/'
# source folder containing slides with annotated ROIs
SAMPLE_FOLDER_ID = '5e2a2da8ddda5f83986d18a2'
# This is the girder folder where galleries will be visualized
POST_FOLDERID = "5e3ce440ddda5f839875b33e"
# Connect to an authenticated girder API. You
gc = girder_client.GirderClient(apiUrl=APIURL)
gc.authenticate(interactive=True) # need this to post!
# gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb')
# GT codes dict for parsing into label mask
GTCODE_PATH = os.path.join(
'/home/mtageld/Desktop/HistomicsTK/histomicstk/annotations_and_masks/',
'tests/test_files', 'sample_GTcodes_v2.csv')
GTCodes_dict = read_csv(GTCODE_PATH)
GTCodes_dict.index = GTCodes_dict.loc[:, 'group']
GTCodes_dict = GTCodes_dict.to_dict(orient='index')
# just a temp directory to save masks
BASE_SAVEPATH = tempfile.mkdtemp()
SAVEPATHS = {
'contours': os.path.join(BASE_SAVEPATH, 'contours'),
'rgb': os.path.join(BASE_SAVEPATH, 'rgbs'),
'visualization': os.path.join(BASE_SAVEPATH, 'vis'),
}
for _, savepath in SAVEPATHS.items():
os.mkdir(savepath)
# where to save gallery
combinedvis_savepath = os.path.join(BASE_SAVEPATH, 'combinedvis')
os.mkdir(combinedvis_savepath)
Let's take a look at some of the inputs
In [3]:
print(list(GTCodes_dict.keys()))
print(GTCodes_dict['tumor'])
In [4]:
print(SAVEPATHS)
print('combinedvis_savepath:', combinedvis_savepath)
The first step is to retrieve annotations and images of the ROIs from the server and store these locally in the temporary directory we created. The ROI images and annotations will be combined to form new images that embed the annotations and generate visualizations. Later, another method will be used to combine these ROIs into a gallery image and to post it to DSA.
In [5]:
print(get_all_rois_from_folder_v2.__doc__)
This calls the method get_all_rois_from_slide_v2() to get the rois for each
individual slide. We don't need the masks here, only the contours and
visualization.
In [6]:
print(get_all_rois_from_slide_v2.__doc__)
We will be get_all_rois_from_folder_v2() will the callback function
_plot_rapid_review_vis() to create a "combined" side-by-side visualization
of the annotations and the RGB image, along with a lower magnification RGB image that provides context for the ROI.
In [7]:
print(_plot_rapid_review_vis.__doc__)
In [8]:
# params for getting all rois for slide
get_all_rois_kwargs = {
'GTCodes_dict': GTCodes_dict,
'save_directories': SAVEPATHS,
'annotations_to_contours_kwargs': {
'MPP': 0.2,
'linewidth': 0.2,
'get_rgb': True,
'get_visualization': True,
},
'verbose': False,
'get_mask': False,
# we use this callback so that we have results compatible
# of being used as input for create_review_galleries()
'callback': _plot_rapid_review_vis,
'callback_kwargs': {
'combinedvis_savepath': combinedvis_savepath,
'zoomout': 4,
},
}
Execute this function using the girder client and folder id.
In [9]:
# Get al rois to prep for gallery
get_all_rois_from_folder_v2(
gc=gc, folderid=SAMPLE_FOLDER_ID,
get_all_rois_kwargs=get_all_rois_kwargs, monitor='test')
In [10]:
all_fovs = os.listdir(combinedvis_savepath)
for i in range(3):
cvis = imread(os.path.join(combinedvis_savepath, all_fovs[i]))
plt.imshow(cvis)
plt.title(all_fovs[i], fontsize=6)
plt.show()
After the visualizations are created we need to assemble them into larger gallery images for review and post these back to the server.
This is the method we will be using:
In [11]:
print(create_review_galleries.__doc__)
In [12]:
create_review_galleries_kwargs = {
'tilepath_base': combinedvis_savepath,
'upload_results': True,
'gc': gc,
'url': URL,
'gallery_folderid': POST_FOLDERID,
'gallery_savepath': None,
'padding': 25,
'tiles_per_row': 2,
'tiles_per_column': 5,
}
In [13]:
# create (+/- post) review gallery
resps = create_review_galleries(**create_review_galleries_kwargs)
Now you can go to the girder folder where galleries will be visualized on HistomicsUI.
The contents of the temporary directory are no longer needed after posting to the server.
In [14]:
shutil.rmtree(BASE_SAVEPATH)