In [1]:
import scipy.io
import matplotlib as plt
from NetworKit import *
from segmentation import *
from community import *
This notebooks shows how segment images with NetworKit based on graph community detection algorithms. The communinity detection algorithm are applied on a graph representation of the image, i.e. the 8-neighborhood of an pixel. There are many ways to construct a edgeweight function for those graphs. We depict two ways. Graph g[0] uses the edgeweight function introduced by Jianbo Shi et al. The graphs[1:4] measure the difference in intensity in each color channel (RGB). On each of those graphs a community detection is applied. The resulting image segmentation is the intersection of all those segmentations. The variale gt_data contains the ground-truth data (only within the BSDS500 folder structure)
The Berkeley Segmentation Dataset and Benchmark[1] supplies a comprehensive set of images with ground truth data. Applying a blur filter on the images may improve the results.
[1] http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/
In [2]:
source_path="../../input/segmentation/sample3.png"
has_ground_truth = False
graphs = readImgGraphs(source_path)
if has_ground_truth:
gt_data = readGroundTruth(source_path)
g = graphs[0]
g_channel = graphs[1:4]
In [13]:
img = Image(filename=source_path)
display(img)
The ground-truth data is represented as matlab matrix. With python function is possible to visualize the handmade segmentations. The function matToPartitons generates a NetworKit-partition to the matrix mat. This allows comparing the compare the ground-truth data with the computed segmentation with (dis-)similiarty measures implemented in NetworKit.
In [3]:
if has_ground_truth:
i = 0
gt_img_path = "../../output/gt"+str(i)+".png"
mat = gt_data['groundTruth'].item(0,i)[0][0][0]
plt.image.imsave("../../output/gt"+str(i)+".png", mat)
gt_img = Image(filename=gt_img_path)
display(gt_img)
gt_p = matToPartition(mat)
This paragraph shows the result of the image segmentation algorithm introduced by Pedro F. Felzenwalb. The k parameter regulates between the cluster size and the dissimlarity between two clustern. A large k potentially allows larger clusters.
The function createSegmentation takes a list of graphs. The community detection algorithm is applied on all of those graphs. The results of each segmentation is intersected.
In [17]:
KR = MSTBasedSegmentation(k=300)
KR_M = createSegmentation(source_path, [g], KR)
In [18]:
KR = MSTBasedSegmentation(k=600)
KR_CH = createSegmentation(source_path, g_channel, KR)
The function createSegmentation generates information just like the detectCommunity function. Another way to generate a segmentation is to call the algorithm an to visulaize only the segmentation without any further information.
In [19]:
KR = MSTBasedSegmentation(k=300)
showSegmentation(source_path, g, KR.run(g))
This parahraph shows how to use the graph clustering algorithm PLM to segment images.
In [7]:
plm = PLM(refine=True, gamma=1, maxIter=1)
plm_m = createSegmentation(source_path, [g], plm)
In [8]:
plm = PLM(refine=False, gamma=1, maxIter=64)
plm_ch = createSegmentation(source_path, g_channel, plm)
There is small workarround to combine two community detections algorithms. Segment an image, write the result into an image and segment this again.
In [20]:
KR = MSTBasedSegmentation(600)
KR_CH = createSegmentation(source_path, g_channel, KR)
source_path2 = source_path.replace("input", "output")
writer = ImageGraphWriter()
writer.writePartitionsToSingleImage(g, KR_CH, source_path, source_path2)
plm = PLM(refine=False, gamma=2, maxIter=32)
ng = readImgGraphs(source_path2)
plm_on_top = createSegmentation(source_path2, [ng[0]], plm, "plm")
In [10]:
plm = PLM(refine=True, gamma=1, maxIter=1)
plm_m = createSegmentation(source_path, [g], plm)
source_path2 = source_path.replace("input", "output")
writer = ImageGraphWriter()
writer.writePartitionsToSingleImage(g, KR_CH, source_path, source_path2)
KR = MSTBasedSegmentation(10000)
ng = readImgGraphs(source_path2)
KR_ch = createSegmentation(source_path2, ng[1:4], KR, "mst")
Another way to combine PLM and the Kruskal-algorithm is to use the CombineSegmentation class. This algorithm applies plm on the graph g. The result is used to build a communication graph (there is an edge for each adjecents cluster) for each color channel. As seen before, KR is applied on each channel and the result intersected. If the variable initSize is set to False the initial cluster size for each node in the communication graph is one. Otherwise, the initial cluster size is equal to the cluster size computed by plm.
In [24]:
KR = MSTBasedSegmentation(k=25000, initSize=False)
plm = PLM(refine=False, gamma=1, maxIter = 1)
c = CombineSegmentation(plm, KR)
KR_CH = createSegmentation(source_path, [g], c)
It is difficult to compare the quality of a segmentation by only visual aspects. Therefore, more objectiv ways to measure the quality of a segmentation are necessary.
In [12]:
if has_ground_truth:
i = 0
gt_img_path = "../../output/gt"+str(i)+".png"
mat = gt_data['groundTruth'].item(0,i)[0][0][0]
plt.image.imsave("../../output/gt"+str(i)+".png", mat)
gt_img = Image(filename=gt_img_path)
display(gt_img)
gt_p = matToPartition(mat)
In [19]:
KR = MSTBasedSegmentation(k=30)
KR_M = createSegmentation(source_path, [g], KR)
First of all, NetworKit intern measures like the Jaccard measures are able to compare a segmentation to one ground-truth image. Very similar segmentations tend to have value near 0. Values near 1 indicate two dissimilar segmentation.
In [34]:
if has_ground_truth:
j = JaccardMeasure()
j.getDissimilarity(g,KR_M, gt_p)
By now only one image segmentation specific measure is implemented within the class MeasureRadermacher. The *BCE** measure introduced by David R. Martin (An Empirical Approach to Grouping and Segmentation). This measure is able to compute the dissimilarity between a segmentation and a set of ground-truth segmentations.
In [21]:
if has_ground_truth:
m = BCEMeasures(KR_M, gt_data)
m.bcestar()