In [0]:
from sklearn.metrics import normalized_mutual_info_score
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
from sklearn.datasets.samples_generator import make_blobs
import numpy as np
make_blobs is used to generate sample data where:
n_samples : the total number of points equally divided among clusters.
centers : the number of centers to generate, or the fixed center locations.
n_features : the number of features for each sample.
random_state: determines random number generation for dataset creation.
This function returns two outputs:
X: the generated samples.
y: The integer labels for cluster membership of each sample.
Then we use plt.scatter to plot the data points in the figure below.
In [0]:
X, y = make_blobs(n_samples=90, centers=4, n_features=3, random_state=4)
plt.scatter(X[:, 0], X[:, 1])
plt.show()
In this part, we are performing agglomerative hierarchical clustering using linkage function from scipy library::
method: is the linkage method, 'single' means the linkage method will be single linkage method.
metric: is our similarity metric, 'euclidean' means the metric will be euclidean distance.
"A (n-1) by 4 matrix Z is returned. At the -th iteration, clusters with indices Z[i, 0] and Z[i, 1] are combined to form cluster with index (n+i) . A cluster with an index less than n corresponds to one of the n original observations. The distance between clusters Z[i, 0] and Z[i, 1] is given by Z[i, 2]. The fourth value Z[i, 3] represents the number of original observations in the newly formed cluster.
The following linkage methods are used to compute the distance d(s,t)between two clusters sand t. The algorithm begins with a forest of clusters that have yet to be used in the hierarchy being formed. When two clusters s and tfrom this forest are combined into a single cluster u, sand t are removed from the forest, and u is added to the forest. When only one cluster remains in the forest, the algorithm stops, and this cluster becomes the root.
A distance matrix is maintained at each iteration. The d[i,j]`` entry corresponds to the distance between clusteriiandj` in the original forest.
At each iteration, the algorithm must update the distance matrix to reflect the distance of the newly formed cluster u with the remaining clusters in the forest."
For more details check the docmentation of linkage: https://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html
In [0]:
Z = linkage(X, method="single", metric="euclidean")
print(Z.shape)
Z
Out[0]:
In [0]:
plt.figure(figsize=(25, 10))
plt.title("Hierarchical Clustering Dendrogram")
plt.xlabel("Samples indexes")
plt.ylabel("distance")
dendrogram(Z, leaf_rotation=90., leaf_font_size=8. )
plt.show()
In [0]:
cluster = fcluster(Z, 2, criterion="distance")
cluster
Out[0]:
In [0]:
plt.figure(figsize=(10, 8))
plt.scatter(X[:, 0], X[:, 1], c=cluster, cmap="Accent")
plt.savefig("clusters.png")
plt.show()
Finally we will use Normalized Mutual Information (NMI) score to evaluate our clusters. Mutual information is a symmetric measure for the degree of dependency between the clustering and the manual classification. When NMI value is close to one, it indicates high similarity between clusters and actual labels. But if it was close to zero, it indicates high dissimilarity between them.
In [0]:
normalized_mutual_info_score(y, cluster)
Out[0]:
In [0]: