In [ ]:
# Assignment

For this assignment you will calculate and plot the distribution of the path lengths of a graph. First we will generate the graphs:

In [3]:
import networkx as nx
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
from collections import deque

%matplotlib inline

#random_graph = nx.erdos_renyi_graph(10, 0.6)
random_graph = nx.erdos_renyi_graph(1200, 0.008)
print(nx.info(random_graph))


Name: gnp_random_graph(1200,0.008)
Type: Graph
Number of nodes: 1200
Number of edges: 5733
Average degree:   9.5550

1. Write shortest path algorithm

Let's write an algorithm that will calculate the shortest paths between all the nodes and then place the lengths of those shortest paths in a sequence (such as a list or array).

The first task will be creating a function that calculate the shortest path lengths from a single node to everyone else, like the following:


In [6]:
def all_shortest_path_lengths_from(G, node):      
    """
    BSF implementation copied from MIT Open Courseware Lecture 13, Breadth-First Search (BSF): 
    https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-006-introduction-to-algorithms-fall-2011/lecture-videos/lecture-13-breadth-first-search-bfs
    """
    
    level = {node: 0}
    parent = {node: None}
    i = 1
    frontier = [node]
    
    while frontier:
        next_frontier = []
        for u in frontier:
            for v in G[u]:
                if v not in level:
                    level[v] = i
                    parent[v] = u
                    next_frontier.append(v)
        frontier = next_frontier
        i += 1
    
    return level

def all_shortest_path_lengths_from_deque(G, node):
    """
    BSF implementation adapted from Grokking algorithms, Adity Y. Bhargava. Manning Publications, 2016.
    Uses the deque collection object to store the search queue. 
    
    I added a for-loop to the original implementation in order to properly record the level of each 
    cluster of neighbors.    
    """
    
    search_queue = deque([node])    
    current = deque()
    searched = {node}
    
    level = {node: 0}
    parent = {node: None}
    i = 1
    
    while search_queue:                    
        current = deque()
        for neighbor in search_queue:                                
            for n in G[neighbor]:            
                if not n in searched:                
                    level[n] = i
                    parent[n] = neighbor      
                    current.append(n)                                         
                    searched.add(n)                                            
        search_queue = current            
        node = neighbor            
                        
        i += 1                   
                
    return level

In [37]:
p = all_shortest_path_lengths_from_deque(random_graph, 0)
print(p)


{0: 0, 1: 4, 2: 4, 3: 3, 4: 3, 5: 3, 6: 3, 7: 3, 8: 4, 9: 3, 10: 4, 11: 1, 12: 4, 13: 4, 14: 4, 15: 3, 16: 3, 17: 4, 18: 4, 19: 4, 20: 3, 21: 4, 22: 3, 23: 4, 24: 3, 25: 4, 26: 3, 27: 3, 28: 3, 29: 3, 30: 4, 31: 4, 32: 3, 33: 3, 34: 3, 35: 4, 36: 3, 37: 4, 38: 4, 39: 1, 40: 3, 41: 4, 42: 4, 43: 3, 44: 4, 45: 4, 46: 3, 47: 3, 48: 4, 49: 3, 50: 4, 51: 3, 52: 3, 53: 3, 54: 4, 55: 3, 56: 4, 57: 4, 58: 4, 59: 2, 60: 4, 61: 4, 62: 3, 63: 3, 64: 3, 65: 3, 66: 4, 67: 2, 68: 2, 69: 4, 70: 3, 71: 4, 72: 3, 73: 3, 74: 4, 75: 3, 76: 4, 77: 3, 78: 3, 79: 3, 80: 3, 81: 4, 82: 3, 83: 4, 84: 3, 85: 4, 86: 2, 87: 3, 88: 3, 89: 4, 90: 3, 91: 3, 92: 2, 93: 3, 94: 3, 95: 4, 96: 4, 97: 3, 98: 4, 99: 4, 100: 4, 101: 4, 102: 3, 103: 4, 104: 4, 105: 4, 106: 4, 107: 3, 108: 3, 109: 4, 110: 4, 111: 3, 112: 4, 113: 4, 114: 3, 115: 3, 116: 4, 117: 3, 118: 3, 119: 4, 120: 3, 121: 3, 122: 3, 123: 2, 124: 3, 125: 3, 126: 2, 127: 3, 128: 4, 129: 3, 130: 3, 131: 4, 132: 2, 133: 4, 134: 4, 135: 4, 136: 3, 137: 2, 138: 4, 139: 3, 140: 2, 141: 3, 142: 4, 143: 4, 144: 1, 145: 3, 146: 1, 147: 2, 148: 3, 149: 3, 150: 4, 151: 3, 152: 4, 153: 2, 154: 3, 155: 2, 156: 4, 157: 2, 158: 4, 159: 3, 160: 4, 161: 3, 162: 3, 163: 4, 164: 4, 165: 3, 166: 2, 167: 3, 168: 2, 169: 3, 170: 3, 171: 3, 172: 3, 173: 4, 174: 4, 175: 3, 176: 4, 177: 4, 178: 4, 179: 4, 180: 3, 181: 3, 182: 3, 183: 2, 184: 3, 185: 4, 186: 3, 187: 4, 188: 3, 189: 4, 190: 4, 191: 4, 192: 3, 193: 3, 194: 2, 195: 3, 196: 3, 197: 3, 198: 4, 199: 4, 200: 3, 201: 4, 202: 3, 203: 3, 204: 3, 205: 3, 206: 4, 207: 3, 208: 4, 209: 3, 210: 3, 211: 4, 212: 4, 213: 3, 214: 2, 215: 4, 216: 3, 217: 3, 218: 4, 219: 3, 220: 2, 221: 3, 222: 3, 223: 3, 224: 4, 225: 3, 226: 4, 227: 3, 228: 3, 229: 3, 230: 3, 231: 4, 232: 4, 233: 3, 234: 3, 235: 4, 236: 2, 237: 3, 238: 3, 239: 2, 240: 3, 241: 3, 242: 4, 243: 4, 244: 4, 245: 4, 246: 3, 247: 3, 248: 4, 249: 3, 250: 3, 251: 4, 252: 3, 253: 3, 254: 4, 255: 4, 256: 3, 257: 3, 258: 3, 259: 4, 260: 3, 261: 4, 262: 3, 263: 3, 264: 4, 265: 4, 266: 1, 267: 3, 268: 3, 269: 4, 270: 4, 271: 4, 272: 3, 273: 4, 274: 2, 275: 4, 276: 4, 277: 3, 278: 4, 279: 4, 280: 4, 281: 2, 282: 3, 283: 3, 284: 3, 285: 3, 286: 3, 287: 3, 288: 4, 289: 4, 290: 2, 291: 3, 292: 3, 293: 4, 294: 4, 295: 3, 296: 3, 297: 4, 298: 3, 299: 2, 300: 3, 301: 4, 302: 4, 303: 3, 304: 4, 305: 3, 306: 4, 307: 3, 308: 4, 309: 3, 310: 3, 311: 2, 312: 3, 313: 3, 314: 3, 315: 4, 316: 4, 317: 3, 318: 3, 319: 3, 320: 4, 321: 3, 322: 4, 323: 3, 324: 3, 325: 4, 326: 4, 327: 4, 328: 3, 329: 4, 330: 4, 331: 3, 332: 2, 333: 3, 334: 4, 335: 3, 336: 3, 337: 3, 338: 3, 339: 2, 340: 3, 341: 3, 342: 4, 343: 4, 344: 4, 345: 3, 346: 4, 347: 3, 348: 3, 349: 2, 350: 4, 351: 3, 352: 4, 353: 2, 354: 3, 355: 3, 356: 4, 357: 3, 358: 3, 359: 3, 360: 3, 361: 4, 362: 3, 363: 2, 364: 3, 365: 4, 366: 3, 367: 3, 368: 4, 369: 3, 370: 4, 371: 1, 372: 3, 373: 4, 374: 4, 375: 3, 376: 3, 377: 4, 378: 3, 379: 4, 380: 3, 381: 4, 382: 3, 383: 2, 384: 3, 385: 4, 386: 4, 387: 4, 388: 3, 389: 4, 390: 3, 391: 4, 392: 4, 393: 3, 394: 4, 395: 3, 396: 3, 397: 3, 398: 2, 399: 4, 400: 4, 401: 3, 402: 3, 403: 4, 404: 3, 405: 3, 406: 4, 407: 2, 408: 3, 409: 3, 410: 4, 411: 4, 412: 2, 413: 3, 414: 3, 415: 3, 416: 3, 417: 4, 418: 3, 419: 2, 420: 3, 421: 3, 422: 2, 423: 4, 424: 4, 425: 3, 426: 3, 427: 2, 428: 2, 429: 3, 430: 4, 431: 4, 432: 3, 433: 3, 434: 4, 435: 4, 436: 3, 437: 3, 438: 4, 439: 4, 440: 3, 441: 4, 442: 4, 443: 3, 444: 4, 445: 4, 446: 4, 447: 3, 448: 4, 449: 4, 450: 2, 451: 4, 452: 4, 453: 4, 454: 4, 455: 3, 456: 4, 457: 4, 458: 3, 459: 4, 460: 3, 461: 3, 462: 4, 463: 3, 464: 4, 465: 4, 466: 3, 467: 4, 468: 4, 469: 3, 470: 4, 471: 3, 472: 4, 473: 4, 474: 4, 475: 4, 476: 4, 477: 3, 478: 4, 479: 2, 480: 3, 481: 4, 482: 4, 483: 3, 484: 4, 485: 4, 486: 4, 487: 3, 488: 3, 489: 3, 490: 3, 491: 3, 492: 2, 493: 4, 494: 3, 495: 4, 496: 3, 497: 3, 498: 4, 499: 2, 500: 4, 501: 3, 502: 4, 503: 3, 504: 3, 505: 4, 506: 3, 507: 3, 508: 4, 509: 3, 510: 4, 511: 4, 512: 4, 513: 3, 514: 3, 515: 2, 516: 4, 517: 3, 518: 4, 519: 3, 520: 4, 521: 4, 522: 4, 523: 2, 524: 4, 525: 3, 526: 3, 527: 4, 528: 4, 529: 3, 530: 4, 531: 3, 532: 3, 533: 4, 534: 4, 535: 4, 536: 3, 537: 4, 538: 4, 539: 3, 540: 4, 541: 4, 542: 3, 543: 3, 544: 4, 545: 4, 546: 4, 547: 4, 548: 4, 549: 3, 550: 3, 551: 3, 552: 3, 553: 3, 554: 3, 555: 4, 556: 3, 557: 3, 558: 3, 559: 3, 560: 4, 561: 2, 562: 4, 563: 3, 564: 4, 565: 4, 566: 3, 567: 4, 568: 3, 569: 4, 570: 4, 571: 3, 572: 3, 573: 3, 574: 3, 575: 4, 576: 4, 577: 3, 578: 3, 579: 4, 580: 4, 581: 3, 582: 2, 583: 3, 584: 4, 585: 4, 586: 3, 587: 4, 588: 1, 589: 3, 590: 3, 591: 4, 592: 4, 593: 3, 594: 3, 595: 4, 596: 3, 597: 3, 598: 3, 599: 3, 600: 4, 601: 3, 602: 4, 603: 4, 604: 4, 605: 4, 606: 3, 607: 3, 608: 4, 609: 4, 610: 3, 611: 3, 612: 3, 613: 4, 614: 3, 615: 3, 616: 4, 617: 3, 618: 3, 619: 2, 620: 4, 621: 3, 622: 4, 623: 3, 624: 4, 625: 3, 626: 4, 627: 3, 628: 2, 629: 4, 630: 4, 631: 3, 632: 3, 633: 2, 634: 4, 635: 3, 636: 3, 637: 3, 638: 4, 639: 3, 640: 3, 641: 3, 642: 3, 643: 4, 644: 4, 645: 3, 646: 4, 647: 3, 648: 4, 649: 4, 650: 4, 651: 3, 652: 4, 653: 3, 654: 3, 655: 4, 656: 3, 657: 3, 658: 4, 659: 4, 660: 4, 661: 3, 662: 3, 663: 3, 664: 3, 665: 4, 666: 4, 667: 4, 668: 3, 669: 3, 670: 3, 671: 3, 672: 3, 673: 5, 674: 3, 675: 3, 676: 3, 677: 2, 678: 4, 679: 4, 680: 3, 681: 4, 682: 3, 683: 3, 684: 4, 685: 4, 686: 3, 687: 4, 688: 3, 689: 4, 690: 3, 691: 3, 692: 4, 693: 4, 694: 3, 695: 3, 696: 4, 697: 4, 698: 3, 699: 2, 700: 3, 701: 3, 702: 3, 703: 4, 704: 3, 705: 3, 706: 4, 707: 4, 708: 2, 709: 4, 710: 3, 711: 3, 712: 2, 713: 3, 714: 3, 715: 3, 716: 4, 717: 4, 718: 4, 719: 3, 720: 3, 721: 4, 722: 4, 723: 4, 724: 3, 725: 3, 726: 3, 727: 4, 728: 4, 729: 3, 730: 1, 731: 3, 732: 3, 733: 4, 734: 3, 735: 2, 736: 4, 737: 4, 738: 3, 739: 1, 740: 4, 741: 4, 742: 3, 743: 3, 744: 3, 745: 3, 746: 3, 747: 2, 748: 3, 749: 3, 750: 3, 751: 3, 752: 4, 753: 4, 754: 3, 755: 3, 756: 3, 757: 4, 758: 4, 759: 4, 760: 2, 761: 2, 762: 4, 763: 4, 764: 3, 765: 3, 766: 2, 767: 3, 768: 3, 769: 3, 770: 3, 771: 3, 772: 4, 773: 3, 774: 3, 775: 3, 776: 3, 777: 3, 778: 3, 779: 4, 780: 3, 781: 3, 782: 2, 783: 2, 784: 3, 785: 3, 786: 3, 787: 4, 788: 4, 789: 4, 790: 4, 791: 3, 792: 3, 793: 4, 794: 3, 795: 3, 796: 3, 797: 3, 798: 4, 799: 4, 800: 3, 801: 4, 802: 4, 803: 3, 804: 4, 805: 1, 806: 4, 807: 3, 808: 3, 809: 3, 810: 3, 811: 3, 812: 3, 813: 4, 814: 4, 815: 2, 816: 3, 817: 2, 818: 2, 819: 4, 820: 2, 821: 3, 822: 3, 823: 3, 824: 4, 825: 2, 826: 3, 827: 4, 828: 2, 829: 3, 830: 3, 831: 3, 832: 4, 833: 4, 834: 3, 835: 3, 836: 3, 837: 3, 838: 4, 839: 3, 840: 4, 841: 4, 842: 4, 843: 3, 844: 3, 845: 3, 846: 4, 847: 3, 848: 1, 849: 3, 850: 3, 851: 3, 852: 4, 853: 3, 854: 4, 855: 4, 856: 3, 857: 4, 858: 3, 859: 3, 860: 3, 861: 3, 862: 4, 863: 4, 864: 3, 865: 3, 866: 3, 867: 4, 868: 3, 869: 4, 870: 3, 871: 4, 872: 2, 873: 4, 874: 3, 875: 3, 876: 3, 877: 3, 878: 4, 879: 4, 880: 3, 881: 4, 882: 3, 883: 2, 884: 3, 885: 3, 886: 3, 887: 3, 888: 4, 889: 3, 890: 3, 891: 4, 892: 4, 893: 4, 894: 3, 895: 4, 896: 3, 897: 3, 898: 3, 899: 4, 900: 4, 901: 3, 902: 4, 903: 4, 904: 3, 905: 2, 906: 3, 907: 4, 908: 3, 909: 4, 910: 4, 911: 4, 912: 4, 913: 4, 914: 3, 915: 4, 916: 2, 917: 3, 918: 4, 919: 4, 920: 3, 921: 3, 922: 3, 923: 4, 924: 4, 925: 4, 926: 3, 927: 3, 928: 3, 929: 2, 930: 4, 931: 3, 932: 2, 933: 2, 934: 3, 935: 3, 936: 4, 937: 4, 938: 4, 939: 4, 940: 3, 941: 3, 942: 4, 943: 3, 944: 3, 945: 4, 946: 4, 947: 4, 948: 3, 949: 3, 950: 3, 951: 3, 952: 3, 953: 2, 954: 3, 955: 3, 956: 3, 957: 4, 958: 4, 959: 4, 960: 3, 961: 2, 962: 4, 963: 3, 964: 4, 965: 2, 966: 4, 967: 4, 968: 2, 969: 4, 970: 2, 971: 4, 972: 3, 973: 3, 974: 3, 975: 4, 976: 4, 977: 3, 978: 3, 979: 4, 980: 2, 981: 3, 982: 3, 983: 3, 984: 4, 985: 4, 986: 2, 987: 3, 988: 3, 989: 3, 990: 4, 991: 4, 992: 3, 993: 3, 994: 3, 995: 3, 996: 4, 997: 4, 998: 4, 999: 4, 1000: 3, 1001: 4, 1002: 3, 1003: 3, 1004: 3, 1005: 3, 1006: 3, 1007: 3, 1008: 4, 1009: 4, 1010: 3, 1011: 4, 1012: 4, 1013: 4, 1014: 4, 1015: 4, 1016: 3, 1017: 3, 1018: 3, 1019: 4, 1020: 4, 1021: 4, 1022: 2, 1023: 4, 1024: 4, 1025: 2, 1026: 4, 1027: 2, 1028: 4, 1029: 3, 1030: 3, 1031: 3, 1032: 3, 1033: 3, 1034: 4, 1035: 3, 1036: 4, 1037: 3, 1038: 3, 1039: 4, 1040: 2, 1041: 3, 1042: 3, 1043: 3, 1044: 3, 1045: 2, 1046: 3, 1047: 4, 1048: 3, 1049: 3, 1050: 4, 1051: 4, 1052: 4, 1053: 4, 1054: 3, 1055: 3, 1056: 2, 1057: 4, 1058: 3, 1059: 3, 1060: 2, 1061: 3, 1062: 4, 1063: 3, 1064: 3, 1065: 3, 1066: 3, 1067: 3, 1068: 4, 1069: 3, 1070: 3, 1071: 3, 1072: 3, 1073: 3, 1074: 2, 1075: 3, 1076: 3, 1077: 4, 1078: 4, 1079: 4, 1080: 4, 1081: 3, 1082: 3, 1083: 4, 1084: 2, 1085: 4, 1086: 3, 1087: 3, 1088: 4, 1089: 3, 1090: 4, 1091: 4, 1092: 4, 1093: 3, 1094: 3, 1095: 3, 1096: 3, 1097: 4, 1098: 3, 1099: 3, 1100: 3, 1101: 4, 1102: 4, 1103: 4, 1104: 3, 1105: 3, 1106: 4, 1107: 3, 1108: 3, 1109: 3, 1110: 4, 1111: 3, 1112: 2, 1113: 3, 1114: 3, 1115: 4, 1116: 3, 1117: 3, 1118: 2, 1119: 3, 1120: 3, 1121: 3, 1122: 4, 1123: 3, 1124: 3, 1125: 4, 1126: 4, 1127: 3, 1128: 4, 1129: 4, 1130: 3, 1131: 4, 1132: 2, 1133: 4, 1134: 3, 1135: 3, 1136: 2, 1137: 3, 1138: 3, 1139: 4, 1140: 3, 1141: 3, 1142: 4, 1143: 3, 1144: 4, 1145: 3, 1146: 4, 1147: 3, 1148: 3, 1149: 3, 1150: 3, 1151: 3, 1152: 3, 1153: 3, 1154: 2, 1155: 3, 1156: 3, 1157: 4, 1158: 3, 1159: 3, 1160: 3, 1161: 3, 1162: 3, 1163: 2, 1164: 4, 1165: 3, 1166: 3, 1167: 3, 1168: 2, 1169: 4, 1170: 3, 1171: 3, 1172: 4, 1173: 2, 1174: 3, 1175: 4, 1176: 3, 1177: 3, 1178: 4, 1179: 3, 1180: 4, 1181: 4, 1182: 4, 1183: 4, 1184: 3, 1185: 4, 1186: 2, 1187: 3, 1188: 3, 1189: 4, 1190: 3, 1191: 4, 1192: 2, 1193: 4, 1194: 3, 1195: 3, 1196: 4, 1197: 3, 1198: 4, 1199: 3}

You need a queue that stores the nodes to visit at the next hop. You pull one from this queue, say i, assign the current distance, and then go through each neighbor of i and see whether we have seen the node before or not. If we have seen it, we can ignore it (we already know the shortest distance). If not, we should put this node to the queue. You may want to have two queues (or lists or sets) for the "current" level and the "next" level of the search.

After getting the single-source shortest path algorithm right, you can apply it to every node in the graph and obtain the list of shortest path length values.


In [78]:
def all_shortest_path_lengths(G):    
    """Creates a list of path lengths starting from each node in the graph."""
    
    paths = deque()
    nodes = deque()
    processed = deque()
    
    for node in G:                
        nodes = all_shortest_path_lengths_from_deque(G, node)
        processed.append(node)              
        for k, v in nodes.items():
            if not k in processed:
                paths.append(v)                         
    return paths

Apply your algorithm to the random_graph.


In [80]:
paths = all_shortest_path_lengths(random_graph)
print(len(paths))


719400

2. Visualizing the results

Now that you have a list of the shortest paths for the graph, make a histogram for it. This can be done with matplotlibs histogram function


In [3]:
%matplotlib inline
import matplotlib.pyplot as plt

In [81]:
data = paths #all_shortest_path_lengths(random_graph)
bins = list(range(0, 20, 2))

plt.title("Histogram of shortest path lengths in random graph")
plt.xlabel("Path lengths")
plt.ylabel("Number of paths")
plt.hist(data, bins, histtype="bar", rwidth=0.8)
plt.show()


Name your notebook: shortest_lastname_firstname.ipynb and submit to Canvas