Visualizing of genetic similarity with Lightning + GraphX

Setup lightning



In [1]:

    
%libraryDependencies += "org.viz.lightning" %% "lightning-scala" % "0.1.6"



In [2]:

    
%update



In [3]:

    
import org.viz.lightning._
import org.apache.spark.graphx._



In [4]:

    
val lgn = Lightning(host="https://lightning-spark-summit.herokuapp.com" )
lgn.enableNotebook()









    









    Out[4]:





org.viz.lightning.Lightning@7cd4a5c1

Load structure similarity data

Public data from http://www.brain-map.org/



In [5]:

    
val source = "/Users/mathisonian/projects/spark-summit/notebooks/data/allen-connectivity.txt"
val g = GraphLoader.edgeListFile(sc, source)









    









    Out[5]:





org.apache.spark.graphx.impl.GraphImpl@36afd257

Show the network (unlabeled)



In [6]:

    
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))

lgn.force(links)









    











    









    Out[6]:





org.viz.lightning.Visualization@7a4932cd

Show the network colored by degree



In [7]:

    
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))
val degrees = g.degrees.sortBy(_._1).collect().map(x => Math.log(x._2))

lgn.force(links, value=degrees, colormap="Lightning")









    











    









    Out[7]:





org.viz.lightning.Visualization@417fda19

Show the network colored by connected components



In [8]:

    
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))
val connectedComponents = g.connectedComponents().vertices.sortBy(_._1).map(_._2.toInt).collect()

lgn.force(links, label=connectedComponents)









    











    









    Out[8]:





org.viz.lightning.Visualization@74ca5c41