PCAP to Graph

This notebook demonstrates how short and sweet a workbench python script can be. :)

Here we're using the workbench server to look at a specific case captured by ThreatGlass. The exploited website for this exercise is gold-xxx.net ThreatGlass_Info.

Tools in this Notebook:

Workbench: Open Source Security Framework Workbench GitHub
Bro Network Security Monitor (http://www.bro.org)

More Info:

See PCAP_to_Dataframe for a short notebook on turning this PCAP into a Pandas Dataframe.
See Workbench Demo Notebook for a lot more info on using workbench.

Lets start up the workbench server...

Run the workbench server (from somewhere, for the demo we're just going to start a local one)

$ workbench_server



In [5]:

    
# Lets start to interact with workbench, please note there is NO specific client to workbench,
# Just use the ZeroRPC Python, Node.js, or CLI interfaces.
import zerorpc
c = zerorpc.Client(timeout=120)
c.connect("tcp://127.0.0.1:4242")









    Out[5]:





[None]

Read in the Data

The data is pulled from [ThreatGlass](http://www.threatglass.com/), the exploited website for this exercise is gold-xxx.net [ThreatGlass_Info](http://www.threatglass.com/malicious_urls/141deabbc8741175d9f51559cf4ef3dd?process_date=2014-05-29).



In [6]:

    
# Load in the PCAP file
with open('../data/pcap/gold_xxx.pcap','rb') as f:
    pcap_md5 = c.store_sample(f.read(), 'gold_xxx', 'pcap')



In [7]:

    
# Now give us a HTTP graph of all the activities within that PCAP.
# Workbench also has DNS and CONN graphs, but for now we're just interested in HTTP.
c.work_request('pcap_http_graph', pcap_md5)









    Out[7]:





{'pcap_http_graph': {'md5': 'c8e58ff22b9a8e48838373fbb1692bdd',
  'output': 'go to http://localhost:7474/browser and execute this query "match (s:origin), (t:file), p=allShortestPaths((s)--(t)) return p"'}}

Workbench + Neo4j = Awesome

The HTTP graph has quite a bit of info, but you can see that we've conducted a shortest paths search from all nodes of type 'origin' (any node originating http communications) to any node of type 'file'. So for this use case we're interested in all of the various files that got downloaded from our network tap in the last few minutes.

Wrap Up

Well for this short notebook we used 3 lines of python to go from PCAP file to Neo4j graph. We hope this exercise showed some neato functionality using Workbench, we encourage you to check out the GitHub repository and our other notebooks:

PCAP_to_Dataframe for a short notebook on turning this PCAP into a Pandas Dataframe.
Workbench Demo general introduction to Workbench.
PCAP_DriveBy a detail look at a Web DriveBy from the ThreatGlass repository.
PE File Sim Graph using Neo4j to generate a similarity graph using PE File features.
Generator Pipelines using the client/server streaming generators to demonstrate 'chaining' generators.