In [ ]:
import os, sys
try:
    from synapse.lib.jupyter import *
except ImportError as e:
    # Insert the root path of the repository to sys.path.
    # This assumes the notebook is located three directories away
    # From the root synapse directory. It may need to be varied
    synroot = os.path.abspath('../../../')
    sys.path.insert(0, synroot)
    from synapse.lib.jupyter import *

In [ ]:
# Create a cortex
core = await getTempCoreCmdr()
.. highlight:: none .. _storm-adv-methods: Storm Reference - Advanced - Methods ==================================== Some of Storm’s :ref:`vars-builtin` support **methods** used to perform various actions on the object represented by the variable. The built-in variables that support methods, along with their methods and examples, are listed below. The built-in :ref:`vars-global-lib` variable is used to access Storm libraries, described in :ref:`storm-adv-libs`. .. NOTE:: In the examples below, the ``$lib.print()`` library function is used to display the value returned when a specific built-in variable or method is called. This is done for illustrative purposes only; ``$lib.print()`` is not required in order to use variables or methods. In some instances we have also included "use-case" examples, where the variable or method is used in one or more sample queries to illustrate possible practical use cases. These represent exemplar Storm queries for how a variable or method might be used in practice. While we have attempted to use relatively simple examples for clarity, some examples may leverage additional Storm features such as subqueries, subquery filters, or flow control elements such as "for" loops or "switch" statements. .. _meth-node: $node ----- :ref:`vars-node-node` is a built-in Storm variable that references **the current node in the Storm query pipeline.** ``$node`` can be used as a variable on its own or with the methods listed below. .. NOTE:: As the ``$node`` variable and related methods reference the current node in the Storm pipeline, the variable and its methods will contain (and return) a null value if the inbound result set is empty (i.e., contains no nodes). **Examples** - Print the value of ``$node`` for an ``inet:dns:a`` node:

In [ ]:
# Make a node
q = '[inet:dns:a=(woot.com,54.173.9.236) .seen=("2016/12/28 20:46:31.000","2016/12/28 20:46:31.001")]'
podes = await core.eval(q, num=1, cmdr=False)

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node)'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)

In [ ]:
# Make a node
q = '[inet:fqdn=aunewsonline.com +#aka.feye.thr.apt1 +#cno.infra.sink.hole.kleissner=(2013/11/26,2016/11/26)]'
podes = await core.eval(q, num=1, cmdr=False)
- Print the value of ``$node`` for an ``inet:fqdn`` node with tags present:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node)'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. NOTE:: The value of ``$node`` is the entire node object and associated properties and tags, as opposed to a specific aspect of the node, such as its iden or primary property value. As demonstrated below, some node constructors can "intelligently" leverage the relevant aspects of the full node object (the value of the ``$node`` variable) when creating new nodes. - Use the ``$node`` variable to create an ``edge:refs`` node showing that a news article references the domain woot[.]com:

In [ ]:
# Make a node
q = '[media:news=(d41d8cd98f00b204e9800998ecf8427e,)]'
podes = await core.eval(q, num=1, cmdr=False)

In [ ]:
# Define and print test query
q = 'media:news=a3759709982377809f28fc0555a38193 [ edge:refs=($node,(inet:fqdn,woot.com)) ]'
print(q)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=2, cmdr=False)
In the example above, the :ref:`meth-node-ndef` method could have been used instead of ``$node`` to create the ``edge:refs`` node. In this case, the node constructor knows to use the ndef from the ``$node`` object to create the node.
- Use the ``$node`` variable to create multiple whois name server records (``inet:whois:recns``) from a set of inbound recent whois record nodes for the domain woot[.]com:

In [ ]:
# Make some nodes
q = '[ (inet:whois:rec=(woot.com,2019/03/02) :text=ns1.differentdomain.com) (inet:whois:rec=(woot.com,2019/06/13) :text=ns1.somedomain.com) (inet:whois:rec=(woot.com,2019/09/12) :text=ns1.somedomain.com)]'
podes = await core.eval(q, num=3, cmdr=False)

In [ ]:
# Define and print test query
q = 'inet:whois:rec:fqdn=woot.com +:asof>=2019/06/13 [ inet:whois:recns=(ns1.somedomain.com,$node) ]'
print(q)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=4, cmdr=False)
In the example above, the :ref:`meth-node-value` method could have been used instead of ``$node`` to create the ``inet:whois:recns`` nodes. In this case, the node constructor knows to use the primary property value from the ``inet:whois:rec`` nodes to create the ``inet:whois:recns`` nodes.
.. _meth-node-data: $node.data ++++++++++ TBD .. _meth-node-data-set: $node.data.set() ~~~~~~~~~~~~~~~~ TBD .. _meth-node-data-get: $node.data.get() ~~~~~~~~~~~~~~~~ TBD .. _meth-node-data-list: $node.data.list() ~~~~~~~~~~~~~~~~~ TBD .. _meth-node-data-pop: $node.data.pop() ~~~~~~~~~~~~~~~~ TBD
.. _meth-node-form: $node.form() ++++++++++++ The ``$node.form()`` method returns the **form** of the current node in the Storm pipeline. The method takes no arguments. **Examples** - Print the form of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.form())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-globtags: $node.globtags() ++++++++++++++++ The ``$node.globtags()`` method returns a **list of string matches from the set of tags applied to the current node** in the Storm pipeline. The method takes a single argument consisting of a wildcard expression for the substring to match. - The argument requires at least one wildcard ( ``*`` ) representing the substring(s) to match. - The method performs an **exclusive match** and returns **only** the matched substring(s), not the entire tag containing the substring match. - The wildcard ( ``*`` ) character can be used to match full or partial tag elements. - Single wildcards are constrained by tag element boundaries (i.e., the dot ( ``.`` ) character. Single wildcards can match an entire tag element or a partial string within an element. - The double wildcard ( ``**`` ) can be used to match across any number of tag elements; that is, the double wildcard is not constrained by the dot boundary. - If the string expression starts with a wildcard, it must be enclosed in quotes in accordance with the use of :ref:`literals`. See :ref:`meth-node-tags` to access full tags (vs. tag substrings). **Examples** - Print the set of top-level (root) tags from any tags applied to the current node:

In [ ]:
# Add some tags to a node
q = 'inet:fqdn=aunewsonline.com [ +#aka.symantec.thr.commentcrew +#cno.threat.t83.tc +#foo.bar.baz +#faz.baz +#foo.derp]'
podes = await core.eval(q, num=1, cmdr=False)

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.globtags("*"))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the list of numbers associated with any threat group tags applied to the current node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.globtags(cno.threat.t*))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
In the example above, ``$node.globtags()`` returns the matching substring only ("83"), which is the portion matching the wildcard; it does not return the "t" character.
- Print the list of organizations and associated threat group names from any third-party alias ("aka") tags applied to the current node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.globtags(aka.*.thr.*))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print all sub-tags for any tags starting with "foo" applied to the current node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.globtags(foo.**))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-iden: $node.iden() ++++++++++++ The ``$node.iden()`` method returns the :ref:`gloss-iden` of the current node in the Storm pipeline. The method takes no arguments. **Examples** - Print the iden of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.iden())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-isform: $node.isform() ++++++++++++++ The ``$node.isform()`` method returns a Boolean value (true / false) for whether the current node in the Storm pipeline is of a specified form. The method takes a single argument of a form name. **Examples** - Print the Boolean value for whether a node is an ``inet:dns:a`` form:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.isform(inet:dns:a))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the Boolean value for whether a node is an ``inet:fqdn`` form:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.isform(inet:fqdn))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-ndef: $node.ndef() ++++++++++++ The ``$node.ndef()`` method returns the :ref:`gloss-ndef` ("node definition") of the current node in the Storm pipeline. The method takes no arguments. **Examples** - Print the ndef of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.ndef())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-repr: $node.repr() ++++++++++++ The ``$node.repr()`` method returns the human-friendly :ref:`gloss-repr` ("representation") of the specified property of the current node in the Storm pipeline. The method can optionally take one argument. - If no arguments are provided, the method returns the repr of the node's primary property value. - If an argument is provided, it should be the string of the secondary property name (i.e., without the leading colon ( ``:`` ) from relative property syntax). - If a universal property string is provided, it must be preceded by the dot / period ( ``.`` ) and enclosed in quotes in accordance with the use of :ref:`literals`. See :ref:`meth-node-value` to return the raw value of a property. **Examples** - Print the repr of the primary property value of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.repr())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the repr of the ``:ipv4`` secondary property value of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.repr(ipv4))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the repr of the ``.seen`` universal property value of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.repr(".seen"))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-tags: $node.tags() ++++++++++++ The ``$node.tags()`` method returns a **list of the tags applied to the current node** in the Storm pipeline. The method can optionally take one argument. - If no arguments are provided, the method returns the full list of all tags applied to the node. - An optional argument consisting of a wildcard string expression can be used to match a subset of tags. - If a string is used with no wildcards, the string must be an exact match for the tag element. - The wildcard ( ``*`` ) character can be used to match full or partial tag elements. - The method performs an **inclusive match** and returns the full tag for all tags that match the provided expression. - Single wildcards are constrained by tag element boundaries (i.e., the dot ( ``.`` ) character). Single wildcards can match an entire tag element or a partial string within an element. - The double wildcard ( ``**`` ) can be used to match across any number of tag elements; that is, the double wildcard is not constrained by the dot boundary. - If the string expression starts with a wildcard, it must be enclosed in quotes in accordance with the use of :ref:`literals`. See :ref:`meth-node-globtags` to access tag substrings (vs. full tags). **Examples** - Print the list of all tags associated with an ``inet:fqdn`` node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.tags())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the tag matching the string "cno" if present on an ``inet:fqdn`` node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.tags(cno))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the list of all tags two elements in length that start with "foo":

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.tags(foo.*))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the list of all tags of any length that start with "f":

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.tags(f**))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
- Print the list of all tags of any length whose first element starts with "a" and whose third element is "thr":

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($node.tags(a*.*.thr.**))'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-node-value: $node.value() +++++++++++++ The ``$node.value()`` method returns the raw value of the primary property of the current node in the Storm pipeline. The method takes no arguments. See :ref:`meth-node-repr` to return the human-friendly value of a property. .. NOTE:: The ``$node.value()`` method is only used to return the primary property value of a node. Secondary property values can be accessed via a user-defined variable (i.e., ``$myvar = :``). **Examples** - Print the value of the primary property value of an ``inet:dns:a`` node:

In [ ]:
# Define and print test query
q = 'inet:dns:a=(woot.com,54.173.9.236) $lib.print($node.value())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. _meth-path: $path ----- :ref:`vars-node-path` is a built-in Storm variable that **references the path of a node as it travels through the pipeline of a Storm query.** The ``$path`` variable is generally not used on its own, but in conjunction with its methods.
.. _meth-path-idens: $path.idens() +++++++++++++ The ``$path.idens()`` method returns the list of idens (:ref:`gloss-iden`) of each node in a node's path through a Storm query. The method takes no arguments. **Examples** - Print the list of iden(s) for the path of a single lifted node:

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com $lib.print($path.idens())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
.. NOTE:: A lift operation contains no pivots (i.e., no "path"), so the method returns only the iden of the lifted node. - Print the list of idens for the path of a single node through two pivots to a single end node:

In [ ]:
# Make some nodes:
q = '[ (inet:dns:a=(aunewsonline.com,67.215.66.149)) (inet:dns:a=(aunewsonline.com,184.168.221.92)) (inet:dns:a=(aunewsonline.com,104.239.213.7)) ]'
podes = await core.eval(q, num=3, cmdr=False)

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com -> inet:dns:a +:ipv4=67.215.66.149 -> inet:ipv4 $lib.print($path.idens())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=1, cmdr=True)
The example above returns the idens of the original ``inet:fqdn`` node, the ``inet:dns:a`` node with the specified IP, and the ``inet:ipv4`` node. - Print the list of idens for the path of a single node through two pivots to three different end nodes (i.e., three paths):

In [ ]:
# Define and print test query
q = 'inet:fqdn=aunewsonline.com -> inet:dns:a -> inet:ipv4 $lib.print($path.idens())'
q1 = '\n'
print(q + q1)
# Execute the query to test it and get the packed nodes (podes).
podes = await core.eval(q, num=3, cmdr=True)
In the example above, the FQDN has three DNS A records, thus there are three different paths that the original node takes through the query.
.. _meth-path-trace: $path.trace() +++++++++++++ TBD

In [ ]:
# Close cortex because done
_ = await core.fini()