

In [1]:

    
# Delete this cell to re-enable tracebacks
import sys
ipython = get_ipython()

def hide_traceback(exc_tuple=None, filename=None, tb_offset=None,
                   exception_only=False, running_compiled_code=False):
    etype, value, tb = sys.exc_info()
    value.__cause__ = None  # suppress chained exceptions
    return ipython._showtraceback(etype, value, ipython.InteractiveTB.get_exception_only(etype, value))

ipython.showtraceback = hide_traceback



In [2]:

    
# JSON output syntax highlighting
from __future__ import print_function
from pygments import highlight
from pygments.lexers import JsonLexer, TextLexer
from pygments.formatters import HtmlFormatter
from IPython.display import display, HTML
from IPython.core.interactiveshell import InteractiveShell

InteractiveShell.ast_node_interactivity = "all"

def json_print(inpt):
    string = str(inpt)
    formatter = HtmlFormatter()
    if string[0] == '{':
        lexer = JsonLexer()
    else:
        lexer = TextLexer()
    return HTML('<style type="text/css">{}</style>{}'.format(
                formatter.get_style_defs('.highlight'),
                highlight(string, lexer, formatter)))

globals()['print'] = json_print

STIX2 Patterns

The Python stix2 library supports STIX 2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX 2 specification. stix2 does not evaluate patterns against STIX 2 content; for that functionality see cti-pattern-matcher.

Patterns in the stix2 library are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.

API Tips

ObservationExpression

Within the STIX 2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets([]) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.

This requirement manifests itself within the stix2 library via ObservationExpression. When creating STIX 2 observation expressions, whenever the current expression is complete, wrap it with ObservationExpression(). This allows the complete pattern expression - no matter its complexity - to be rendered as a proper specification-adhering string. Note: When pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations. While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings.

In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ObservationExpression().

ParentheticalExpression

Do not be confused by the ParentheticalExpression object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ObservationExpression, wrapping completed subcomponent expressions with ParentheticalExpression() if explicit ordering is required. For usage examples with ParentheticalExpression's, see here.

BooleanExpressions vs CompoundObservationExpressions

Be careful to note the difference between these two very similar pattern components.

BooleanExpressions

AndBooleanExpression
OrbooleanExpression

Usage: When the boolean sub-expressions refer to the same root object

Example: [domain-name:value = "www.5z8.info" AND domain-name:resolvess_to_refs[*].value = "'198.51.100.1/32'"]

Rendering: when pattern is rendered, brackets or parenthesis will encapsulate boolean expression

CompoundObservationExpressions

AndObservationExpression
OrObservationExpression

Usage: When the boolean sub-expressions refer to different root objects

Example: [file:name="foo.dll"] AND [process:name = "procfoo"]

Rendering: when pattern is rendered, brackets will encapsulate each boolean sub-expression

Examples

Comparison Expressions



In [3]:

    
from stix2 import DomainName, File, IPv4Address
from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,
                   GreaterThanComparisonExpression, IsSubsetComparisonExpression,
                   FloatConstant, StringConstant)

Equality Comparison expressions



In [4]:

    
lhs = ObjectPath("domain-name", ["value"])
ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, "site.of.interest.zaz"))
print("\t{}\n".format(ece_1))

lhs = ObjectPath("file", ["parent_directory_ref","path"])
ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, "C:\\Windows\\System32"))
print("\t{}\n".format(ece_2))









    Out[4]:




	[domain-name:value = 'site.of.interest.zaz']








    Out[4]:




	[file:parent_directory_ref.path = 'C:\\Windows\\System32']

Greater-than Comparison expressions



In [5]:

    
lhs = ObjectPath("file", ["extensions", "windows-pebinary-ext", "sections[*]", "entropy"])
gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant("7.0")))
print("\t{}\n".format(gte))









    Out[5]:




	[file:extensions.'windows-pebinary-ext'.sections[*].entropy > 7.0]

IsSubset Comparison expressions



In [6]:

    
lhs = ObjectPath("network-traffic", ["dst_ref", "value"])
iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant("2001:0db8:dead:beef:0000:0000:0000:0000/64")))
print("\t{}\n".format(iss))









    Out[6]:




	[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']

Compound Observation Expressions



In [7]:

    
from stix2 import (IntegerConstant, HashConstant, ObjectPath,
                   EqualityComparisonExpression, AndBooleanExpression,
                   OrBooleanExpression, ParentheticalExpression,
                   AndObservationExpression, OrObservationExpression,
                   FollowedByObservationExpression, ObservationExpression)

AND boolean



In [8]:

    
ece3 = EqualityComparisonExpression(ObjectPath("email-message", ["sender_ref", "value"]), "stark@example.com")
ece4 = EqualityComparisonExpression(ObjectPath("email-message", ["subject"]), "Conference Info")
abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))
print("(AND)\n{}\n".format(abe))









    Out[8]:




(AND)
[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']

OR boolean



In [9]:

    
ece5 = EqualityComparisonExpression(ObjectPath("url", ["value"]), "http://example.com/foo")
ece6 = EqualityComparisonExpression(ObjectPath("url", ["value"]), "http://example.com/bar")
obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))
print("(OR)\n{}\n".format(obe))









    Out[9]:




(OR)
[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']

( OR ) AND boolean



In [10]:

    
ece7 = EqualityComparisonExpression(ObjectPath("file", ["name"]), "pdf.exe")
ece8 = EqualityComparisonExpression(ObjectPath("file", ["size"]), IntegerConstant("371712"))
ece9 = EqualityComparisonExpression(ObjectPath("file", ["created"]), "2014-01-13T07:03:17Z")
obe1 = OrBooleanExpression([ece7, ece8])
pobe = ParentheticalExpression(obe1)
abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))
print("(OR,AND)\n{}\n".format(abe1))









    Out[10]:




(OR,AND)
[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]

( AND ) OR ( OR ) observation



In [11]:

    
ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath("file", ["name"]), "foo.dll"))
ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath("win-registry-key", ["key"]), "HKEY_LOCAL_MACHINE\\foo\\bar"))
ece22 = EqualityComparisonExpression(ObjectPath("process", ["name"]), "fooproc")
ece23 = EqualityComparisonExpression(ObjectPath("process", ["name"]), "procfoo")
# NOTE: we need to use AND/OR observation expression instead of just boolean 
# expressions as the operands are not on the same object-type
aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))
obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))
ooe = OrObservationExpression([aoe, obe2])
print("(AND,OR,OR)\n{}\n".format(ooe))









    Out[11]:




(AND,OR,OR)
([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\foo\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']

FOLLOWED-BY



In [12]:

    
ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath("file", ["hashes", "MD5"]), HashConstant("79054025255fb1a26e4bc422aef54eb4", "MD5")))
ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath("win-registry-key", ["key"]), "HKEY_LOCAL_MACHINE\\foo\\bar"))
fbe = FollowedByObservationExpression([ece10, ece11])
print("(FollowedBy)\n{}\n".format(fbe))









    Out[12]:




(FollowedBy)
[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\foo\\bar']

Qualified Observation Expressions



In [13]:

    
from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,
                   AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,
                   QualifiedObservationExpression, FollowedByObservationExpression,
                   ParentheticalExpression, ObservationExpression)

WITHIN



In [14]:

    
ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath("file", ["hashes", "MD5"]), HashConstant("79054025255fb1a26e4bc422aef54eb4", "MD5")))
ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath("win-registry-key", ["key"]), "HKEY_LOCAL_MACHINE\\foo\\bar"))
fbe = FollowedByObservationExpression([ece10, ece11])
par = ParentheticalExpression(fbe)
qoe = QualifiedObservationExpression(par, WithinQualifier(300))
print("(WITHIN)\n{}\n".format(qoe))









    Out[14]:




(WITHIN)
([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\foo\\bar']) WITHIN 300 SECONDS

REPEATS, WITHIN



In [15]:

    
ece12 = EqualityComparisonExpression(ObjectPath("network-traffic", ["dst_ref", "type"]), "domain-name")
ece13 = EqualityComparisonExpression(ObjectPath("network-traffic", ["dst_ref", "value"]), "example.com")
abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))
qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))
print("(REPEAT, WITHIN)\n{}\n".format(qoe1))









    Out[15]:




(REPEAT, WITHIN)
[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS

START, STOP



In [16]:

    
ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath("file", ["name"]), "foo.dll"))
ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))
qoe2 = QualifiedObservationExpression(ece14, ssq)
print("(START-STOP)\n{}\n".format(qoe2))









    Out[16]:




(START-STOP)
[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'

Attaching patterns to STIX2 Domain objects

Example



In [17]:

    
from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression

ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath("file", ["name"]), "$$t00rzch$$.elf"))
ind = Indicator(name="Cryptotorch", pattern_type="stix", pattern=ece14)
print(ind)









    Out[17]:




{
    "type": "indicator",
    "spec_version": "2.1",
    "id": "indicator--30dbe788-c10c-4905-b2f0-8f738c847f4b",
    "created": "2020-06-26T18:45:26.0832Z",
    "modified": "2020-06-26T18:45:26.0832Z",
    "name": "Cryptotorch",
    "pattern": "[file:name = '$$t00rzch$$.elf']",
    "pattern_type": "stix",
    "pattern_version": "2.1",
    "valid_from": "2020-06-26T18:45:26.0832Z"
}