Development Examples

This notebook contains Insights Core examples. All of them can be executed as stand alone scripts or loaded and executed together.

Simple Script

The simplest example is a script that uses the datasources, parsers, or combiners Insights Core provides. This one uses the RedhatRelease parser. The only custom component is a rule that makes use of a parser.

#!/usr/bin/env python
from insights.core.plugins import make_response, rule
from insights.parsers.redhat_release import RedhatRelease

@rule(RedhatRelease)
def report(rel):
    if "Fedora" in rel.product:
        return make_response("IS_FEDORA")

if __name__ == "__main__":
    from insights import run
    run(report, print_summary=True)

Notice the main block at the bottom. It allows our scripts to be loaded dynamically as modules or executed directly.

Output

When I run this script on my machine, I get the following output:

└> python -m demo.rules.sample_script
INFO:insights.core.dr:Trying insights.specs.redhat_release
INFO:insights.core.dr:Trying insights.parsers.redhat_release.RedhatRelease
INFO:insights.core.dr:Trying __main__.report


datasource instances:
'insights.specs.redhat_release:'
TextFileProvider("/etc/redhat-release")


parser instances:
'insights.parsers.redhat_release.RedhatRelease:'
<insights.parsers.redhat_release.RedhatRelease object at 0x7fb324db6310>


rule instances:
'__main__.report:'
{'error_key': 'IS_FEDORA', 'type': 'rule'}

Script with Custom Components

Here's a script that defines its own datasource, parser, and rule. It also depends on the RedhatRelease parser provided by Insights Core.

#!/usr/bin/env python
from collections import namedtuple

from insights import get_active_lines, make_response, parser, Parser, rule
from insights.core.spec_factory import SpecFactory
from insights.parsers.redhat_release import RedhatRelease

# define the data source using a `SpecFactory`.
sf = SpecFactory()
hosts_file = sf.simple_file("/etc/hosts", name="hosts_file")

# a parser to convert the data source into a structured python object.
@parser(hosts_file)
class HostParser(Parser):
    Host = namedtuple("Host", ["ip", "host", "aliases"])

    def parse_content(self, content):
        self.hosts = []
        for line in get_active_lines(content):
            # remove inline comments
            line = line.partition("#")[0].strip()

            # break the line into parts
            parts = line.split()
            ip, host = parts[:2]
            aliases = parts[2:]

            self.hosts.append(HostParser.Host(ip, host, aliases))

    def __repr__(self):
        me = self.__class__.__name__
        msg = "%s([" + ", ".join([str(d) for d in self.hosts]) + "])"
        return msg % me

# a rule that uses the custom `HostParser` and the built-in `RedHatRelease`.
@rule(HostParser, RedhatRelease)
def report(hp, rhr):
    if len(hp.hosts) > 1:
        return make_response("ERROR_KEY_TOO_MANY_HOSTS",
                             number=len(hp.hosts),
                             product=rhr.product)

if __name__ == "__main__":
    from insights import run
    run(report, print_summary=True)

Output

It produces:

└> python -m demo.rules.stand_alone
INFO:insights.core.dr:Trying insights.specs.redhat_release
INFO:insights.core.dr:Trying __main__.hosts_file
INFO:insights.core.dr:Trying insights.parsers.redhat_release.RedhatRelease
INFO:insights.core.dr:Trying __main__.HostParser
INFO:insights.core.dr:Trying __main__.report


datasource instances:
'__main__.hosts:'
TextFileProvider("/etc/hosts")

'insights.specs.redhat_release:'
TextFileProvider("/etc/redhat-release")


parser instances:
'__main__.HostParser:'
HostParser([Host(ip='127.0.0.1', host='localhost', aliases=['localhost.localdomain', 'localhost4', 'localhost4.localdomain4']), Host(ip='::1', host='localhost', aliases=['localhost.localdomain', 'localhost6', 'localhost6.localdomain6'])])

'insights.parsers.redhat_release.RedhatRelease:'
<insights.parsers.redhat_release.RedhatRelease object at 0x7ff7cf986b10>


rule instances:
'__main__.report:'
{'error_key': 'ERROR_KEY_TOO_MANY_HOSTS',
 'number': 2,
 'product': 'Fedora',
 'type': 'rule'}

Wildfly/JBoss Script

This script is an example of running against a JBoss/Wildfly installation or a JDR file. It expects the JBOSS_HOME environment variable to be set when run against an installation. You should write your datasources so that they use $JBOSS_HOME if you want them to collect content from an installation or both an installation and a JDR file.

#!/usr/bin/env python
import json
import lxml.etree as ET

from insights import make_response, parser, Parser, run, rule
from insights.core.context import JBossContext, JDRContext
from insights.core.spec_factory import RawFileProvider, SpecFactory

sf = SpecFactory()

# This file only exists in JDR archives, so we set the context to JDRContext.
# It's an xml file, so we set the kind to RawFileProvider, which has a content
# attribute that contains the raw contents of the file as a single string
# instead of a list of lines.
activeconfig = sf.simple_file("sos_strings/wildfly_full-10/configuration.json",
                              context=JDRContext,
                              kind=RawFileProvider,
                              name="activeconfig")

# These files should only be collected when run in a JBoss or JDR context.
# The JBOSS_HOME environment variable must be set when run in JBossContext.
sacfg = sf.simple_file("$JBOSS_HOME/standalone/configuration/standalone.xml",
                       context=[JBossContext, JDRContext],
                       kind=RawFileProvider,
                       name="sacfg")

salog = sf.simple_file("$JBOSS_HOME/standalone/log/server.log",
                       context=[JBossContext, JDRContext],
                       name="salog")

@parser(activeconfig)
class ActiveConfig(Parser):
    def parse_content(self, content):
        doc = json.loads(content)
        self.data = doc["result"]

@parser(sacfg)
class StandAloneXML(Parser):
    def parse_content(self, content):
        self.root = ET.fromstring(content)

@rule(salog, StandAloneXML)
def report(log, cfg):
    return make_response("SOMETHING_HAPPENED")

@rule(StandAloneXML)
def report2(cfg):
    return make_response("SOMETHING_ELSE_HAPPENED")

if __name__ == "__main__":
    run(print_summary=True,
        run_context=JBossContext,
        archive_context=JDRContext)

Output

└> python -m demo.rules.wildfly
INFO:insights.core.dr:Trying __main__.activeconfig
INFO:insights.core.dr:Trying __main__.sacfg
INFO:insights.core.dr:Trying __main__.salog
INFO:insights.core.dr:Trying __main__.StandAloneXML
INFO:insights.core.dr:Trying __main__.ActiveConfig
INFO:insights.core.dr:Trying __main__.report
INFO:insights.core.dr:Trying __main__.report2


datasource instances:
'__main__.sacfg:'
RawFileProvider("/home/csams/Downloads/wildfly-10.1.0.Final/standalone/configuration/standalone.xml")

'__main__.salog:'
TextFileProvider("/home/csams/Downloads/wildfly-10.1.0.Final/standalone/log/server.log")


parser instances:
'__main__.StandAloneXML:'
<__main__.StandAloneXML object at 0x7f5e39da81d0>


rule instances:
'__main__.report:'
{'error_key': 'SOMETHING_HAPPENED', 'type': 'rule'}

'__main__.report2:'
{'error_key': 'SOMETHING_ELSE_HAPPENED', 'type': 'rule'}

Collections of Scripts

Over time you may accumulate a collection of these individual files. You can run them one at a time as shown above, or you can put them into a python package (just a directory with an empty __init__.py in it) and execute python -m insights -p <package> to run them all. The insights module will recursively load and run all components in the package and its sub-packages. This lets you group components together and run them selectively by sub-package or all at the same time. To specify the runtime and archive contexts to use (HostContext and HostArchiveContext are used by default), pass --rc <context> or --ac <context> to the insights module. Here are two examples running collections: one is a "regular" collection, and the other is for JBoss and JDR. To run against an archive, append -- <archive> to the end of the command: python -m insights -p demo.rules --rc JBossContext --ac JDRContext -- my-jdr-archive.zip.

Regular Collection on local host

└> python -m insights -p demo
INFO:insights.core.dr:Trying demo.rules.fobmaster.report
INFO:insights.core.dr:Trying demo.rules.wildfly.activeconfig
INFO:insights.core.dr:Trying demo.rules.wildfly.sacfg
INFO:insights.core.dr:Trying demo.rules.wildfly.salog
INFO:insights.core.dr:Trying demo.rules.stand_alone.hosts_file
INFO:insights.core.dr:Trying insights.specs.redhat_release
INFO:insights.core.dr:Trying demo.rules.wildfly.ActiveConfig
INFO:insights.core.dr:Trying demo.rules.wildfly.StandAloneXML
INFO:insights.core.dr:Trying demo.rules.stand_alone.HostParser
INFO:insights.core.dr:Trying insights.parsers.redhat_release.RedhatRelease
INFO:insights.core.dr:Trying demo.rules.wildfly.report
INFO:insights.core.dr:Trying demo.rules.wildfly.report2
INFO:insights.core.dr:Trying demo.rules.stand_alone.report
INFO:insights.core.dr:Trying demo.rules.sample_script.report


datasource instances:
'demo.rules.stand_alone.hosts_file:'
TextFileProvider("/etc/hosts")

'insights.specs.redhat_release:'
TextFileProvider("/etc/redhat-release")


parser instances:
'demo.rules.stand_alone.HostParser:'
HostParser([Host(ip='127.0.0.1', host='localhost', aliases=['localhost.localdomain', 'localhost4', 'localhost4.localdomain4']), Host(ip='::1', host='localhost', aliases=['localhost.localdomain', 'localhost6', 'localhost6.localdomain6'])])

'insights.parsers.redhat_release.RedhatRelease:'
<insights.parsers.redhat_release.RedhatRelease object at 0x7f29c5827f10>


rule instances:
'demo.rules.fobmaster.report:'
{'error_key': 'GREATER_EQUAL_5', 'foo': 10, 'type': 'rule'}

'demo.rules.sample_script.report:'
{'error_key': 'IS_FEDORA', 'type': 'rule'}

'demo.rules.stand_alone.report:'
{'error_key': 'ERROR_KEY_TOO_MANY_HOSTS',
 'number': 2,
 'product': 'Fedora',
 'type': 'rule'}

 'demo.rules.wildfly.report:'
{'details': "All: ['demo.rules.wildfly.salog', 'demo.rules.wildfly.StandAloneXML'] Any: ",
 'reason': 'MISSING_REQUIREMENTS',
 'rule_fqdn': 'demo.rules.wildfly.report',
 'type': 'skip'}

'demo.rules.wildfly.report2:'
{'details': "All: ['demo.rules.wildfly.StandAloneXML'] Any: ",
 'reason': 'MISSING_REQUIREMENTS',
 'rule_fqdn': 'demo.rules.wildfly.report2',
 'type': 'skip'}

JBoss collection on local host

└> python -m insights -p demo --rc JBossContext
INFO:insights.core.dr:Trying demo.rules.fobmaster.report
INFO:insights.core.dr:Trying demo.rules.wildfly.activeconfig
INFO:insights.core.dr:Trying demo.rules.wildfly.sacfg
INFO:insights.core.dr:Trying demo.rules.wildfly.salog
INFO:insights.core.dr:Trying demo.rules.stand_alone.hosts_file
INFO:insights.core.dr:Trying insights.specs.redhat_release
INFO:insights.core.dr:Trying demo.rules.stand_alone.HostParser
INFO:insights.core.dr:Trying demo.rules.wildfly.ActiveConfig
INFO:insights.core.dr:Trying demo.rules.wildfly.StandAloneXML
INFO:insights.core.dr:Trying insights.parsers.redhat_release.RedhatRelease
INFO:insights.core.dr:Trying demo.rules.wildfly.report
INFO:insights.core.dr:Trying demo.rules.wildfly.report2
INFO:insights.core.dr:Trying demo.rules.stand_alone.report
INFO:insights.core.dr:Trying demo.rules.sample_script.report


datasource instances:
'demo.rules.stand_alone.hosts_file:'
TextFileProvider("/etc/hosts")

'demo.rules.wildfly.sacfg:'
RawFileProvider("/home/csams/Downloads/wildfly-10.1.0.Final/standalone/configuration/standalone.xml")

'demo.rules.wildfly.salog:'
TextFileProvider("/home/csams/Downloads/wildfly-10.1.0.Final/standalone/log/server.log")

'insights.specs.redhat_release:'
TextFileProvider("/etc/redhat-release")


parser instances:
'demo.rules.stand_alone.HostParser:'
HostParser([Host(ip='127.0.0.1', host='localhost', aliases=['localhost.localdomain', 'localhost4', 'localhost4.localdomain4']), Host(ip='::1', host='localhost', aliases=['localhost.localdomain', 'localhost6', 'localhost6.localdomain6'])])

'demo.rules.wildfly.StandAloneXML:'
<demo.rules.wildfly.StandAloneXML object at 0x7f14e1431fd0>

'insights.parsers.redhat_release.RedhatRelease:'
<insights.parsers.redhat_release.RedhatRelease object at 0x7f14e144e190>


rule instances:
'demo.rules.fobmaster.report:'
{'error_key': 'GREATER_EQUAL_5', 'foo': 10, 'type': 'rule'}

'demo.rules.sample_script.report:'
{'error_key': 'IS_FEDORA', 'type': 'rule'}

'demo.rules.stand_alone.report:'
{'error_key': 'ERROR_KEY_TOO_MANY_HOSTS',
 'number': 2,
 'product': 'Fedora',
 'type': 'rule'}

'demo.rules.wildfly.report:'
{'error_key': 'SOMETHING_HAPPENED', 'type': 'rule'}

'demo.rules.wildfly.report2:'
{'error_key': 'SOMETHING_ELSE_HAPPENED', 'type': 'rule'}

JDR Collection

└> python -m insights -p demo --ac JDRContext -- archives/jdr_17-10-15_07-32-43_localhost.zip 
replace /tmp/tmpq_9FIP/jdr_17-10-15_07-32-43_localhost/JBOSS_HOME/standalone/configuration/standalone_xml_history? [y]es, [n]o, [A]ll, [N]one, [r]ename: n
INFO:insights.core.dr:Trying demo.rules.fobmaster.report
INFO:insights.core.dr:Trying demo.rules.wildfly.activeconfig
INFO:insights.core.dr:Trying demo.rules.wildfly.sacfg
INFO:insights.core.dr:Trying demo.rules.wildfly.salog
INFO:insights.core.dr:Trying demo.rules.stand_alone.hosts_file
INFO:insights.core.dr:Trying insights.specs.redhat_release
INFO:insights.core.dr:Trying demo.rules.stand_alone.HostParser
INFO:insights.core.dr:Trying demo.rules.wildfly.ActiveConfig
INFO:insights.core.dr:Trying demo.rules.wildfly.StandAloneXML
INFO:insights.core.dr:Trying insights.parsers.redhat_release.RedhatRelease
INFO:insights.core.dr:Trying demo.rules.wildfly.report
INFO:insights.core.dr:Trying demo.rules.wildfly.report2
INFO:insights.core.dr:Trying demo.rules.stand_alone.report
INFO:insights.core.dr:Trying demo.rules.sample_script.report


datasource instances:
'demo.rules.wildfly.activeconfig:'
RawFileProvider("/tmp/tmpq_9FIP/jdr_17-10-15_07-32-43_localhost/sos_strings/wildfly_full-10/configuration.json")

'demo.rules.wildfly.sacfg:'
RawFileProvider("/tmp/tmpq_9FIP/jdr_17-10-15_07-32-43_localhost/JBOSS_HOME/standalone/configuration/standalone.xml")

'demo.rules.wildfly.salog:'
TextFileProvider("/tmp/tmpq_9FIP/jdr_17-10-15_07-32-43_localhost/JBOSS_HOME/standalone/log/server.log")


parser instances:
'demo.rules.wildfly.ActiveConfig:'
<demo.rules.wildfly.ActiveConfig object at 0x7f3fa546d0d0>

'demo.rules.wildfly.StandAloneXML:'
<demo.rules.wildfly.StandAloneXML object at 0x7f3fa543bc10>


rule instances:
'demo.rules.fobmaster.report:'
{'error_key': 'GREATER_EQUAL_5', 'foo': 10, 'type': 'rule'}

'demo.rules.sample_script.report:'
{'details': "All: ['insights.parsers.redhat_release.RedhatRelease'] Any: ",
 'reason': 'MISSING_REQUIREMENTS',
 'rule_fqdn': 'demo.rules.sample_script.report',
 'type': 'skip'}

 'demo.rules.stand_alone.report:'
{'details': "All: ['demo.rules.stand_alone.HostParser', 'insights.parsers.redhat_release.RedhatRelease'] Any: ",
 'reason': 'MISSING_REQUIREMENTS',
 'rule_fqdn': 'demo.rules.stand_alone.report',
 'type': 'skip'}

'demo.rules.wildfly.report:'
{'error_key': 'SOMETHING_HAPPENED', 'type': 'rule'}

'demo.rules.wildfly.report2:'
{'error_key': 'SOMETHING_ELSE_HAPPENED', 'type': 'rule'}