DTrace, Python and You

Mark Allen

DTrace is an observability tool; it can observe all the layers of your program, all the way down to the kernel.

• DTrace vs. profiler?
  • DTrace is more of a complement to a profile, not a competitor. DTrace is a more general tool than a domain-specific profiler.
• DTrace vs. debugger?
  • again, it's a complement

DTrace enabled OSes:

• Solaris 10+
• Illumos
  • SmartOS
  • OmniOS
• FreeBSD
• Mac OS X

What about Linux? Nope. Licensing problems. Look at DTrace 4; it has about 40% of the features, but no application insight.

Terms

Probe: ...
Provider: In our case, the python binary is the provider
Consumer: A user-mode pgoram that calls into DTrace; written in a language called D

D Language Overview

awk-like

• define a probe, optional predicate, and optional actions in a braced clause
• supports BEGIN and END block
• local variables (this->foo = 42)
• aggregate/associative variables (prefixed with @)
• one liner support in the form

Example1:

#!/usr/sbin/dtrace -qFZs
# -q: quiet (no extraneous output)
# -F: track trace depth
# -Z: OK if no probes match
# -s: probes I care about...

# DTrace docs for python list all of the hooks
function-entry,
function-return
{
    /* arg1 is subroutine name */
    trace(copyinstr(arg1))
}

Example 2:

#!/usr/sbin/dtrace -qZs

# get the 10 most-called functions

function-entry
{
    /* count sub entries by package and sub name
     * arg0 = source file name
     * arg1 = function name */
    @[strjoin(strjoin(copyinstr(arg0),"-"),copyinstr(arg1))] = count()
}

END
{
    /* give me top 10 higest counts; throw away rest */
    trunc(@, 10)
}

Common aggregation functions

• avg
• count
• lquantize (linear)
• quantize (log - power of 2)
• sum
• min
• max

Normally, you see the average of 1000 executions of your code with a profiler, but you lose the outliers. DTrace, using quantize, gives you a histogram so you can catch the outliers.

DTrace and Python

One of the few languages that's not supported out of the box.

Easiest way to get DTrace support is to use homebrew to build python with DTrace

Available Probes (just some?)

function-entry
function-return
line
(FILE, FUNCTION, LINE)

instance-new-start
instance-new-done
instance-delete-start
instance-delete-done
(CLASS_NAME, FILE)

You can write your own DTrace probes in python using http://tmetsch.github.io/python-dtrace/ (on PyPI, too).

Resources

• http://dtracehol.com/#Intro (hol == hands-on lab; step-by-step lab using virtualbox)
• http://dtrace.org/guide/preface.html
• http://dtracebook.com/index.php/Languages#Python
• http://www.amazon.com/dp/0132091518

Because of Joyent, JavaScript probably has the best DTrace support right now