Capturing C-level stdout/stderr with `wurlitzer`

Sometimes in Python you are calling some C code. Sometimes that C code makes calls to printf, or otherwise writes to the stdout/stderr of the process.



In [1]:

    
import ctypes
libc = ctypes.CDLL(None)

try:
    c_stderr_p = ctypes.c_void_p.in_dll(libc, 'stderr')
except ValueError:
    # libc.stdout is has a funny name on OS X
    c_stderr_p = ctypes.c_void_p.in_dll(libc, '__stderrp')


def printf(msg):
    """Call C printf"""
    libc.printf((msg + '\n').encode('utf8'))

def printf_err(msg):
    """Cal C fprintf on stderr"""
    libc.fprintf(c_stderr_p, (msg + '\n').encode('utf8'))

IPython forwards the Python-level sys.stdout and sys.stderr, but it leaves the process-level file descriptors that C code will write to untouched. That means that in a context like this notebook, these functions will print to the terminal, because they are not captured:



In [2]:

    
printf("Hello?")
printf_err("Stderr? Anybody?")

With wurlitzer, we can capture these C-level functions:



In [3]:

    
from wurlitzer import pipes, sys_pipes, STDOUT, PIPE



In [4]:

    
with pipes() as (stdout, stderr):
    printf("Hello, stdout!")
    printf_err("Hello, stderr!")

and redisplay them if we like:



In [5]:

    
import sys
sys.stdout.write(stdout.read())
sys.stderr.write(stderr.read())









    



Hello, stdout!






    



Hello, stderr!

Some tools, such as the IPython kernel for Jupyter, capture the Python-level sys.stdout and sys.stderr and forward them somewhere. In the case of Jupyter, this is over a network socket, so that it ends up in the browser.

If we know that's going on, we can easily hook up the C outputs to the Python-forwarded ones with a single call:



In [6]:

    
import time

with sys_pipes():
    for i in range(5):
        time.sleep(1)
        printf("Hello from C, %i!" % i)









    



Hello from C, 0!
Hello from C, 1!
Hello from C, 2!
Hello from C, 3!
Hello from C, 4!

We can also capture the pipes to any writeable streams, such as a StringIO object:



In [7]:

    
import io

stdout = io.StringIO()
with pipes(stdout=stdout, stderr=STDOUT):
    printf("Hello, stdout!")
    printf_err("Hello, stderr!")

print(stdout.getvalue())









    



Hello, stdout!
Hello, stderr!

IPython extension

You can also enable wurlitzer as an IPython extension, so that it always forwards C-level output during execution:



In [8]:

    
%load_ext wurlitzer



In [9]:

    
for i in range(5):
    time.sleep(1)
    printf("Hello from C, %i!" % i)









    



Hello from C, 0!
Hello from C, 1!
Hello from C, 2!
Hello from C, 3!
Hello from C, 4!

Capturing C-level stdout/stderr with wurlitzer

IPython extension

Capturing C-level stdout/stderr with `wurlitzer`