Accessing C Struct Data

This notebook illustrates the use of @cfunc to connect to data defined in C.

Via CFFI

Numba can map simple C structure types (i.e. with scalar members only) into NumPy structured dtypes.

Let's start with the following C declarations:


In [1]:
from cffi import FFI

src = """

/* Define the C struct */
typedef struct my_struct {
    int    i1;
    float  f2;
    double d3;
    float  af4[7];
} my_struct;

/* Define a callback function */
typedef double (*my_func)(my_struct*, size_t);
"""


ffi = FFI()
ffi.cdef(src)

We can create my_struct data by doing:


In [2]:
# Make a array of 3 my_struct
mydata = ffi.new('my_struct[3]')
ptr = ffi.cast('my_struct*', mydata)
for i in range(3):
    ptr[i].i1 = 123 + i
    ptr[i].f2 = 231 + i
    ptr[i].d3 = 321 + i
    for j in range(7):
        ptr[i].af4[j] = i * 10 + j

Using numba.cffi_support.map_type we can convert the cffi type into a Numba Record type.


In [3]:
from numba import cffi_support

cffi_support.map_type(ffi.typeof('my_struct'), use_record_dtype=True)


Out[3]:
Record(i1[type=int32;offset=0],f2[type=float32;offset=4],d3[type=float64;offset=8],af4[type=nestedarray(float32, (7,));offset=16];48;True)

The function type can be mapped in a signature:


In [4]:
sig = cffi_support.map_type(ffi.typeof('my_func'), use_record_dtype=True)
sig


Out[4]:
(Record(i1[type=int32;offset=0],f2[type=float32;offset=4],d3[type=float64;offset=8],af4[type=nestedarray(float32, (7,));offset=16];48;True)*, uint64) -> float64

and @cfunc can take that signature directly:


In [5]:
from numba import cfunc, carray

@cfunc(sig)
def foo(ptr, n):
    base = carray(ptr, n)  # view pointer as an array of my_struct
    tmp = 0
    for i in range(n):
        tmp += base[i].i1 * base[i].f2 / base[i].d3 + base[i].af4.sum()
    return tmp

Testing the cfunc via the .ctypes callable:


In [6]:
addr = int(ffi.cast('size_t', ptr))
print("address of data:", hex(addr))
result = foo.ctypes(addr, 3)
result


address of data: 0x7fb70a611b30
Out[6]:
541.025912236192

Manually creating a Numba Record type

Sometimes it is useful to create a numba.types.Record type directly. The easiest way is to use the Record.make_c_struct() method. Using this method, the field offsets are calculated from the natural size and alignment of prior fields.

In the example below, we will manually create the my_struct structure from above.


In [7]:
from numba import types


my_struct = types.Record.make_c_struct([
    # Provides a sequence of 2-tuples i.e. (name:str, type:Type)
    ('i1', types.int32),
    ('f2', types.float32),
    ('d3', types.float64),
    ('af4', types.NestedArray(dtype=types.float32, shape=(7,)))
])

my_struct


Out[7]:
Record(i1[type=int32;offset=0],f2[type=float32;offset=4],d3[type=float64;offset=8],af4[type=nestedarray(float32, (7,));offset=16];48;True)

Here's another example to demonstrate the offset calculation:


In [8]:
padded = types.Record.make_c_struct([
    ('i1', types.int32),
    ('pad0', types.int8),    # padding bytes to move the offsets
    ('f2', types.float32),
    ('pad1', types.int8),    # padding bytes to move the offsets
    ('d3', types.float64),
])

padded


Out[8]:
Record(i1[type=int32;offset=0],pad0[type=int8;offset=4],f2[type=float32;offset=8],pad1[type=int8;offset=12],d3[type=float64;offset=16];24;True)

Notice how the byte at pad0 and pad1 moves the offset of f2 and d3.

A function signature can also be created manually:


In [9]:
new_sig = types.float64(types.CPointer(my_struct), types.uintp)
print('signature:', new_sig)
# Our new signature matches the previous auto-generated one.
print('signature matches:', new_sig == sig)


signature: (Record(i1[type=int32;offset=0],f2[type=float32;offset=4],d3[type=float64;offset=8],af4[type=nestedarray(float32, (7,));offset=16];48;True)*, uint64) -> float64
signature matches: True