overlay_integration


Writing Python for new Overlay

This example will show how to interface to an overlay or hardware library from Python.

In this example, we will assume a new overlay has been created with an accelerator that receives data from Python, processes it, and returns the results.

A command and data will be sent to the accelerator from Python, the accelerator will process the data, return the results to memory, and acknowledge the transaction has completed.

Rather than go through the process or creating a new overlay, for the purposes of this example, the Base overlay will be used to illustrate the process. The IOP1 memory will be used to act like the accelerator memory, although no processing will be carried out on the data.

For this example, we will define the following addresses in the overlay, which are in the IOP1 memory space, and are accessible from Python:

Address Name Memory Location
Accelerator address BASE_ADDRESS 0x40000000
Command Address offset CMD_OFFSET 0x800
Acknowledge Address offset ACK_OFFSET 0x804
Raw Data Address offset RAW_DATA_OFFSET 0x0
Data Address offset DATA_OFFSET 0x400

Assume we only have the following commands for this simple accelerator:

Command Value
Idle 0x0
Process 0x1

Create a new Python module

The pynq MMIO module will be used to read and write to memory, or memory mapped peripherals in the Overlay. First MMIO is imported, and then the new class for this module is defined.

from pynq import MMIO

class my_new_accelerator:
    """Brief description of Module goes here

    Attributes
    ----------
    array_size : int
        Describe  parameters used in this module's functions.
    """

Instantiate the MMIO

Next the MMIO will be instantiated inside the new module.

mmio = MMIO(0x40000000,0x808)
   array_length = 0

Note that a variable, array_length, for this module will also be declared. You will see how this is used later.

Assume that the accelerator will check the command address when it starts.

The Python module must first initialize the command location (BASE_ADDRESS + CMD_OFFSET) to 0x0 ("idle").

Declare an initialization function

Declare the function and write zero to the command location:

def __init__(self, array_size):
      self.mmio.write(CMD_OFFSET, 0)

Define the API

For this example, we will define two functions; load_data() and process().

load_data() will write data to the accelerator memory.

process_data() will send the start command to the accelerator, wait for an acknowledge, and read back the processed data.

  • 0x1 will be written to the command location from Python
  • The accelerator will write 0x1 to the acknowledge location when processing is complete.

Note how the array_length variable is used.

def load_data(self, raw_data):
    self.array_length = len(raw_data)
    for i in range(0 , self.array_length):
        self.mmio.write(RAW_DATA_OFFSET, raw_data[i])

def process(self):     
    # Send start command to accelerator
    self.mmio.write(CMD_OFFSET, 0x1)
    processed_data = [0] *self.array_length
    #ACK is set to check for 0x0 in the ACK offset
    while (self.mmio.read(ACK_OFFSET)) != 0x1:
        pass
    # Ack has been received

    for i in range(0 , self.array_length):
        processed_data[i] = self.mmio.read(PROCESSED_DATA_OFFSET)

    # Reset Ack
    self.mmio.write(ACK_OFFSET, 0)      
    return processed_data

Final code

The complete code can be found below, and can be executed and tested in this notebook by running the cells below. The code could be copied to a python file, and run directly on the board.


In [1]:
BASE_ADDRESS = 0x40000000
CMD_OFFSET = 0x800
ACK_OFFSET = 0x804
RAW_DATA_OFFSET = 0
PROCESSED_DATA_OFFSET = 0x400
        
from pynq import MMIO
  
class my_new_accelerator:
    """Brief description of Module goes here.
    
    Attributes
    ----------
    array_size : int
       Describe  parameters used in this module's functions.
    raw_data : int
       Input Data
    processed_data : int
       Return data
       
   """
    mmio = MMIO(0x40000000,0x808)
    array_length = 0
 
    def __init__(self):
        self.mmio.write(CMD_OFFSET, 0)
     
    def load_data(self, raw_data):
        self.array_length = len(raw_data)
        for i in range(0 , self.array_length):
            self.mmio.write(RAW_DATA_OFFSET, raw_data[i])
            
    def process(self):     
        # Send start command to accelerator
        self.mmio.write(CMD_OFFSET, 0x1)
        processed_data = [0] *self.array_length
        
        # ACK is set to check for 0x0 in the ACK offset
        while (self.mmio.read(ACK_OFFSET)) != 0x1:
            pass
        # Ack has been received

        for i in range(0 , self.array_length):
            processed_data[i] = self.mmio.read(PROCESSED_DATA_OFFSET)
            
        # Reset Ack
        self.mmio.write(ACK_OFFSET, 0)      
        return processed_data

Executing the cell above loads the module into this notebook. This is the equivalent of importing the module (import my_new_accelerator) if it was included as part of the pynq package.

As explained previously, this notebook does not show you how to create a custom accelerator, however, the python code can be tested with the Base overlay. In the Base overlay, the IOP memory (starting at 0x40000000) will be used to simulate writing to an accelerator, and reading back from the accelerator. Notice how the code writes to one area of memory (BASE_ADDRESS + RAW_DATA_OFFSET), and expects to read back results from another area in memory (BASE_ADDRESS + PROCESSED_DATA_OFFSET).

Execute the cell below to load the Pmod overlay, instantiate the accelerator, and send some data to the accelerator.


In [2]:
from pynq import Overlay
Overlay("base.bit").download()

# declare acc with a Maximum allowable array size
acc = my_new_accelerator()
raw_data = [1]*20
print("Some data to be sent to the accelerator:", raw_data)
acc.load_data(raw_data)


Some data to be sent to the accelerator: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

As the accelerator doesn't exist, any data loaded to memory won't be processed, and the acknowledge will not be written.

Execute the cell below to use the MMIO to manually write some data to the results area of the memory to simulate data being processed, and to write 0x1 to the acknowledge address.

The MMIO can be very useful to peak and poke memory and memory mapped peripherals in the overlay to debug Python code.


In [3]:
from pynq import MMIO
       
mmio = MMIO(0x40000000,2056)

for i in range (0,len(raw_data)):
    mmio.write(PROCESSED_DATA_OFFSET, raw_data[i]+1)

for i in range (0,len(raw_data)):
    mmio.write(ACK_OFFSET, 1)

The process() function can now send a start command, read the acknowledge (which has already been set manually in the cell above), and read back from data from the processed data area. You can change the code above to write different data to the processed data area, or to set the acknowlege to 0 (which will cause the code below to hang).


In [4]:
processed_data = acc.process()
print("Input Data     : ", raw_data)
print("Processed Data : ", processed_data)


Input Data     :  [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
Processed Data :  [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]