Don't forget to delete the hdmi_out and hdmi_in when finished

Edge Detection Filter Example

In this notebook we will demonstrate a hardware accelerated edge detection filter.

This filter uses the Sobel operator to estimate the gradients of the image in the X and Y directions. The two gradients are then summed displayed onscreen.

For more information on the Sobel operator, consult Wikipedia.

1. Download base overlay to the board

Ensure that the camera is not connected to the board. Run the following script to provide the PYNQ with its base overlay.



In [1]:

    
from pynq.drivers.video import HDMI
from pynq import Bitstream_Part
from pynq.board import Register
from pynq import Overlay

Overlay("demo.bit").download()

2. Connect camera

Physically connect the camera to the HDMI-in port of the PYNQ. Run the following code to instruct the PYNQ to capture the video from the camera and to begin streaming video to your monitor (connected to the HDMI-out port).



In [2]:

    
hdmi_in = HDMI('in')
hdmi_out = HDMI('out', frame_list=hdmi_in.frame_list)
hdmi_out.mode(2)
hdmi_out.start()
hdmi_in.start()

3. Program board with Edge Filter

Run the following script to download the Edge Filter to the PYNQ.



In [3]:

    
Bitstream_Part("sobel_p.bit").download()

4. Create a user interface

We will communicate with the filter using a nice user interface. Run the following code to activate that interface.



In [4]:

    
import ipywidgets as widgets

R0 = Register(0)
R1 = Register(1)
R2 = Register(2)
R3 = Register(3)
R4 = Register(4)

R0.write(128)
R1.write(1)
R2.write(0)
R3.write(hdmi_in.frame_width()//2)
R4.write(0)

R0_s = widgets.IntSlider(
    value=128,
    min=0,
    max=255,
    step=1,
    description='Threshold',
    disabled=False,
    continuous_update=True,
    orientation='vertical',
    readout=True,
    readout_format='i',
    slider_color='red',
    width='80px'
)

R1_s = widgets.IntSlider(
    value=14,
    min=0,
    max=15,
    step=1,
    description='Sensitivity',
    disabled=False,
    continuous_update=True,
    orientation='vertical',
    readout=True,
    readout_format='i',
    slider_color='green',
    width='80px'
)

R2_s = widgets.ToggleButton(
    value=False,
    description='Invert',
    disabled=False,
    #button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Invert',
    icon='check',
    width='120px'
)

R3_s = widgets.IntSlider(
    value=hdmi_in.frame_width()//2,
    min=0,
    max=hdmi_in.frame_width(),
    step=1,
    description='Window',
    disabled=False,
    continuous_update=True,
    orientation='vertical',
    readout=True,
    readout_format='i',
    slider_color='yellow',
    width='80px'
)

R4_s = widgets.ToggleButton(
    value=False,
    description='Rotoscope',
    disabled=False,
    #button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Invert',
    icon='check',
    width='120px'
)

def update_r0(*args):
    R0.write(R0_s.value)
def update_r1(*args):
    R1.write(15 - R1_s.value)
def update_r2(*args):
    R2.write(int(R2_s.value))
def update_r3(*args):
    R3.write(R3_s.value)
def update_r4(*args):
    R4.write(int(R4_s.value))

R0_s.observe(update_r0, 'value')
R1_s.observe(update_r1, 'value')
R2_s.observe(update_r2, 'value')
R3_s.observe(update_r3, 'value')
R4_s.observe(update_r4, 'value')

widgets.VBox([ widgets.HBox( [R0_s, R1_s, R3_s]), widgets.HBox([R2_s, R4_s]) ])

5. Exploration

Feel free to play with the sliders and buttons above. For reference, their functions are as follows:

Threshold - Determines the lower bound of the video output. The higher the threshold, the less noise gets through.
Sensitivity - Adjusts the sensitivity of the filter ().
Window - Allows for the screen to be split between the edge filter and the standard output.
Invert - Inverts the output of the edge filter.
Rotoscope - Superimposes the edges on top of the original image (note that the threshhold will implicitly be 255). Also, the color depth is reduced to give the image a more "cartoonish" look.

Have fun!

5. Clean up

When you are done playing with the edge filter, run the following code to stop the video stream.



In [5]:

    
hdmi_out.stop()
hdmi_in.stop()
del hdmi_out
del hdmi_in