Image_Duotone_Overlay_Filter


Don't forget to delete the hdmi_out and hdmi_in when finished

Image Overlay Duotone Color Filter Example

In this notebook, we will overlay an image on the output videofeed. By default, an image showing Abraham Lincoln will be displayed at the top of the screen.

In order to store larger images, just two colors are allowed. These colors can be controlled by registers. Each color can also be replaced with "transparency" so that you can see the video feed behind it.

1. Download base overlay to the board

Ensure that the camera is not connected to the board. Run the following script to provide the PYNQ with its base overlay.


In [1]:
from pynq.drivers.video import HDMI
from pynq import Bitstream_Part
from pynq.board import Register
from pynq import Overlay

Overlay("demo.bit").download()

2. Connect camera

Physically connect the camera to the HDMI-in port of the PYNQ. Run the following code to instruct the PYNQ to capture the video from the camera and to begin streaming video to your monitor (connected to the HDMI-out port). The "2" represents a resolution of 1280x720, which is the output streaming resolution of the camera.


In [2]:
hdmi_in = HDMI('in')
hdmi_out = HDMI('out', frame_list=hdmi_in.frame_list)
hdmi_out.mode(2)
hdmi_out.start()
hdmi_in.start()

3. Program board with 2 Color Image Overlay Filter

Run the following script to download the RGB Filter to the PYNQ. This will allow us to modify the colors of the video stream.


In [3]:
Bitstream_Part("img_overlay_duotone_p.bit").download()

4. Create a user interface

We will communicate with the filter using a nice user interface. Run the following code to activate that interface.

7 Registers are used to interact with this particular filter. R0 : Origin X-Coordinate. The origin of the image is the top left corner. Writing to R0 allows you to specify where the image appears horizontally on the feed. R1 : Origin Y-Coordinate. Writing to R1 allows you to specify where the image appears vertically on the feed. R2 : Width. This specifies how wide (in pixels) the image is. R3 : Height. This specifies how tall (in pixels) the image is. R4 : This specifies the index of the first color (the background of the overlay image). Because the color is 24 bits (23:0), the 25th bit (bit 24) signifies whether this color should be transparent or not. R5 : This specifies the index of the second color (the foreground of the overlay image). Like with R4, bit 24 signifies whether this color should be transparent or not. R5 : This is used to write a new image to the filter. 8 pixels are written at a time (1 byte = 8 bits = 8 pixels). The 16-bit pixel address and 1-byte representation of 8 pixels are concatenated and written to this register.

The current minimum and maximum values for the X- and Y-Coordinates as well as image width and height are based on a 1280x720 screen resolution.


In [7]:
import ipywidgets as widgets

from IPython.display import HTML, display

display(HTML('''<style>
    .widget-label { min-width: 15ex !important; }
</style>'''))

R0 =Register(0)
R1 =Register(1)
R2 =Register(2)
R3 =Register(3)
R4 =Register(4)
R5 =Register(5)
R6 =Register(6)

R0.write(1)
R1.write(1)
R2.write(400)
R3.write(449)
R4.write(0xFFFFFF)
R5.write(0x000000)

R0_s = widgets.IntSlider(
    value=1,
    min=1,
    max=1279,
    step=1,
    description='X Origin:',
    disabled=False,
    continuous_update=True,
    orientation='horizontal',
    readout=True,
    readout_format='i',
    slider_color='red',
    width = '600px'
)
R1_s = widgets.IntSlider(
    value=1,
    min=1,
    max=719,
    step=1,
    description='Y Origin:',
    disabled=False,
    continuous_update=True,
    orientation='horizontal',
    readout=True,
    readout_format='i',
    slider_color='green',
    width = '600px'
)
R2_b = widgets.BoundedIntText(
    value=200,
    min=1,
    max=1280,
    step=1,
    description='Image Width:',
    disabled=True
)
R3_b = widgets.BoundedIntText(
    value=200,
    min=1,
    max=720,
    step=1,
    description='Image Height:',
    disabled=True
)
R4_b = widgets.ColorPicker(
    concise=False,
    description='Color 1:',
    value='white'
)
R4_c = widgets.Checkbox(
    value=False,
    description='Color 1 Clear',
    disabled=False
)
R5_b = widgets.ColorPicker(
    concise=False,
    description='Color 2:',
    value='black'
)
R5_c = widgets.Checkbox(
    value=False,
    description='Color 2 Clear',
    disabled=False
)
R6_s = widgets.Select(
    options=['Abe Lincoln', 'Emma Watson', 'Giraffe'],
     value='Abe Lincoln',
    description='Display Image:',
    disabled=False,
)

def update_r0(*args):
    R0.write(R0_s.value)
R0_s.observe(update_r0, 'value')
def update_r1(*args):
    R1.write(R1_s.value)
R1_s.observe(update_r1, 'value')
def update_r2(*args):
    R2.write(R2_b.value)
R2_b.observe(update_r2, 'value')
def update_r3(*args):
    R3.write(R3_b.value)
R3_b.observe(update_r3, 'value')
def update_r4(*args):
    if R4_b.value == "black":
        color1 = 0x000000
    elif R4_b.value == "blue":
        color1 = 0x0000FF
    elif R4_b.value == "brown":
        color1 = 0x996633
    elif R4_b.value == "cyan":
        color1 = 0x00FFFF
    elif R4_b.value == "green":
        color1 = 0x00FF00
    elif R4_b.value == "magenta":
        color1 = 0xFF00FF
    elif R4_b.value == "orange":
        color1 = 0xFF8000
    elif R4_b.value == "purple":
        color1 = 0x800080
    elif R4_b.value == "red":
        color1 = 0xFF0000
    elif R4_b.value == "yellow":
        color1 = 0xFFFF00
    elif R4_b.value == "white":
        color1 = 0xFFFFFF
    else:
        color1 = int(R4_b.value[1:],16)
    if R4_c.value == True:
        color1 |= 0x1000000
    else:
        color1 &= 0xFFFFFF
    R4.write(color1)
R4_b.observe(update_r4, 'value')
R4_c.observe(update_r4, 'value')
def update_r5(*args):
    if R5_b.value == "black":
        color2 = 0x000000
    elif R5_b.value == "blue":
        color2 = 0x0000FF
    elif R5_b.value == "brown":
        color2 = 0x996633
    elif R5_b.value == "cyan":
        color2 = 0x00FFFF
    elif R5_b.value == "green":
        color2 = 0x00FF00
    elif R5_b.value == "magenta":
        color2 = 0xFF00FF
    elif R5_b.value == "orange":
        color2 = 0xFF8000
    elif R5_b.value == "purple":
        color2 = 0x800080
    elif R5_b.value == "red":
        color2 = 0xFF0000
    elif R5_b.value == "yellow":
        color2 = 0xFFFF00
    elif R5_b.value == "white":
        color2 = 0xFFFFFF
    else:
        color2 = int(R5_b.value[1:],16)
    if R5_c.value == True:
        color2 |= 0x1000000
    else:
        color2 &= 0xFFFFFF
    R5.write(color2)
R5_b.observe(update_r5, 'value')
R5_c.observe(update_r5, 'value')
def update_r6(*args):
    filename = "nofile.bin"
    if R6_s.value == 'Abe Lincoln':
        filename = "./data/lincoln.bin"
    elif R6_s.value == 'Emma Watson':
        filename = "./data/emma_watson.bin"
    elif R6_s.value == 'Giraffe':
        filename = "./data/giraffe.bin"
    with open(filename, "rb") as f:
        w = f.read(2)
        h = f.read(2)
        width = (w[1] << 8) | w[0]
        height = (h[1] << 8) | h[0]
        R2.write(width)
        R3.write(height)
        num_pixels = (width*height)/8
        for i in range(0, int(num_pixels)-1):
            byte = f.read(1)
            b = int('{:08b}'.format(byte[0])[::-1], 2)
            b = byte[0];
            x = (i<<8) | b;
            R6.write(x);
R6_s.observe(update_r6, 'value')
widgets.VBox([R0_s,R1_s,R4_b,R4_c,R5_b,R5_c,R6_s])


5. Exploration

  1. Image Position. Try moving the sliders back and forth. Moving the X Origin slider right should move the image to the right. Moving the Y Origin slider right should move the image down.

  2. Colors. Click on the color box for Color 1 and browse to change the color. You can also enter the hex representation of the color in the text box. You can find the codes here: http://www.rapidtables.com/web/color/RGB_Color.htm. Do the same for Color 2.

  3. Transparency. Check the Color 1 Clear box. All pixels of the image overlay background should become transparent. Uncheck the box and check the Color 2 Clear box. Now the foreground should become transparent.

  4. Upload New Image. Try selecting a different image. The new image file should be written to replace the previous image.

6. Clean up

When you are done experimenting with the filter, run the following code to stop the video stream.


In [5]:
hdmi_out.stop()
hdmi_in.stop()
del hdmi_out
del hdmi_in

In [ ]: