Don't forget to delete the hdmi_out and hdmi_in when finished

Image Overlay 256 Color Filter Example

In this notebook, we will overlay an image on the output videofeed. By default, an image showing the BYU logo will be displayed at the top of the screen.

In order to store larger images, a 256 color pallette is used so that each pixel color can be represented as 8 bits instead of the 24 bits used to represent the RGB values in the HDMI standard. The filter maps an 8-bit index to its corresponding 24-bit RGB color. The 256 colors used by this filter use the same mapping as xterm colors, which can be found at the following link: 256 Color Chart

This filter also allows the user to specify a "transparent" color. By specifying the index to represent transparency, it will replace all pixels that use the specified color index with the background video pixels.

1. Download base overlay to the board

Ensure that the camera is not connected to the board. Run the following script to provide the PYNQ with its base overlay.


In [1]:
from pynq.drivers.video import HDMI
from pynq import Bitstream_Part
from pynq.board import Register
from pynq import Overlay

Overlay("demo.bit").download()

2. Connect camera

Physically connect the camera to the HDMI-in port of the PYNQ. Run the following code to instruct the PYNQ to capture the video from the camera and to begin streaming video to your monitor (connected to the HDMI-out port). The "2" represents a resolution of 1280x720, which is the output streaming resolution of the camera.


In [2]:
hdmi_in = HDMI('in')
hdmi_out = HDMI('out', frame_list=hdmi_in.frame_list)
hdmi_out.mode(2)
hdmi_out.start()
hdmi_in.start()

3. Program board with 256 Color Image Overlay Filter

Run the following script to download the 256 color image overlay filter to the PYNQ. This will allow us to add image overlays on top of the video feed.


In [3]:
Bitstream_Part("img_overlay_256color_p.bit").download()

4. Create a user interface

We will communicate with the filter using a nice user interface. Run the following code to activate that interface.

6 Registers are used to interact with this particular filter. R0 : Origin X-Coordinate. The origin of the image is the top left corner. Writing to R0 allows you to specify where the image appears horizontally on the feed. R1 : Origin Y-Coordinate. Writing to R1 allows you to specify where the image appears vertically on the feed. R2 : Width. This specifies how wide (in pixels) the image is. R3 : Height. This specifies how tall (in pixels) the image is. R4 : This specifies the index of the color that should be made transparent. Any pixels with that color index will be made transparent. R5 : This is used to write a new image to the filter. The 16-bit pixel address and 8-bit pixel color index are concatenated and written to this register. The color index will then be written to the BRAMs at the pixel address.

The current minimum and maximum values for the X- and Y-Coordinates as well as image width and height are based on a 1280x720 screen resolution.


In [7]:
import ipywidgets as widgets

from IPython.display import HTML, display

display(HTML('''<style>
    .widget-label { min-width: 25ex !important; }
</style>'''))

R0 =Register(0)
R1 =Register(1)
R2 =Register(2)
R3 =Register(3)
R4 =Register(4)
R5 =Register(5)

R0.write(1)
R1.write(1)
R2.write(200)
R3.write(200)
R4.write(0)

R0_s = widgets.IntSlider(
    value=1,
    min=1,
    max=1279,
    step=1,
    description='X Origin:',
    disabled=False,
    continuous_update=True,
    orientation='vertical',
    readout=True,
    readout_format='i',
    slider_color='red',
    width = '150px'
)
R1_s = widgets.IntSlider(
    value=1,
    min=1,
    max=719,
    step=1,
    description='Y Origin:',
    disabled=False,
    continuous_update=True,
    orientation='vertical',
    readout=True,
    readout_format='i',
    slider_color='green',
    width = '150px'
)
R2_b = widgets.BoundedIntText(
    value=200,
    min=1,
    max=1280,
    step=1,
    description='Image Width:',
    disabled=True
)
R3_b = widgets.BoundedIntText(
    value=200,
    min=1,
    max=720,
    step=1,
    description='Image Height:',
    disabled=True
)
R4_b = widgets.BoundedIntText(
    value=0,
    min=0,
    max=255,
    step=1,
    description='Transparent Color Index:',
    disabled=False,
    width = '225px'
)
R5_s = widgets.Select(
    options=['BYU Medallion', 'BYU Cougar', 'BYU Logo'],
     value='BYU Medallion',
    description='Display Image:',
    disabled=False,
    width = '400px'
)

def update_r0(*args):
    R0.write(R0_s.value)
R0_s.observe(update_r0, 'value')
def update_r1(*args):
    R1.write(R1_s.value)
R1_s.observe(update_r1, 'value')
def update_r2(*args):
    R2.write(R2_b.value)
R2_b.observe(update_r2, 'value')
def update_r3(*args):
    R3.write(R3_b.value)
R3_b.observe(update_r3, 'value')
def update_r4(*args):
    R4.write(R4_b.value)
R4_b.observe(update_r4, 'value')
def update_r5(*args):
    #print("Hello")
    filename = "nofile.bin"
    if R5_s.value == 'BYU Medallion':
        filename = "./data/medallion_256.bin"
    elif R5_s.value == 'BYU Cougar':
        filename = "./data/cougar_256.bin"
    elif R5_s.value == 'BYU Logo':
        filename = "./data/BYU_Stretch_Y.bin"
    with open(filename, "rb") as f:
        width = f.read(1)
        height = f.read(1)
        R2.write(width[0])
        R3.write(height[0])
        num_pixels = width[0]*height[0]-1
        for i in range(0, num_pixels):
            byte = f.read(1)
            x = (i<<8) | byte[0];
            R5.write(x);
R5_s.observe(update_r5, 'value')
widgets.HBox([R0_s,R1_s,R4_b,R5_s])


5. Exploration

  1. Image Position. Try moving the sliders up and down. Moving the X Origin slider up should move the image to the right. Moving the Y Origin slider up should move the image down.

  2. Transparency. Try entering the value '15' in the 'Transparent Color Index' box. Notice that the white pixels in the image have become transparent. Now try entering the value '17' in the box. Notice that the blue pixels in the image have become transparent. Now enter the value '0' so that the black pixels will be transparent for the next steps.

  3. Upload New Image. Try selecting a different image. The new image file should be written to replace the previous image.

6. Clean up

When you are done experimenting with the image overlay filter, run the following code before exiting.


In [28]:
hdmi_out.stop()
hdmi_in.stop()
del hdmi_out
del hdmi_in

In [ ]: