Video Codec Unit (VCU) Demo Example: DECODE -> DISPLAY

Introduction

Video Codec Unit (VCU) in ZynqMP SOC is capable of decoding AVC/HEVC compressed video streams.

This notebook example shows File playback usecase - where it reads compressed video and audio(optional) data from file or HTTP URL (e.g. youtube video), does decoding of video content using VCU, decoding of audio using software Gstreamer element and renders output on DP/HDMI display.

Implementation Details

Board Setup

  1. Connect 4k DP/HDMI display to board.
  2. Connect Ethernet cable. Check Internet connectivity. It is required for downloading compressed file from web-server.
  3. Connect serial cable to monitor logs on serial console.
  4. If Board is connected to private network, then export proxy settings in /home/root/.bashrc file as below,
    • create/open a bashrc file using "vi ~/.bashrc"
      • Insert below line to bashrc file
        • export http_proxy="< private network proxy address >"
        • export https_proxy="< private network proxy address >"
      • Save and close bashrc file.
  5. Determine Audio output device names based on requirements. Please refer Determine AUDIO Device Names section.

Determine Audio Device Names

The audio device name of audio source(Input device) and playback device(output device) need to be determined using arecord and aplay utilities installed on platform.

Audio Output

ALSA sound device names for playback devices:

  • Run below command to get ALSA playback device names for output devices

    root@zcu106-zynqmp:~#aplay -l

    It shows list of Playback Hardware Devices. For e.g

     - card 0: monitor [DisplayPort monitor], device 0: (null) xilinx-dp-snd-codec-dai-0 []
         - Subdevices: 1/1
         - Subdevice #0: subdevice #0
     - card 0: monitor [DisplayPort monitor], device 1: (null) xilinx-dp-snd-codec-dai-1 []
         - Subdevices: 1/1
         - Subdevice #0: subdevice #0

Here card number "0" is being used for playback device for display port channel 0 and device id is 0, so "hw:0,0" Hence it can be passed as auido output device.

PulseAudio sound device names playback devices

  • Run below command to get PULSE playback device names for output devices

    root@zcu106-zynqmp:~#pactl list short sinks

    It shows list of Playback Hardware Devices. For e.g

    - 0       alsa_output.platform-fd4a0000.zynqmp-display_zynqmp_dp_snd_card.analog-stereo ...

Here "alsa_output.platform-fd4a0000.zynqmp-display_zynqmp_dp_snd_card.analog-stereo" is the name of audio playback device. Hence it can be passed as auido output device.


In [1]:
from IPython.display import HTML

HTML('''<script>
code_show=true; 
function code_toggle() {
 if (code_show){
 $('div.input').hide();
 } else {
 $('div.input').show();
 }
 code_show = !code_show
} 
$( document ).ready(code_toggle);
</script>
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')


Out[1]:

Run the Demo


In [2]:
from ipywidgets import interact
import ipywidgets as widgets
from common import common_vcu_demo_decode_display
import os
from ipywidgets import HBox, VBox, Text, Layout, HTML

Insert Input file path Or Youtube URL


In [3]:
input_path=widgets.Text(value='',
    placeholder='Insert file path',
    description=r'Input File:',
    #style={'description_width': 'initial'},
    disabled=False)
input_url=widgets.Text(value='',
    placeholder='"Insert Youtube URL"',
    description=r'<b>OR</b> Youtube URL (in quotes):',
    style={'description_width': 'initial'},
    layout=Layout(width='50%', height='30px'),
    #style={'flex container': 'cross-end'},
    disabled=False)
display(HBox([input_path, input_url]))
#display(HBox([input_url, input_path]))



In [4]:
proxy_url=widgets.Text(value='',
    placeholder='(optional)',
    description='Proxy URL:',
    #style={'description_width': 'initial'},
    disabled=False)
proxy_url
#HBox([proxy_url, optional])


Video


In [5]:
codec_type=widgets.RadioButtons(
    options=['avc', 'hevc'],
    description='Video Codec:',
    #style={'description_width': 'initial'},
    disabled=False)

video_sink={'kmssink':['DP', 'HDMI'], 'fakevideosink':['none']}

def print_video_sink(VideoSink):
    pass

def select_video_sink(VideoCodec):
    display_type.options = video_sink[VideoCodec]

sink_name = widgets.RadioButtons(options=sorted(video_sink.keys(), key=lambda k: len(video_sink[k]), reverse=True), description='Video Sink:')

init = sink_name.value

display_type = widgets.RadioButtons(options=video_sink[init], description='Display:')
j = widgets.interactive(print_video_sink, VideoSink=display_type)
i = widgets.interactive(select_video_sink, VideoCodec=sink_name)

HBox([codec_type, i, j])


Audio


In [6]:
audio_output=Text(value='',
    placeholder='(optional) "hw:0"',
    description='Output Dev:',
    style={'flex container': 'cross-end'},
    description_tooltip='To select the values of this argument, please refer Determine Audio Device Names section',
    disabled=False)
audio_output
#HBox([audio_output, optional])



In [7]:
audio_sink={'none':['none'], 'aac':['auto','alsasink','pulsesink'],'vorbis':['auto','alsasink','pulsesink']}
audio_src={'none':['none'], 'aac':['auto','alsasrc','pulsesrc'],'vorbis':['auto','alsasrc','pulsesrc']}

#val=sorted(audio_sink, key = lambda k: (-len(audio_sink[k]), k))
def print_audio_sink(AudioSink):
    pass
    
def print_audio_src(AudioSrc):
    pass

def select_audio_sink(AudioCodec):
    audio_sinkW.options = audio_sink[AudioCodec]
    audio_srcW.options = audio_src[AudioCodec]

audio_codecW = widgets.RadioButtons(options=sorted(audio_sink.keys(), key=lambda k: len(audio_sink[k])), description='Audio Codec:')

init = audio_codecW.value

audio_sinkW = widgets.RadioButtons(options=audio_sink[init], description='Audio Sink:')
audio_srcW = widgets.RadioButtons(options=audio_src[init], description='Audio Src:')
j = widgets.interactive(print_audio_sink, AudioSink=audio_sinkW)
i = widgets.interactive(select_audio_sink, AudioCodec=audio_codecW)

HBox([i, j])


Advanced options:


In [8]:
entropy_buffers=widgets.Dropdown(
    options=['2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15'],
    value='5',
    description='Entropy Buffers Nos:',
    style={'description_width': 'initial'},
    disabled=False,)
show_fps=widgets.Checkbox(
    value=False,
    description='show-fps',
    #style={'description_width': 'initial'},
    disabled=False)
loop=widgets.Checkbox(
    value=False,
    description='Repeat Play',
    #style={'description_width': 'initial'},
    disabled=False)
HBox([entropy_buffers, show_fps, loop])



In [9]:
from IPython.display import clear_output
from IPython.display import Javascript

def run_all(ev):
    display(Javascript('IPython.notebook.execute_cells_below()'))

def clear_op(event):
    clear_output(wait=True)
    return

button1 = widgets.Button(
    description='Clear Output',
    style= {'button_color':'lightgreen'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '300px'}
)
button2 = widgets.Button(
    description='',
    style= {'button_color':'white'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '83px'},
    disabled=True
)
button1.on_click(run_all)
button1.on_click(clear_op)

In [10]:
def start_demo(event):
    #clear_output(wait=True)
    arg = [];
    arg = common_vcu_demo_decode_display.cmd_line_args_generator(input_path.value, input_url.value, codec_type.value, audio_codecW.value, display_type.value, show_fps.value, loop.value, entropy_buffers.value, proxy_url.value, audio_sinkW.value, audio_output.value, sink_name.value);
    #!sh vcu-demo-decode-display.sh $arg > logs.txt 2>&1
    !sh vcu-demo-decode-display.sh $arg
    return

button = widgets.Button(
    description='click to start vcu-decode-display demo',
    style= {'button_color':'lightgreen'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '300px'}
)

button.on_click(start_demo)
HBox([button, button2, button1])