Video Codec unit (VCU) in ZynqMP SOC is capable of decoding AVC and HEVC compressed video streams in real time.
This notebook example runs Live playback usecase where it captures compressed video and audio(optional) content from USB Camera connected to board, decodes video using VCU, decodes audio using software gstreamer decode element and renders output on DP/HDMI Display.
The audio device name of audio source(Input device) and playback device(output device) need to be determined using arecord and aplay utilities installed on platform.
Audio Input
ALSA sound device names for capture devices
Run below command to get ALSA sound device names for capture devices
root@zcu106-zynqmp:~#arecord -l
It shows list of Audio Capture Hardware Devices. For e.g
- card 1: C920 [HD Pro Webcam C920], device 0: USB Audio [USB Audio]
- Subdevices: 1/1
- Subdevice #0: subdevice #0
Here card number of capture device is 1 and device id is 0. Hence " hw:1,0 " to be passed as auido input device.
Pulse sound device names for capture devices
Run below command to get PULSE sound device names for capture devices
root@zcu106-zynqmp:~#pactl list short sources
It shows list of Audio Capture Hardware Devices. For e.g
- 0 alsa_input.usb-046d_HD_Pro_Webcam_C920_758B5BFF-02.analog-stereo ...
Here "alsa_input.usb-046d_HD_Pro_Webcam_C920_758B5BFF-02.analog-stereo" is the name of audio capture device. Hence it can be passed as auido input device.
Audio Output
ALSA sound device names for playback devices:
Run below command to get ALSA playback device names for output devices
root@zcu106-zynqmp:~#aplay -l
It shows list of Playback Hardware Devices. For e.g
- card 0: monitor [DisplayPort monitor], device 0: (null) xilinx-dp-snd-codec-dai-0 []
- Subdevices: 1/1
- Subdevice #0: subdevice #0
- card 0: monitor [DisplayPort monitor], device 1: (null) xilinx-dp-snd-codec-dai-1 []
- Subdevices: 1/1
- Subdevice #0: subdevice #0
Here card number "0" is being used for playback device for display port channel 0 and device id is 0, so "hw:0,0" Hence it can be passed as auido output device.
PulseAudio sound device names playback devices
Run below command to get PULSE playback device names for output devices
root@zcu106-zynqmp:~#pactl list short sinks
It shows list of Playback Hardware Devices. For e.g
- 0 alsa_output.platform-fd4a0000.zynqmp-display_zynqmp_dp_snd_card.analog-stereo ...
Here "alsa_output.platform-fd4a0000.zynqmp-display_zynqmp_dp_snd_card.analog-stereo" is the name of audio playback device. Hence it can be passed as auido output device.
Resolutions for this example need to set based on USB Camera Capabilities
Capabilities can be found by executing below command on board
root@zcu106-zynqmp:~#"v4l2-ctl -d < dev-id > --list-formats-ext".
< dev-id >:- It can be found using dmesg logs. Mostly it would be like "/dev/video0"
In [1]:
from IPython.display import HTML
HTML('''<script>
code_show=true;
function code_toggle() {
if (code_show){
$('div.input').hide();
} else {
$('div.input').show();
}
code_show = !code_show
}
$( document ).ready(code_toggle);
</script>
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')
Out[1]:
In [2]:
from ipywidgets import interact
import ipywidgets as widgets
from common import common_vcu_demo_camera_decode_display
import os
from ipywidgets import HBox, VBox, Text, Layout
In [3]:
video_capture_device=widgets.Text(value='',
placeholder='"/dev/video1"',
description='Camera Dev Id:',
style={'description_width': 'initial'},
#layout=Layout(width='35%', height='30px'),
disabled=False)
video_capture_device
In [4]:
codec_type=widgets.RadioButtons(
options=['avc', 'hevc'],
description='Codec Type:',
disabled=False)
video_size=widgets.RadioButtons(
options=['640x480', '1280x720', '1920x1080', '3840x2160'],
description='Resolution:',
description_tooltip='To select the values, please refer USB Camera Capabilities section',
disabled=False)
#video_size
HBox([codec_type, video_size])
In [5]:
video_sink={'kmssink':['DP', 'HDMI'], 'fakevideosink':['none']}
def print_video_sink(VideoSink):
pass
def select_video_sink(VideoCodec):
display_type.options = video_sink[VideoCodec]
sink_name = widgets.RadioButtons(options=sorted(video_sink.keys(), key=lambda k: len(video_sink[k]), reverse=True), description='Video Sink:')
init = sink_name.value
display_type = widgets.RadioButtons(options=video_sink[init], description='Display:')
j = widgets.interactive(print_video_sink, VideoSink=display_type)
i = widgets.interactive(select_video_sink, VideoCodec=sink_name)
HBox([i, j])
In [6]:
device_id=Text(value='',
placeholder='(optional) "hw:1"',
description='Input Dev:',
description_tooltip='To select the values, please refer Determine Audio Device Names section',
disabled=False)
audio_output=Text(value='',
placeholder='(optional) "hw:0"',
#description=r'<p>Output Dev:</p>',
description='Output Dev:',
description_tooltip='To select the values, please refer Determine Audio Device Names section',
style={'flex container': 'cross-end'},
#width = 100,
disabled=False)
display(HBox([device_id, audio_output]))
In [7]:
audio_sink={'none':['none'], 'aac':['auto','alsasink','pulsesink'],'vorbis':['auto','alsasink','pulsesink']}
audio_src={'none':['none'], 'aac':['auto','alsasrc','pulsesrc'],'vorbis':['auto','alsasrc','pulsesrc']}
#val=sorted(audio_sink, key = lambda k: (-len(audio_sink[k]), k))
def print_audio_sink(AudioSink):
pass
def print_audio_src(AudioSrc):
pass
def select_audio_sink(AudioCodec):
audio_sinkW.options = audio_sink[AudioCodec]
audio_srcW.options = audio_src[AudioCodec]
audio_codecW = widgets.RadioButtons(options=sorted(audio_sink.keys(), key=lambda k: len(audio_sink[k])), description='Audio Codec:')
init = audio_codecW.value
audio_sinkW = widgets.RadioButtons(options=audio_sink[init], description='Audio Sink:')
audio_srcW = widgets.RadioButtons(options=audio_src[init], description='Audio Src:')
j = widgets.interactive(print_audio_sink, AudioSink=audio_sinkW)
k = widgets.interactive(print_audio_src, AudioSrc=audio_srcW)
i = widgets.interactive(select_audio_sink, AudioCodec=audio_codecW)
HBox([i, k, j])
In [8]:
frame_rate=widgets.Text(value='',
placeholder='(optional) 15, 30, 60',
description='Frame Rate:',
disabled=False)
bit_rate=widgets.Text(value='',
placeholder='(optional) 1000, 20000',
description='Bit Rate(Kbps):',
style={'description_width': 'initial'},
disabled=False)
display(HBox([bit_rate, frame_rate]))
In [9]:
entropy_buffers=widgets.Dropdown(
options=['2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15'],
value='5',
description='Entropy Buffers Nos:',
style={'description_width': 'initial'},
disabled=False,)
no_of_frames=Text(value='',
placeholder='(optional) 1000, 2000',
description=r'<p>Frame Nos:</p>',
#layout=Layout(width='33%', height='30px'),
disabled=False)
HBox([entropy_buffers, no_of_frames])
In [10]:
show_fps=widgets.Checkbox(
value=False,
description='show-fps',
disabled=False)
show_fps
In [11]:
from IPython.display import clear_output
from IPython.display import Javascript
def run_all(ev):
display(Javascript('IPython.notebook.execute_cells_below()'))
def clear_op(event):
clear_output(wait=True)
return
button1 = widgets.Button(
description='Clear Output',
style= {'button_color':'lightgreen'},
#style= {'button_color':'lightgreen', 'description_width': 'initial'},
layout={'width': '300px'}
)
button2 = widgets.Button(
description='',
style= {'button_color':'white'},
#style= {'button_color':'lightgreen', 'description_width': 'initial'},
layout={'width': '80px'},
disabled=True
)
button1.on_click(run_all)
button1.on_click(clear_op)
In [12]:
def start_demo(event):
#clear_output(wait=True)
arg = [];
arg = common_vcu_demo_camera_decode_display.cmd_line_args_generator(device_id.value, video_capture_device.value, video_size.value, codec_type.value, audio_codecW.value, display_type.value, sink_name.value, no_of_frames.value, bit_rate.value, entropy_buffers.value, show_fps.value, audio_srcW.value, audio_output.value, audio_sinkW.value, frame_rate.value);
#!sh vcu-demo-camera-decode-display.sh $arg > logs.txt 2>&1
!sh vcu-demo-camera-decode-display.sh $arg
return
button = widgets.Button(
description='click to start vcu-camera-decode-display demo',
style= {'button_color':'lightgreen'},
#style= {'button_color':'lightgreen', 'description_width': 'initial'},
layout={'width': '300px'}
)
button.on_click(start_demo)
HBox([button, button2, button1])
In [ ]:
[1] https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842546/Xilinx+Video+Codec+Unit
[2] https://www.xilinx.com/support.html#documentation (Refer to PG252)