Video Codec Unit (VCU) Demo Example: STREAM_IN->DECODE ->DISPLAY

Introduction

Video Codec Unit (VCU) in ZynqMP SOC is capable of encoding and decoding AVC/HEVC compressed video streams in real time.

This notebook example acts as Client pipeline in streaming use case. It needs to be run along with Server notebook (vcu-demo-transcode-to-streamout. ipynb or vcu-demo-camera-encode-streamout. ipynb). It receives encoded data over network, decode using VCU and render it on DP/HDMI Monitor.

Implementation Details

This example requires two boards, board-1 is used for transcode and stream-out (as a server) and board 2 is used for streaming-in and decode purpose (as a client) or VLC player on the host machine can be used as client instead of board-2 (More details regarding Test Setup for board-1 can be found in transcode → stream-out Example).

Note: This notebook needs to be run along with "vcu-demo-transcode-to-streamout.ipynb" or "vcu-demo-camera-encode-streamout.ipynb". The configuration settings below are for Client-side pipeline.

Board Setup

Board 2 is used for streaming-in and decode purpose (as a client)

  1. Connect 4k DP/HDMI display to board.
  2. Connect serial cable to monitor logs on serial console.
  3. If Board is connected to private network, then export proxy settings in /home/root/.bashrc file on board as below,
    • create/open a bashrc file using "vi ~/.bashrc"
      • Insert below line to bashrc file
        • export http_proxy="< private network proxy address >"
        • export https_proxy="< private network proxy address >"
      • Save and close bashrc file.
  4. Connect two boards in the same network so that they can access each other using IP address.
  5. Check server IP on server board.
    • root@zcu106-zynqmp:~#ifconfig
  6. Check client IP.
  7. Check connectivity for board-1 & board-2.
    • root@zcu106-zynqmp:~#ping <board-2's IP>
  8. Run stream-in → Decode on board-2

Create test.sdp file on host with below content (Add separate line in test.sdp for each item below) and play test.sdp on host machine.

  1. v=0 c=IN IP4
  2. m=video 50000 RTP/AVP 96
  3. a=rtpmap:96 H264/90000
  4. a=framerate=30

Trouble-shoot for VLC player setup:

  1. IP4 is client-IP address
  2. H264/H265 is used based on received codec type on the client
  3. Turn-off firewall in host machine if packets are not received to VLC.

In [1]:
from IPython.display import HTML

HTML('''<script>
code_show=true; 
function code_toggle() {
 if (code_show){
 $('div.input').hide();
 } else {
 $('div.input').show();
 }
 code_show = !code_show
} 
$( document ).ready(code_toggle);
</script>
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')


Out[1]:

Run the Demo


In [2]:
from ipywidgets import interact
import ipywidgets as widgets
from common import common_vcu_demo_streamin_decode_display
import os
from ipywidgets import HBox, VBox, Text, Layout

Video


In [3]:
codec_type=widgets.RadioButtons(
    options=['avc', 'hevc'],
    description='Codec Type:',
    disabled=False)
video_sink={'kmssink':['DP', 'HDMI'], 'fakevideosink':['none']}

def print_video_sink(VideoSink):
    pass

def select_video_sink(VideoCodec):
    display_type.options = video_sink[VideoCodec]

sink_name = widgets.RadioButtons(options=sorted(video_sink.keys(), key=lambda k: len(video_sink[k]), reverse=True), description='Video Sink:')

init = sink_name.value

display_type = widgets.RadioButtons(options=video_sink[init], description='Display:')
j = widgets.interactive(print_video_sink, VideoSink=display_type)
i = widgets.interactive(select_video_sink, VideoCodec=sink_name)

HBox([codec_type, i, j])


Audio


In [4]:
audio_sink={'none':['none'], 'aac':['auto','alsasink','pulsesink'],'vorbis':['auto','alsasink','pulsesink']}
audio_src={'none':['none'], 'aac':['auto','alsasrc','pulsesrc'],'vorbis':['auto','alsasrc','pulsesrc']}

#val=sorted(audio_sink, key = lambda k: (-len(audio_sink[k]), k))
def print_audio_sink(AudioSink):
    pass
    
def print_audio_src(AudioSrc):
    pass

def select_audio_sink(AudioCodec):
    audio_sinkW.options = audio_sink[AudioCodec]
    audio_srcW.options = audio_src[AudioCodec]

audio_codecW = widgets.RadioButtons(options=sorted(audio_sink.keys(), key=lambda k: len(audio_sink[k])), description='Audio Codec:')

init = audio_codecW.value

audio_sinkW = widgets.RadioButtons(options=audio_sink[init], description='Audio Sink:')
audio_srcW = widgets.RadioButtons(options=audio_src[init], description='Audio Src:')
j = widgets.interactive(print_audio_sink, AudioSink=audio_sinkW)
i = widgets.interactive(select_audio_sink, AudioCodec=audio_codecW)

HBox([i, j])


Advanced options:


In [5]:
kernel_recv_buffer_size=widgets.Text(value='',
    placeholder='(optional) 16000000',
    description='Kernel Recv Buf Size:',
    style={'description_width': 'initial'},
    #layout=Layout(width='33%', height='30px'),
    disabled=False)
port_number=widgets.Text(value='',
    placeholder='(optional) 50000, 42000',
    description=r'Port No:',
    #style={'description_width': 'initial'},
    #
    disabled=False)
#kernel_recv_buffer_size
HBox([kernel_recv_buffer_size, port_number])



In [6]:
entropy_buffers=widgets.Dropdown(
    options=['2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15'],
    value='5',
    description='Entropy Buffers Nos:',
    style={'description_width': 'initial'},
    disabled=False,)
show_fps=widgets.Checkbox(
    value=False,
    description='show-fps',
    #style={'description_width': 'initial'},
    disabled=False)
HBox([entropy_buffers, show_fps])



In [7]:
from IPython.display import clear_output
from IPython.display import Javascript

def run_all(ev):
    display(Javascript('IPython.notebook.execute_cells_below()'))

def clear_op(event):
    clear_output(wait=True)
    return

button1 = widgets.Button(
    description='Clear Output',
    style= {'button_color':'lightgreen'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '300px'}
)
button2 = widgets.Button(
    description='',
    style= {'button_color':'white'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '38px'},
    disabled=True
)
button1.on_click(run_all)
button1.on_click(clear_op)

In [8]:
def start_demo(event):
    #clear_output(wait=True)
    arg = common_vcu_demo_streamin_decode_display.cmd_line_args_generator(port_number.value, codec_type.value, audio_codecW.value, display_type.value, kernel_recv_buffer_size.value, sink_name.value, entropy_buffers.value, show_fps.value, audio_sinkW.value);
    #sh vcu-demo-streamin-decode-display.sh $arg > logs.txt 2>&1
    !sh vcu-demo-streamin-decode-display.sh $arg
    return

button = widgets.Button(
    description='click to start vcu-stream_in-decode-display demo',
    style= {'button_color':'lightgreen'},
    #style= {'button_color':'lightgreen', 'description_width': 'initial'},
    layout={'width': '350px'}
)
button.on_click(start_demo)
HBox([button, button2, button1])