Connecting people: WebRTC and the Pexip collaboration platform - - PowerPoint PPT Presentation

▶
connecting people webrtc and the pexip collaboration
SMART_READER_LITE
LIVE PREVIEW

Connecting people: WebRTC and the Pexip collaboration platform - - PowerPoint PPT Presentation

Dec 18, 2023 172 likes •553 views

Connecting people: WebRTC and the Pexip collaboration platform TF-WebRTC : May 19, 2015 Who are Pexip? Founded in April 2012 Strong video heritage Manufacture Infjnity a pure-software, virtualized, distributed, scalable

slide-1
SLIDE 1

Connecting people: WebRTC and the Pexip collaboration platform

TF-WebRTC : May 19, 2015

slide-2
SLIDE 2

Who are Pexip?

  • Founded in April 2012
  • Strong video heritage
  • Manufacture Infjnity – a pure-software, virtualized,

distributed, scalable collaboration platform

slide-3
SLIDE 3

Who am I?

  • John-Mark Bell
  • Software engineer in Pexip's UK R&D team
  • Been with Pexip since day one
  • Responsible for Pexip's OS & product security
  • Implemented Pexip's initial WebRTC stack
slide-4
SLIDE 4

Collaboration technologies

SIP H.323

slide-5
SLIDE 5

Some important trends

  • Hardware -> Software
  • Personalisation & BYoD
  • User driven demand

It's no longer about the technology

slide-6
SLIDE 6

Where does WebRTC fjt?

  • Browser-based clients are (becoming!) universal
  • Can be used in isolation, or to augment traditional

voice/video solutions

– Screen/presentation sharing – Control functionality

  • May enhance existing web collaboration tools

Opens up novel solutions and use-cases

slide-7
SLIDE 7

Pexip's philosophy

  • Interoperability matters

– People want to use whatever client/device they have – Collaboration with people using difgerent technologies

needs to work (within and between organisations)

  • Scale is inevitable

– Ease of access to capable clients drives demand – Generational change – people expect to use this stufg

slide-8
SLIDE 8

Pexip's philosophy

  • Extensibility is vital

– We provide a solid, feature-rich foundation – But we don't do everything or solve every use case – Therefore, we expose open, documented APIs which

allow an ecosystem to build around us

– This also allows customers to be proactive in

addressing their specifjc needs by building on our foundations

slide-9
SLIDE 9

Interoperability

Softphones Lync2013 Rooms Lync2010 VoIP Phones Desktops H.264 H.263 G.7xx AAC Opus VP8 H.264HP* SIP H.323 SVC WebRTC H.265* Siren RDP RTV VP9* Skype for Business Lync2011 Skype H.261 Silk Speex

slide-10
SLIDE 10

Pexip Distributed Architecture

  • T

ypical Deployment

  • 1 x Management Node
  • Multiple Conferencing Nodes
  • Resilience
  • Resilient to temporary network splits
  • Leverages VMware and Hyper-V

resilience mechanisms

  • Benefjts
  • Management: Single point of

management & diagnostics for a global deployment

  • Keeps bandwidth usage local
  • Minimizes bandwidth usage between

regions

Distributed Pexip Infjnity Deployment

Management Node

Conferencing Nodes Conferencing Nodes Conferencing Nodes

slide-11
SLIDE 11

Distributed Architecture – Conferencing

High defjnition video stream Low resolution video stream

T raditional Centralized Deployment Distributed Conferencing

Multiple HD streams One centralized data center Only one HD stream in each direction Multiple low bandwidth PIP streams Resilient to network outages

slide-12
SLIDE 12

Distributed Architecture - Gateway

Headquarters New York Remote Offjce Singapore

Lync H.323 Room System SIP Desktop System Lync

Headquarters New York Remote Offjce Singapore

Lync H.323 Room System SIP Desktop System Lync

Centralized Deployment Distributed Deployment

Centralized deployment due to high custom hardware cost Excessive WAN bandwidth used due to hairpinning of media Long latency for some local calls Reduced user experience No media hairpinning No WAN bandwidth for local calls Lowest possible latency Better user experience

slide-13
SLIDE 13

Management Node

Endpoint registration and call routing

SIP registration SIP registration Advanced call control through distributed policy server and distributed registrar

Policy Server Registrar and Conferencing Nodes Registrar and Conferencing Nodes Registrar and Conferencing Nodes

slide-14
SLIDE 14

Pexip Infjnity APIs in a Nutshell

Pexip Infjnity Pexip Infjnity

Mgmnt API

Policy Servers Policy Servers

Client API Policy API

Client Voice/Video Applications Client Voice/Video Applications Management Applications Management Applications

slide-15
SLIDE 15

Client Applications Client Applications

Pexip App Framework (Browser) Pexip App Framework (Browser)

Pexip Infjnity Pexip Infjnity

Management API (REST) Management API (REST)

Status API

Get active conferences Get Conf Node status ...

Confjguration API

Add/delete/edit VMR Deploy new Conf Node ...

Command API

Dial participant Mute participant ...

Client API (REST) Client API (REST)

Status API

Get conference info Get participant info ...

EventSource API

Conference info (push) ...

Command API

Escalate to video Start presentation ...

App (iOS) App (iOS)

Management Applications

Management REST API can be used for Automatic deployment Automatic provisioning Custom web interfaces Custom conference control ...and more

Management Applications

Management REST API can be used for Automatic deployment Automatic provisioning Custom web interfaces Custom conference control ...and more

Media Media

WebRTC RTMP App (Android) App (Android) App (Browser) App (Browser) Pexip App Framework (Android) Pexip App Framework (Android) Pexip App Framework (iOS) Pexip App Framework (iOS)

Policy Server

Third party policy server

Policy Server

Third party policy server

External Policy API External Policy API

slide-16
SLIDE 16

Client API

slide-17
SLIDE 17

Client API overview

  • REST, with JSON payloads
  • We use it to build our:

– web application – mobile applications for iOS/Android – installable client for Windows/Mac/Linux

slide-18
SLIDE 18

Client API fmow

  • Connect to conference
  • (Regularly) refresh connection to conference
  • (Optionally) register for event notifjcations
  • (Optionally) activate media functionality
  • Disconnect from conference

The same fmow works with gateways, too

slide-19
SLIDE 19

Connect to conference

  • HTTP POST to

https://cn/api/client/v2/conferences/conference/request_token

C S | request_token | |-------------------------------------->| | {'display_name': 'Test'} | | | | response| |<--------------------------------------| | {'status': 'success', | | 'result': { | | 'token' : '...', | | 'expires' : '120', | | 'participant_uuid': '...', | | 'version' : '...', | | 'role' : 'HOST', | | 'chat_enabled' : true, | | 'service_type' : 'conference', | | 'stun' : '...' | | } | | } | | |

import json import requests import time base_url = 'https://10.0.0.1/api/client/v2/conferences/test/' response = requests.post(base_url + 'request_token', headers = { 'Content-Type' : 'application/json' }, data = json.dumps({ 'display_name' : 'Test' })).json() token = response['result']['token'] expires = time.time() + int(response['result']['expires']) / 2 participant = response['result']['participant_uuid']

slide-20
SLIDE 20

Refresh connection to conference

  • HTTP POST to

https://cn/api/client/v2/conferences/conference/refresh_token

C S | refresh_token | |-------------------------------------->| | | | | | response| |<--------------------------------------| | {'status': 'success', | | 'result': { | | 'token' : '...', | | 'expires' : '120', | | 'participant_uuid': '...', | | 'version' : '...', | | 'role' : 'HOST', | | 'chat_enabled' : true, | | 'service_type' : 'conference', | | 'stun' : '...' | | } | | } | | |

response = requests.post(base_url + 'refresh_token', headers = { 'Content-Type' : 'application/json', 'token' : token }).json() token = response['result']['token'] expires = time.time() + int(response['result']['expires']) / 2

slide-21
SLIDE 21

Register for event notifjcations

  • HTTP GET to

https://cn/api/client/v2/conferences/conference/events

C S | events | |----------------------------------------------->| | | | | | response| |<-----------------------------------------------| | Event stream: | | http://www.w3.org/TR/eventsource/ | | e.g.: | | : hello | | | | event: conference_update | | id: MTAuNDQuOTkuMjE= | | data: {"guests_muted": false, “locked": false} | | | | event: participant_sync_begin | | id: MTAuNDQuOTkuMjI= | | data: null |

response = requests.get(base_url + 'events', headers = { 'token' : token }, stream = True) for event in parse_events(response): process_event(event) if time.time() > expires: refresh_token()

slide-22
SLIDE 22

Parse the event stream

C S | events | |----------------------------------------------->| | | | | | response| |<-----------------------------------------------| | Event stream: | | http://www.w3.org/TR/eventsource/ | | e.g.: | | : hello | | | | event: conference_update | | id: MTAuNDQuOTkuMjE= | | data: {"guests_muted": false, “locked": false} | | | | event: participant_sync_begin | | id: MTAuNDQuOTkuMjI= | | data: null | | |

def parse_events(response): state = 'BEFORE_EVENT' for line in response.iter_lines(chunk_size=1): if state == 'BEFORE_EVENT' and line.startswith('event:'): state = 'IN_EVENT' event_name = line[6:].strip() elif state == 'BEFORE_EVENT' and line.startswith(': ping'): yield ('PING', None) elif state == 'IN_EVENT' and line.startswith('data:'): state = 'BEFORE_EVENT' event_data = json.loads(line[5:]) yield (event_name, event_data) def process_event(event): name, data = event if name != 'PING': # Do something with the event

slide-23
SLIDE 23

T ypes of event

  • Participant create/update/delete

– May be used to display roster list in client

  • Presentation start/stop/new slide
  • Conference status updated (e.g. lock, guest mute)
  • Chat message received
  • Disconnected by server
slide-24
SLIDE 24

Activate media functionality

  • HTTP POST to

https://cn/api/client/v2/conferences/conference/participants/participant/calls C S | participants/<uuid>/calls | |-------------------------------------->| | {'call_type': 'WEBRTC', | | ['present' : 'send|receive'] | | 'sdp' : '...' | | } | | | | response| |<--------------------------------------| | {'status': 'success', | | 'result': { | | 'call_uuid': '...', | | 'sdp' : '...' | | } | | } | | |

base_url_p = base_url + 'participants/' + participant + '/' response = requests.post(base_url_p + 'calls', headers = { 'Content-Type' : 'application/json', 'token' : token }, data = json.dumps({ 'call_type' : 'WEBRTC', 'sdp' : local_sdp })).json() call = response['result']['call_uuid'] remote_sdp = response['result']['sdp']

slide-25
SLIDE 25

Start media fmowing (after ICE completes)

  • HTTP POST to

https://cn/api/client/v2/conferences/conference/participants/participant/calls/call/ack C S | participants/<uuid>/calls/<call>/ack | |-------------------------------------->| | | | response| |<--------------------------------------| | |

base_url_c = base_url_p + 'calls/' + call + '/' requests.post(base_url_c + 'ack', headers = { 'Content-Type' : 'application/json', 'token' : token })

slide-26
SLIDE 26

Disconnect from conference

  • HTTP POST to

https://cn/api/client/v2/conferences/conference/release_token

C S | release_token | |-------------------------------------->| | | | | | response| |<--------------------------------------| | |

requests.post(base_url + 'release_token', headers = { 'Content-Type' : 'application/json', 'token' : token })

slide-27
SLIDE 27

Other client API functionality

  • Conference control

– (Un)lock conference (and grant access to individual

participants if they join when the conference is locked)

– (Un)mute all guests – Disconnect some/all participants – Lock spotlight on a participant, so they are always visible

  • Media de-escalation and DTMF entry
  • Outbound dialling from conference
  • Send chat messages to conference
slide-28
SLIDE 28

Client API forthcoming features

  • Presentation sending

– Allows a client to present a JPEG or PNG image into the

conference

– API is currently beta (web application in Infjnity 9 uses it

today)

  • Client registration, and incoming calls

– Client may log in to the service and be notifjed of incoming

calls

– Allows dial-out to WebRTC clients (from conferences, and

through gateway rules)

slide-29
SLIDE 29

Client API SDKs

  • Javascript

– Provides a Javascript-friendly abstraction over the client REST API – We use it ourselves to build the Infjnity web client – Designed to be used by third-parties to produce custom UIs and

workfmows

  • IOS

– Available later in 2015 – The basis of Infjnity Connect for iOS – Voice and video capable, in addition to exposing the Infjnity client

API in a more appropriate way for iOS application developers

slide-30
SLIDE 30

Javascript SDK usage

<!DOCTYPE html> <html> <head> <script type=”text/javascript” src=”https://cn/static/webrtc/js/pexrtc.js”> <script type=”text.javascript”> window.onload = function() { var rtc = new PexRTC(); rtc.onSetup = function(url, pin_status) { rtc.connect(null); }; rtc.onConnect = function(url) { if (url !== null) { document.getElementById('remote-video').src = url; } }; rtc.makeCall('cn', 'test', 'Test', null); }; </script> </head> <body> <video width=”100%” id=”remote-video” autoplay=”autoplay”> </body> </html>

slide-31
SLIDE 31

Policy API overview

  • New in Infjnity version 9
  • Simple HTTP(S) GET request/response
  • May be authenticated
  • Permits external control over (in Infjnity v9):

– Service (VMR/Gateway/IVR) to use – Media location – Audio avatar – Participant avatar (i.e. for display in client roster list)

slide-32
SLIDE 32

Simple policy example

  • Goal: allow internal users to connect directly to a

conference, but require external users to provide a pin

E S P | INVITE a@b.com | | |---------------------->| | | | service/configuration | | |---------------------->| | | | | | response| | |<----------------------| | 200 OK| | |<----------------------| |

slide-33
SLIDE 33

Service confjguration requests

  • GET /policy/v1/service/confjguration
  • Parameters in URL query string:

?protocol=sip&node_ip=1.2.3.4&remote_address=4.5.6.7& local_alias=sip:a@b.com&call_direction=dial_in&encryption=On& remote_alias=sip:alice@example.com&remote_display_name=Alice& location=London

slide-34
SLIDE 34

Service confjguration responses

  • Depends on desired behaviour:

– Send 404 Not Found to fall back to default

Infjnity behaviour

– Send 200 OK with Content-T

ype: application/json to authoritatively defjne service confjguration:

  • Failure: { 'status' : 'failed', 'reason' : 'Descriptive reason' }
  • Success: { 'status' : 'success', 'result' : { ... } }
slide-35
SLIDE 35

Minimal policy response generator

def create_response(request): config = None if 'sip:a@b.com' in request.args.get('local_alias', []): config = { 'service_type' : 'conference', 'name' : 'test', 'pin' : '' } if not requests.args.get('remote_alias', [''])[0].endswith('@b.com'): config['pin'] = '1234' return config

  • Assumes web server framework exists already!
slide-36
SLIDE 36

Minimal policy response generator

def create_response(request): config = None if 'sip:a@b.com' in request.args.get('local_alias', []): config = { 'service_type' : 'conference', 'name' : 'test', 'pin' : '1234', 'allow_guests' : True } if not requests.args.get('remote_alias', [''])[0].endswith('@b.com'): config['guest_pin'] = '9999' return config

  • What if we want to separate guests and hosts as

well?

slide-37
SLIDE 37

Other policy applications

  • Integrate with Active Directory

– Dynamically manufacture VMRs based on directory contents and dialed local

alias

– Restrict VMR access to users in specifjc AD groups – Dynamically apply bandwidth restrictions to calls based on importance of

participants

  • Set conference themes on the fmy

– e.g. on a per-location basis

  • Force media to a specifjc location

– e.g. to conserve resources

  • Custom billing engines

– e.g. for fjne-grained control over multi-tenant environments

slide-38
SLIDE 38