WebRTC WebRTC: Real-Time Communica2ons on the Web Dan Burne9, PhD - - PowerPoint PPT Presentation

▶
webrtc
SMART_READER_LITE
LIVE PREVIEW

WebRTC WebRTC: Real-Time Communica2ons on the Web Dan Burne9, PhD - - PowerPoint PPT Presentation

Feb 12, 2023 165 likes •782 views

WebRTC WebRTC: Real-Time Communica2ons on the Web Dan Burne9, PhD Alan Johnston, PhD Rowan University h9p://standardsplay.com h9p://allthingsrtc.org IETF 100 Singapore November 2017 0 WebRTC: A Joint Standards Effort Internet

slide-1
SLIDE 1

WebRTC: Real-Time Communica2ons on the Web

Dan Burne9, PhD h9p://standardsplay.com h9p://allthingsrtc.org Alan Johnston, PhD Rowan University

WebRTC

IETF 100 Singapore November 2017

slide-2
SLIDE 2

WebRTC: A Joint Standards Effort

  • Internet Engineering Task Force (IETF) and World Wide Web

ConsorFum (W3C) are working together on WebRTC

  • IETF

– Protocols – “bits on wire” – Main protocols are already RFCs, but many extensions in progress – RTCWEB (Real-Time CommunicaFons on the Web) Working Group is the main focus, but other WGs involved as well

  • MMUSIC WG (ICE and SDP extensions)
  • TRAM WG (STUN and TURN extensions)
  • AVTCORE WG (RTP extensions)
  • RMCAT WG (RTP Media CongesFon Avoidance)
  • W3C

– APIs – used by JavaScript code in HTML5 – WEBRTC WG and Media Capture TF are main groups

1 IETF 100 Singapore November 2017

slide-3
SLIDE 3

WebRTC Protocols

IP UDP WebSock et SRTP SDP TURN STUN ICE TCP TLS Transport Layer Network Layer ApplicaFon Layer DTLS HTTP UDP DTLS SCTP

2 IETF 100 Singapore November 2017

slide-4
SLIDE 4

The Browser RTC FuncFon

  • New Real-Time

CommunicaFon (RTC) FuncFon built-in to browsers

  • Contains

– Audio and video codecs – Ability to negoFate peer- to-peer connecFons – Echo cancellaFon, packet loss concealment

  • In Chrome, Firefox,

Microso^ EDGE, and Safari today

3

HTTP or WebSockets

On-the-wire protocols RTC APIs Other APIs NaFve OS Services

Web Server

JavaScript/HTML/CSS

Web Browser

Browser RTC FuncFon

(Signaling)

(Media or Data) HTTP or WebSockets

Signaling Server

IETF 100 Singapore November 2017

slide-5
SLIDE 5

Benefits of WebRTC

For Developer

  • Streamlined development –
  • ne plaaorm
  • NAT traversal only uses

expensive relays when no

  • ther choice
  • Advanced voice and video

codecs without licensing

For User

  • No download or install –

easy to use

  • All communicaton

encrypted – private

  • Reliable session

establishment – “just works”

  • Excellent voice and video

quality

  • Many more choices for real-

Fme communicaFon

4 IETF 100 Singapore November 2017

slide-6
SLIDE 6

WebRTC Triangle

  • Both browsers running the same web applicaFon from web

server

  • Peer ConnecFon established between them with the help of

the web server

Web Server

(ApplicaFon)

Browser M

(Running HTML5 ApplicaFon from Web Server)

Browser L

(Running HTML5 ApplicaFon from Web Server)

Peer ConnecFon (Audio, Video, and/or Data)

5 IETF 100 Singapore November 2017

slide-7
SLIDE 7

WebRTC Triangle

  • Both browsers running the same web applicaFon from web

server

  • Peer ConnecFon established between them with the help of

the web server ("Signaling")

Web Server

(ApplicaFon & Signaling)

Browser M

(Running HTML5 ApplicaFon from Web Server)

Browser L

(Running HTML5 ApplicaFon from Web Server)

Peer ConnecFon (Audio, Video, and/or Data)

6 IETF 100 Singapore November 2017

"Signaling"

slide-8
SLIDE 8

WebRTC Trapezoid

  • Similar to SIP Trapezoid
  • Web Servers communicate using SIP or Jingle or proprietary

Web Server A

(ApplicaFon A)

Peer ConnecFon (Audio and/or Video) Web Server B

(ApplicaFon B)

SIP

  • r Jingle

Browser M

(Running HTML5 ApplicaFon from Web Server A)

Browser T

(Running HTML5 ApplicaFon from Web Server B)

7 IETF 100 Singapore November 2017

slide-9
SLIDE 9

NAT Traversal and STUN

slide-10
SLIDE 10

Peer-to-Peer Media with WebRTC

Web Server

Internet

Router Browser L Home WiFi Router Browser T Browser M Coffee Shop WiFi Router Browser D

9

Media can go directly between browsers!

IETF 100 Singapore November 2017

slide-11
SLIDE 11

Web Server

Internet

Router with NAT Browser L Home WiFi with NAT Browser T Browser M

Most browsers are behind NATs on the Internet, which complicates the establishment

  • f peer-to-peer media

sessions.

Coffee Shop WiFi with NAT Browser D

NAT Complicates Peer-to-Peer Media

10 IETF 100 Singapore November 2017

slide-12
SLIDE 12

Internet

Home WiFi with NAT Browser T 192.168.0.6 Browser M 192.168.0.5

“Outside” Public IP Address

203.0.113.4

“Inside” Private IP Addresses

192.168.x.x

NAT – Network Address Translator

11

A NAT maps an “inside” address to an “outside” address allowing mulFple hosts to share the same IP address.

IETF 100 Singapore November 2017

slide-13
SLIDE 13

Web Server

Internet

Router with NAT Browser L Home WiFi with NAT 203.0.113.4 Browser T Browser M 192.168.0.5 Coffee Shop WiFi with NAT Browser D TURN Server 198.51.100.2

Browser sends STUN test packet to STUN server to learn its public IP address (address of the NAT).

STUN Server 198.51.100.9

STUN Session Traversal UFliFes for NAT

12 IETF 100 Singapore November 2017

No authenFcaFon of STUN!

slide-14
SLIDE 14

Web Server

Internet

Router with NAT Browser L Home WiFi with NAT Browser T Browser M Coffee Shop WiFi with NAT Browser D TURN Server as a Media Relay

In some cases, hole punching fails, and a TURN Media Relay on the public Internet must be used.

STUN Server

TURN – Traversal Using Relay around NAT

13 IETF 100 Singapore November 2017

slide-15
SLIDE 15

What's New for Web Developers

slide-16
SLIDE 16

Two API pieces

  • Local media capture

– Camera, Microphone, <video> element, screen/ window

  • Peer connecFon

– Media and data between two client browsers (or arbitrary devices using naFve API)

IETF 100 Singapore November 2017 15

slide-17
SLIDE 17

WebRTC usage in brief

Set Up Peer Connection Obtain Local Media Exchange Offer/Answer Attach Media

  • r Data

Get more media All media added Peer ConnecFon established Apach more media or data Ready for call

IETF 100 Singapore November 2017 16

slide-18
SLIDE 18

WebRTC usage in brief

  • getUserMedia()

– Audio and/or video – Constraints – User permissions

  • Browser must ask before

allowing a page to access microphone or camera

  • MediaStream
  • MediaStreamTrack

Set Up Peer Connection Obtain Local Media Exchange Offer/Answer Attach Media

  • r Data

Get more media All media added Peer ConnecFon established Apach more media or data Ready for call

IETF 100 Singapore November 2017 17

slide-19
SLIDE 19

WebRTC usage in brief

  • RTCPeerConnection

– Direct media – Between two peers – ICE processing – SDP processing – DTMF support – Data channels – IdenFty verificaFon – StaFsFcs reporFng

Set Up Peer Connection Obtain Local Media Exchange Offer/Answer Attach Media

  • r Data

Get more media All media added Peer ConnecFon established Apach more media or data Ready for call

IETF 100 Singapore November 2017 18

slide-20
SLIDE 20

WebRTC usage in brief

  • addTrack()

– Doesn't change media state!

  • removeTrack()

– Dipo!

  • createDataChannel()

– Depends on transport Set Up Peer Connection Obtain Local Media Exchange Offer/Answer Attach Media

  • r Data

Get more media All media added Peer ConnecFon established Apach more media or data Ready for call

IETF 100 Singapore November 2017 19

slide-21
SLIDE 21

WebRTC usage in brief

  • createOffer(),

createAnswer()

  • setLocalDescription(),

setRemoteDescription()

  • Offer/answer exchange

needed for this to work

Set Up Peer Connection Obtain Local Media Exchange Offer/Answer Attach Media

  • r Data

Get more media All media added Peer ConnecFon established Apach more media or data Ready for call

IETF 100 Singapore November 2017 20

slide-22
SLIDE 22

WebRTC usage in brief

Set Up Peer ConnecFon Obtain Local Media Exchange Session DescripFons Apach Media

  • r Data

Get more media Apach more media or data

Set Up Signaling Channel

21 IETF 100 Singapore November 2017

slide-23
SLIDE 23

Local Media

slide-24
SLIDE 24

Media capture and use

  • navigator.

mediaDevices.getUserMedia()

– Request camera, microphone access – Permission check required for hpp page

  • <video>.srcObject = <mediastream>

– Direct assignment – Works for <audio> as well

  • new MediaStream()

– Can build from tracks in other streams

IETF 100 Singapore November 2017 23

slide-25
SLIDE 25

Browser Media Model: Sources and Sinks

  • Sources can be:

– staFc: files, RTSP, <canvas>, etc – dynamic:

  • local: webcam, microphone - need getUserMedia to

access

  • remote: RTCPeerConnecFon or streaming media
  • Sinks are <img>, <video>, and <audio> tags

– They can be sized, which can cause scaling depending on the constraints on the source

  • RTCPeerConnecFon can be both a source and a sink

24 IETF 100 Singapore November 2017

slide-26
SLIDE 26

Browser Prompts for Permission

25 IETF 100 Singapore November 2017

slide-27
SLIDE 27

Local: Tracks and Streams

  • A MediaStreamTrack

– is a handle to one real-Fme source of media of

  • ne type (audio/video/depth)
  • A MediaStream

– represents a collecFon of zero or more

MediaStreamTracks, of the same or different

media types – indicates that the collected tracks are to be kept synchronized to the best ability of the browser

IETF 100 Singapore November 2017 26

slide-28
SLIDE 28

Peer ConnecFons

slide-29
SLIDE 29

RTCPeerConnecFon APIs

  • Track stuff

– addTrack(), removeTrack() – onaddtrack, onremovetrack

  • Offer/answer stuff

– createOffer(), createAnswer() – setLocalDescripFon(), setRemoteDescripFon()

  • Much more

IETF 100 Singapore November 2017 28

slide-30
SLIDE 30

SDP Offer/Answer

slide-31
SLIDE 31

Session DescripFon Protocol (SDP)

  • Used by browser and WebRTC APIs to describe

media session

– RTP media flows, candidate transport addresses, codec informaFon, media keying informaFon

  • SDP is widely used in SIP VoIP and video systems
  • Browser use of SDP is based on Offer/Answer

– Defined in JSEP - JavaScript Session Establishment Protocol – NegoFaFon proceeds according to Offer/Answer State Machine in next slides

30 IETF 100 Singapore November 2017

slide-32
SLIDE 32

SDP Offer/Answer in RTCPeerConnecFon

  • Peer ConnecFon APIs treat the protocol-level

config (SDP) as a blob

  • createOffer(), createAnswer()

– generate SDP

  • setLocalDescription(),

setRemoteDescription()

– tell the browser which SDP to use

31 IETF 100 Singapore November 2017

slide-33
SLIDE 33

Offer/Answer State Machine

32

stable have- remote-offer have-local-

  • ffer

IETF 100 Singapore November 2017

createOffer()

slide-34
SLIDE 34

Offer/Answer State Machine

33

stable have- remote-offer have-local-

  • ffer

setLocalDescripFon(localOffer)

IETF 100 Singapore November 2017

createOffer()

slide-35
SLIDE 35

Offer/Answer State Machine

34

stable have- remote-offer have-local-

  • ffer

setLocalDescripFon(localOffer) setRemoteDescripFon(remoteOffer)

IETF 100 Singapore November 2017

SDP offer sent over signaling channel to other browser createOffer() createAnswer()

slide-36
SLIDE 36

Offer/Answer State Machine

35

stable have- remote-offer have-local-

  • ffer

setLocalDescripFon(localAnswer) setLocalDescripFon(localOffer) setRemoteDescripFon(remoteOffer)

IETF 100 Singapore November 2017

SDP offer sent over signaling channel to other browser createOffer() createAnswer()

slide-37
SLIDE 37

Offer/Answer State Machine

36

stable have- remote-offer have-local-

  • ffer

setLocalDescripFon(localAnswer) setRemoteDescripFon( remoteAnswer) setLocalDescripFon(localOffer) setRemoteDescripFon(remoteOffer)

IETF 100 Singapore November 2017

SDP answer sent over signaling channel to other browser SDP offer sent over signaling channel to other browser createOffer() createAnswer()

slide-38
SLIDE 38

SDP Extensions in WebRTC

  • A number of new SDP extensions developed

for WebRTC. They include:

– BUNDLE – a way to signal that a set of m= media lines (e.g. audio and video) should be mulFplexed

  • ver the same transport address and port
  • a=group:BUNDLE audio video

– MSID – a way to signal the Media Stream ID used in JavaScript in SDP

  • a=msid

– Source-Specific Apributes – properFes of a source

  • a=ssrc

37 IETF 100 Singapore November 2017

slide-39
SLIDE 39

SDP Offer/Answer Uses Signaling (not standardized)

slide-40
SLIDE 40

WebRTC Signaling Approaches

  • Signaling is required for exchange of candidate transport

addresses and SDP blobs

  • Many opFons – choice is up to web developer

39 IETF 100 Singapore November 2017

slide-41
SLIDE 41

Example: WebRTC Signaling using SIP

40

Web Server SRTP Media SIP Proxy/Registrar Server Browser M

(running JavaScript SIP UA)

Browser T

(running JavaScript SIP UA)

HTTP

(HTML5/CSS/ JavaScript)

WebSocket

(SIP)

WebSocket

(SIP)

HTTP

(HTML5/CSS/ JavaScript)

  • Browser runs a SIP User Agent by running JavaScript from Web Server
  • SRTP media connecFon uses WebRTC APIs
  • Details in RFC 7118 that defines SIP transport over WebSockets

IETF 100 Singapore November 2017

slide-42
SLIDE 42

Data Channel

slide-43
SLIDE 43

Data Channel Protocols

  • Data channel provides a non-media channel between browsers
  • Stream Control Transport Protocol (SCTP) provides reliability,

congesFon control, and message delivery

  • MulFplexed over same ports as RTP media

42

DTLS

Data Channel Data

Provides congesFon and flow control Provides security (confidenFality) Provides transport through NAT (a^er ICE hole punching)

Internet

UDP DTLS SCTP

IETF 100 Singapore November 2017

slide-44
SLIDE 44

Data channel

  • RTCPeerConnecFon.createDataChannel()

– send() method for async message sending – onmessage handler for async message receipt

  • BidirecFonal by default
  • Can send strings or ArrayBuffer types

IETF 100 Singapore November 2017 43

slide-45
SLIDE 45

Other details

slide-46
SLIDE 46

Media Codecs

45

  • Audio mandatory-to-implement:

– Opus (RFC 6716): Narrowband to wideband Internet audio codec for speech and music – G.711 (RFC 3551): PCM audio encoding for PSTN interworking and backwards compaFbility with VoIP systems – Telephone Events (RFC 4733): Transport of Dual Tone MulF Frequency (DTMF) tones

  • Video mandatory-to-implement:

– H.264 (RFC 6184): Common video codec (requires licensing) – VP8 (RFC 6386): Open source video codec

IETF 100 Singapore November 2017

slide-47
SLIDE 47

Security

  • Media is always encrypted in WebRTC
  • Secure RTP (SRTP) is used to assure

confidenFality and authenFcaFon of RTP and RTCP packets

46 IETF 100 Singapore November 2017

slide-48
SLIDE 48

Privacy

  • SRTP, DTLS guarantee integrity of content but

not the endpoint

  • New IdenFty Proxy proposed in WebRTC

– dra^-iea-rtcweb-security – Allows use of third-party idenFty service (e.g., Facebook Connect) – Browser signs SRTP keys using material from idenFty service, verified at other end using idenFty service

IETF 100 Singapore November 2017 47

slide-49
SLIDE 49

New Low-level Controls

slide-50
SLIDE 50

V A

How it Really Works

IETF 100 Singapore November 2017 49

A V A A V A V V A V

Audio m-line Audio m-line Video m-line Video m-line

Stream 1 Stream 1 Stream 2 Stream 2 SDP m-line: bidirecFonal SRTP stream Browser 1 Browser 2

slide-51
SLIDE 51

Senders/Receivers

  • WebRTC 1.0 recently added:

– RTCRtpSenders – A handle to an outbound RTP

stream (one-half of an m-line)

– RTCRtpReceivers – A handle to an inbound RTP

stream (the other half of an m-line)

  • For every track sent or received over a Peer

ConnecFon there is an associated sender or

  • receiver. These give direct access to and

control over the relevant RTP streams.

IETF 100 Singapore November 2017 50

slide-52
SLIDE 52

V A

How it Really Works

IETF 100 Singapore November 2017 51

A V A A V A V V A V

Audio m-line Audio m-line Video m-line Video m-line

Stream 1 Stream 1 Stream 2 Stream 2

Senders Senders Receivers Receivers

SDP m-line: bidirecFonal SRTP stream

slide-53
SLIDE 53

Transceivers

  • But that wasn't enough, because

– at the SDP level every sender is paired with a receiver, even if only one has a track that is acFvely using it

  • Enter the RTCRtpTransceiver. At each endpoint

– There is precisely one transceiver for each m-line. – There is a sender and a receiver for each transceiver. – The mid of the m-line is the mid (media id) of the

  • transceiver. It is unique per Peer ConnecFon.

IETF 100 Singapore November 2017 52

slide-54
SLIDE 54

V A

How it Really Works

IETF 100 Singapore November 2017 53

A V A A V A V V A V

Audio m-line Audio m-line Video m-line Video m-line

Stream 1 Stream 1 Stream 2 Stream 2

Senders Senders Receivers Receivers

For each side, each of these has a Transceiver SDP m-line: bidirecFonal SRTP stream

slide-55
SLIDE 55

Status of WebRTC APIs

  • JavaScript APIs are being standardized by W3C
  • Two main specificaFons

– “WebRTC 1.0: Real-Fme CommunicaFon Between Browsers” (aka PeerConnecFon)

  • Candidate RecommendaFon: hpp://www.w3.org/TR/webrtc
  • Core is stable, just cleaning up edge cases now

– “Media Capture and Streams” (aka getUserMedia)

  • Candidate RecommendaFon:

– hpps://www.w3.org/TR/mediacapture-streams/

  • Oh so close . . .
  • Need implementaFon experience at this point

54 IETF 100 Singapore November 2017

slide-56
SLIDE 56

Status of WebRTC Protocols

  • Two references to check for this:

– Primary is Cullen Jennings' WebRTC Dependencies dra^ dra^-jennings-rtcweb-deps – RTCWEB documents can be followed as usual hpps://tools.iea.org/wg/rtcweb/

55 IETF 100 Singapore November 2017

slide-57
SLIDE 57

Tools and Services

  • To help WebRTC programming

– Libraries, code snippets

  • EasyRTC, SimpleWebRTC.js, CoTurn (hpps://github.com/coturn/

coturn)

– SIP signaling

  • sipML5, JsSIP, reSIProcate
  • To help WebRTC deployment

– Hosted signaling services

  • PubNub, FireBase

– Hosted STUN and TURN servers

  • XirSys, Twilio

– Hosted SFU and/or network opFmizaFon

  • Jitsi, SwitchRTC, Agora.io

IETF 100 Singapore November 2017 56

slide-58
SLIDE 58

Higher-level APIs

  • Alternate (higher-level) APIs

– Twilio, TokBox – APIDaze – PeerJS, RTCMulFConnecFon

IETF 100 Singapore November 2017 57

slide-59
SLIDE 59

Great online resources

  • InformaFonal sites

– hpp://webrtc.org

– http://html5rocks.com/en/tutorials/webrtc/basics – http://webrtchacks.com

– hpps://webrtcstandards.info

  • Games/demos/apps

– hpp://www.cubeslam.com – hpp://shinydemos.com/facekat – hpp://sharefest.me (github.com/Peer5/Sharefest)

IETF 100 Singapore November 2017 58

slide-60
SLIDE 60

What about deployments?

  • Well-known services

– Facebook Chat – Amazon Mayday (and now Chime) – Google Hangouts, Duo

  • Plaaorm embeddings

– Every UC plaaorm today – And most telecom providers

  • Free(mium) comm services

– Crowdcast.io – GoToMeeFng Free (previously Hu.p) – vLine – Talky.io – Appear.in – Gruveo – Room – Rabbit – GetARoom.io – UberConference

IETF 100 Singapore November 2017 59