IDTV Software/Middleware Pablo Cesar Distributed Multimedia - - PowerPoint PPT Presentation

▶
idtv software middleware
SMART_READER_LITE
LIVE PREVIEW

IDTV Software/Middleware Pablo Cesar Distributed Multimedia - - PowerPoint PPT Presentation

Mar 30, 2023 174 likes •739 views

IDTV Software/Middleware Pablo Cesar Distributed Multimedia Languages & Infrastructures CWI: Centrum voor Wiskunde en Informatica Amsterdam, NL Outline Standards Overview MHP OCAP DASE ARIB Harmonization NCL

slide-1
SLIDE 1

IDTV Software/Middleware

Pablo Cesar

Distributed Multimedia Languages & Infrastructures CWI: Centrum voor Wiskunde en Informatica Amsterdam, NL

slide-2
SLIDE 2

Outline

Standards Overview MHP OCAP DASE ARIB Harmonization NCL STB Evolution

slide-3
SLIDE 3

Standards Overview

slide-4
SLIDE 4

Standards Overview

Europe

  • MHP

USA

  • ACAP

Japan

  • ARIB

Execution Environment

  • Java-based

Presentation Environment

  • XML-Based
slide-5
SLIDE 5

MHP (Europe)

Reception and presentation of applications in an interoperable manner. Horizontal market approach

  • Applications and decoders can be developed by

different developers and manufacturers Expects the DVB standardized broadcast and return channel Two environments:

  • Java-based: DVB-J
  • XML-based: DVB-HTML
slide-6
SLIDE 6

MHP: File System

Digital Storage Media Command and Control Protocol (DSM-CC) Object Carousel, Data Carousel Broadcast file system, Read-only Objects are transmitted in a cycle Objects can be files, directories, data streams, or stream events

slide-7
SLIDE 7

MHP Profiles: Enhanced

Enhanced Broadcast Profile

  • Baseline profile
  • No return channel
  • Applications such as EPGs and one-way

information services

slide-8
SLIDE 8

MHP Profiles: Interactive

Interactive Broadcast Profile

  • Return channel for two-way interaction
  • Enables e-commerce, voting, betting, etc
slide-9
SLIDE 9

MHP Profiles: Internet Access

Internet Access Profile (MHP 1.1)

  • Support for resident applications providing

services such as: WWW, Email, News

slide-10
SLIDE 10

MHP Graphics

32-bit colors (MHP minimum palette 140+48 colors) 3-layers:

  • background, video,

graphics Java drawing primitives Image formats:

  • JPEG, GIF, PNG, MPEG

I-Frame One resident font:

  • Tiresias

Scaled video

slide-11
SLIDE 11

MHP Graphics: Background

Only one still image MPEG I-Frame bitmap Always full-screen Opacity cannot be set

slide-12
SLIDE 12

MHP Graphics: Video

MPEG-video from TV-stream Video is scalable Full screen, 1/4, 1/16 Other if device supports Multiple simultaneous video streams might be supported

slide-13
SLIDE 13

MHP Graphics: Graphic Layer

slide-14
SLIDE 14

MHP: DVB-J

Java-based open standard for interactive TV Current active version: 1.0.2; current version 1.1 Based on J2ME:

  • JavaTV: Application model (lifecycle of the Xlets)
  • HAVi: UI components, event handling
  • DAVIC: File system (DSM-CC)
  • JMF: videos
  • DVB specific APIs:
slide-15
SLIDE 15

MHP: DVB-J

slide-16
SLIDE 16

MHP: HAVi

slide-17
SLIDE 17

MHP: Example of an Xlet

import libraries; public class AWTHelloXlet extends Component implements KeyListener, Xlet { private XletContext context; // The environment private HScene scene; // Where shall all this be rendered? private Image image; // Let us put an image there… private String message; // The message… public AWTHelloXlet() { super(); } /** * Xlet Interface. * Initialize this Xlet. */ public void initXlet(XletContext context) throws XletStateChangeException public void startXlet() throws XletStateChangeException public void pauseXlet() public void destroyXlet(boolean unconditional) public void paint(Graphics graphics) }

slide-18
SLIDE 18

MHP: DVB-HTML

Modularization of XHTML

  • Comparable to XHTML 1.1

CSS Support

  • CSS2 Compatible
  • DVB Extensions: media type “dvb-tv”, compose-rule,

clip-video, viewport Media Types Support

  • DVB-J Applications
  • Video and Audio (MPEG-2)
  • ECMAScript
  • Images (png, gif, mpeg)

DOM

  • Level 2 Modules: Stylesheets, Events
  • DVB Extensions: DVB Events, KeyEvents, DVB CSS
slide-19
SLIDE 19

MHP: DVB-HTML

slide-20
SLIDE 20

MHP: Example DVB-HTML

<?xml version="1.0" encoding="ISO-8859-1"?><!DOCTYPE html SYSTEM "supertext.dtd"> <html> <head> <title>SM-Liiga</title> <link rel="stylesheet" href="styles1Column.css"/> </head> <body> <div class="title"> <span style="width: 200;">Tilastot </span> </div> <div class="main"> <br><span class="lihavointi"></span><br><br> <table> <tr> <td>SARJATAULUKKO</td> <td ></td> </tr> <tr> <td bgcolor="#cccccc"></td> </tr> ........

slide-21
SLIDE 21

MHP: Critical Approach

Return Channel capabilities One screen (on-screen display) One remote control (Wii?) Low graphics capabilities Declarative part

  • no media centric

Most successful service

  • Voting popular programs

Popularity

  • Italy, Spain, Taiwan, Korea
slide-22
SLIDE 22

OCAP (USA)

Open Cable Applications Platform OCAP 1.0 includes Java-based services OCAP 2.0 includes a presentation engine for HTML OCAP is a software middleware layer providing access to operating system and hardware Based on MHP:

  • Java APIs: Java TV, JMF, DAVIC, HAVi

Four different types of applications:

  • Native applications: written for a specific host
  • Bound applications
  • Unbound applications: not dependent of a specific

channel

  • Monitor application: network operator is responsible of

the state of the receiver

slide-23
SLIDE 23

DASE (USA)

DTV TV Application Software Environment DASE Level 1: includes a declarative and Java-based environments DASE Level 2: Return channel capabilities DASE Level 3: Internet access DASE – Declarative:

  • Superset of XHTML Basic comparable to XHTML 1.1
  • Support for a number of media formats:

—video (e.g., MPEG) —animated graphics (e.g., MNG) —audio (e.g., MPEG),

Style sheet support is a subset of CSS2

  • Similar to the approach used in DVB-HTML
slide-24
SLIDE 24

ACAP (USA)

Advanced Common Application Platform Harmonization of DASE and OCAP Features from DASE and from OCAP:

  • Java part of OCAP
  • Declarative part from DASE
slide-25
SLIDE 25

ARIB

ARIB (Japanese solution) standardises:

  • Procedural: STD-B23
  • Declarative: STD-B24 (BML)

BML (Broadcast Markup Language)

  • Superset of XHTML 1.1
  • Support for many media formats:

—MPEG-4 video and audio —Streamed text —Japanese characters

  • Fully CSS2 compliant with extra properties such as nav-

index

  • Quite ambitious
slide-26
SLIDE 26

Standards Comparison

slide-27
SLIDE 27

Standards Comparison (MHP, ACAP, ARIB)

Procedural Environment

  • J2ME (Personal Profile)
  • Java TV, JMF, DAVIC, HAVi

Service information

  • Different mechanisms (e.g., OCAP uses for unbound

applications the eXtended Application Table) Bridge to declarative content

  • Not included in ARIB

Graphics plane

  • Background + Video + Graphics

Declarative Environment

  • Quite different
slide-28
SLIDE 28

Harmonization

Globally Executable MHP (GEM) Based on MHP 1.0.2 MHP without DVB specific parts Freely applicable to various kinds of network types and standards

  • MHP
  • OCAP
  • ACAP
  • ARIB B23 - BML

Makes application interoperability possible Used in Blu-Ray technology

slide-29
SLIDE 29

Harmonization: GEM

Procedural Interpreted: (DVB-J)

  • Basic Java (e.g., java.lang)
  • Java TV: applications lifecycle
  • JMF: audio and video
  • HAVi: 2D User Interface

Declarative: Undecided

  • XHTML: structure of document
  • CSS: look
  • Media types: images, video
  • TV Specific extensions: for example remote

control interaction

slide-30
SLIDE 30

Harmonization: Java Environment

slide-31
SLIDE 31

Harmonization: Presentation Engine

slide-32
SLIDE 32

Harmonization: Application Environment

slide-33
SLIDE 33

Harmonization: Now you get it!

slide-34
SLIDE 34

ACAP

Harmonization

DVB-T/ DVB-S/ DVB-C OCAP data broadcasting ISDB MHP DASE BML NCL GEM ATSC DVB-H Transmission OCAP Software Environment ESG J2ME XHTML

slide-35
SLIDE 35

NCL (Nested Context Language)

Brazilian solution Based on temporal synchronization of multimedia objects and not

  • n user interaction in particular
  • Very similar to SMIL

NCL supports:

  • Media objects (video, images)
  • HTML objects (e.g., DVB-HTML, BML, and ACAP-X)

MAESTRO is a complete solution based on NCL Alternatives:

  • Resident middleware with NCL declarative module (procedural code

can be launched from declarative applications)

  • MAESTRO + procedural middleware
  • MAESTRO as an Xlet
slide-36
SLIDE 36

NCL: MAESTRO Alternatives

Lua Engine ECMA Script Engine

Apl 1 Apl 2 Apl 3 Apl n

…

OS OS OS OS OS

VM (Java)

Procedural Middleware Apl 1 Apl 2 Xlet i Apl n

…

VM (Java)

Xlet i Apl n

…

Xlet

OS OS OS

Procedural Middleware

slide-37
SLIDE 37

SMIL

declarative language for temporal interaction allows scheduled & interactive activation supports rich model-based hyperlinks supports extensible content control allows separation of content from structure & control profiles available for mainframe, PC, PDA and mobile devices – TV Profile is under way

slide-38
SLIDE 38

SMIL: Example

<smil> <head> <layout> <root-layout id=“background“ width=“” height=“”/> <region id=“video"/> <region id="caption"/> </layout> </head> <body> <seq> <!– Program 1--> <seq> <!– Sequence 1--> </seq> <!– Program 2--> <!– Program 3--> </seq> </body> </smil>

slide-39
SLIDE 39

MPEG-4

Evolution:

  • MPEG traditionally targeted to audio/video codecs (MPEG-

1, MPEG-2)

  • Complex toolkit capable of providing solutions for

multimedia applications Scene:

  • Composition of different multimedia objects (2D, 3D, video)

including their spatial and temporal relationships Entry points:

  • BInary Format for Scene (BIFS):

—Hierarchical structure (scene graph) —Properties: color, size, position, and timing —Behavior: BIFS commands (conditional) and Animations

  • MPEG-Java: set of Java APIs
  • eXtensible MPEG-4 Texttual (XMT): XML language that

describes scenes

slide-40
SLIDE 40

MPEG-4

Some of the Scene Nodes:

  • Top: root of the graph (e.g., Layer3D and Layer2D)
  • Grouping: containers of multimedia objects
  • Sensor: nodes capable of detecting events (e.g., Time and Touch)
  • Shape: Graphical Primitives that include two fields: Geometry (e.g.,

rectangle and circle) and Appearance (e.g., texture and material)

  • Face: integration of synthetic 3D human-like objects

Interaction:

  • Sensors detect events and Route distribute them
  • Predefined behaviors: resize, relocate
  • Complex behavior: script or Java

Widgets:

  • Can be implemented (e.g., sensor + Shape)

Layout:

  • Local coordinates of the objects (more complex automatic layout is

not permitted)

slide-41
SLIDE 41

MPEG-4: Example

slide-42
SLIDE 42

MHEG-5

MHEG-5 chosen for digital terrestrial television receivers in the UK Application: composition of several scenes Each scene integrates different multimedia objects Content Classes

  • Multimedia objects (e.g., video or audio clips)
  • Contained in MHEG object (small data) or reference (e.g., filename, web

server address)

  • Author can reference to smaller sections (e.g., track 5)

Behavior Classes:

  • Synchronization of events and user interaction

The action class:

  • Event triggers
  • Sequential and parallel

The link class: establishes relationships between events and objects i.e. what actions to take on what objects in response to a particular event

  • Selection and modification classes:
  • E.g., Push button, checkbox, radio button, slider, text entry field and text

lists

  • Selections, input information and trigger events
slide-43
SLIDE 43

MHEG: Example

(scene:InfoScene1 <other scene attributes here> group-items: (bitmap: BgndInfo content-hook: #bitmapHook

  • riginal-box-size: (320 240)
  • riginal-position: (0 0)

content-data: referenced-content: "InfoBngd" ) (text: content-hook: #textHook

  • riginal-box-size: (280 20)
  • riginal-position: (40 50)

content-data: included-content: "1. Lubricate..." ) links: (link: Link1 event-source: InfoScene1 event-type: #UserInput event-data: #Left link-effect: action: transition-to: InfoScene2 ) )

slide-44
SLIDE 44

STB Evolution

Broadcast: Basic Requirements:

  • Television usage: sit on the

sofa and watch the favorite show at a given time

  • Catch majority of the

population Proposed profile:

  • Audio Visual Content

Digitized

  • Examples: DVB-T/C/S

(Europe), ISDB (Japan), ATSC (USA) Broadcast: Enhanced Requirements:

  • Interactive applications with

internal logic

  • Procedural environment:

Java Proposed profile:

  • DVB-J (as part of GEM,

worldwide accepted version

  • f MHP 1.0)
slide-45
SLIDE 45

STB Evolution

Interactive: Basic Requirements:

  • Browse simple XML

documents

  • Limited navigation
  • Images, text

Proposed profile:

  • XHTML Basic

Interactive: Internet Access Requirements

  • Browse XML Compound

Documents (complex)

  • Interactive
  • Temporal Synchronization
  • Video and audio

Proposed profile:

  • XML Compound Documents

(SMIL + State)

slide-46
SLIDE 46

STB Evolution

Interactive: High-End Requirements

  • Integration of all kinds of

multimedia objects

  • Temporal Dimension

Proposed profile:

  • DVB-J + OpenGL Java

Wrappers

  • JMF media player (all kind of

formats)

slide-47
SLIDE 47

Other Devices

Blu-Ray:

  • DVB - GEM

DVD-HD

  • SMIL-like solution
  • HTML + Time

Mobile phones:

  • MIDP: J2ME
  • HTML/Javascript
  • MMS: restricted SMIL version

IPTV

  • ??
slide-48
SLIDE 48

Metadata: MPEG-7

Uses XML for the textual representation of content description It does not define a monolithic system for content description

  • set of methods and tools for the different domains of the

description of audiovisual content. It is not limited for a specific application type:

  • Other activities such as Dublin Core, TV-Anytime focus on

more specific domains Interactive TV, there is a need to describe the content:

  • Content segmentation (skip commercials)
  • Content searching/indexing
  • Content summaries
  • Content recommendations
slide-49
SLIDE 49

Metadata: TV-Anytime

Consumer device oriented standard for metadata Wide acceptance in TV transmission standards (DVB) TV-Anytime forum for specifying a universal open standard for mass video storage Aims:

  • Enable applications to exploit local persistent storage
  • Network independence (ATSC, DVB, DVB-S)
  • Interoperability: content creators/providers, service providers,

consumers Classification system for describing content and tagging of content for searching and manipulation Two phases:

  • Phase 1: for broadcast and online services – Focus on Personal

Digital Recorder

  • Phase 2: for home networks and micro-navigation - Focus on

Network Digital Recorder

slide-50
SLIDE 50

Metadata: TV-Anytime

Describes:

  • A package as a collection of related content items (programs)
  • A program as a collection of scenes (segments)
  • Different types of content: not only television programs but music for

example Each item can be tagged for choosing, finding, sharing, viewing… TVA defines, for example:

  • Content Reference Identifier (CRID) - a unique identifier similar to a

URL, for a specific piece of video content (it may refer to others CRIDs)

  • User preferences, user consumption habits
  • Descriptors (e.g., title, synopsis, start time)
  • Segmentation metadata: to navigate within a piece of segmented

content Uses MPEG-7 DDL to describe metadata

slide-51
SLIDE 51

Metadata: DVB-H ESG

Data model

  • XML Schema

Instance

  • ESG Data

Representation

  • Fragments

Encapsulation

  • Containers

Transport

  • FLUTE
slide-52
SLIDE 52

Metadata: DVB-H ESG

ESG specification relies on Data types defined by TV-Anytime and MPEG-7 Data model

  • Defines a set of data structures which can be instantiated to

describe available services (based on XML Schema) Instance

  • The data model can be instantiated to describe available services

(actual ESG data) Fragmentation

  • Decompose ESG into self-consistent units of data
  • ESG Fragments can be obtained in random order
  • Each fragment can be transmitted and updated independently

Transport

  • The fragments are encoded in XML, encapsulated into containers

and transmitted over FLUTE/ALC sessions

slide-53
SLIDE 53

Metadata: DVB-H ESG: Data Model

Service:

  • For example a TV channel broadcasted over DVB-H

Event:

  • A particular event that will be broadcasted in a given service
  • Described by the Schedule event (time of broadcast) and the related

contents (e.g., metadata) Acquisition information and the purchase information

  • Provides the means to access these services to the terminal.

Service Bundles

  • Aggregation of services (e.g., purchase options)
slide-54
SLIDE 54

Papers

Goularte at al. (2003). Structuring interactive TV documents.

  • TV-Anytime and MPEG-4
  • Program and Intra-program content selection

Karanastasi et al. (2005). A natural language model for managing TV-Anytime information in mobile environments

  • TV-Anytime
  • Program level management: selection, removal
  • Content can be accessed/ managed using different clients (e.g.,

mobile phone)

  • P. Cesar et al. (2006) Benefits of Structured

Multimedia Documents in IDTV: The End-User Enrichment System

slide-55
SLIDE 55

Papers

DVB Whitepaper (2004) DVB-GEM

  • P. Cesar (2005) A Graphics Software Architecture

for High-End Interactive TV Terminals. (Doctoral Thesis)

  • J. Jones (2002) DVB-MHP Data Transport

Mechanisms

  • S. Morris (2005) Interactive TV Standards: A Guide

to MHP, OCAP and JavaTV

  • C. Peng (2002) Digital Television Applications

(Doctoral Thesis)