Presentation Services in MADEUS: an Authoring Environment for - - PDF document

▶
presentation services in madeus an authoring environment
SMART_READER_LITE
LIVE PREVIEW

Presentation Services in MADEUS: an Authoring Environment for - - PDF document

Jan 15, 2023 373 likes •646 views

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/29622449 Presentation Services in MADEUS: an Authoring Environment for Multimedia Documents Article January 1996 Source: OAI

slide-1
SLIDE 1

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/29622449

Presentation Services in MADEUS: an Authoring Environment for Multimedia Documents

Article · January 1996

Source: OAI

CITATIONS

2

READS

15

3 authors, including: Some of the authors of this publication are also working on these related projects: Structural Health monitoring using Wireless Sensor Networks View project IoT-Based Fire Alarm System View project Loay Sabry Ismail Qatar University

31 PUBLICATIONS 344 CITATIONS

SEE PROFILE

All content following this page was uploaded by Loay Sabry Ismail on 07 January 2014.

The user has requested enhancement of the downloaded file.

slide-2
SLIDE 2

ISSN 0249-6399

apport

  • de recherche

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMA TIQUE

Presentation Services in MADEUS: an Authoring Environment for Multimedia Documents

Muriel Jourdan, Nabil Laya¨ ıda and Loay Sabry-Ismail

N ˚ 2983

TH ` EME 3

slide-3
SLIDE 3
slide-4
SLIDE 4 Presen tation Services in MADEUS: an Authoring En vironmen t for Multimedia Do cumen ts Muriel Jourdan, Nabil La y a
  • da
and Loa y Sabry-Ismail Th
  • eme
3 | In teraction homme-m a c hine, images, donn
  • ees,
connaissances Pro jet Op
  • era
Rapp
  • rt
de rec herc he n2983 | | 22 pages Abstract: The recen t adv ances in m ultim edia systems, together with the adv en t
  • f
high sp eed net w
  • rks,
pa v ed the w a y to a new generation
  • f
applications. In particular, authoring en vironmen ts ha v e found in m ultim edia the means
  • f
increasing the ric hness
  • f
information con tained in electronic do cumen ts. One
  • f
the goals
  • f
the Op era team is designing an authoring en vironmen t for m ulti- media do cumen ts, called MADEUS, whic h meets the follo wing requiremen ts: a high lev el
  • f
expressiv eness for b
  • th
spatial and temp
  • ral
dimensions; a user-friendly in terface allo w- ing highly in teractiv e design pro cess, scriptless and structured-based editing and automatic pro duction
  • f
spatial and temp
  • ral
la y
  • ut;
the p
  • rtabilit
y and cross-platform in terc hange
  • f
m ultimedia do cumen ts. T
  • ac
hiev e this researc h goal w e rst fo cus
  • n
nding a go
  • d
represen tation
  • f
time for m ultimedia do cumen ts. This represen tation is required to capture the temp
  • ral
dimension
  • f
media
  • b
jects lik e video, audio, etc. and is also used to temp
  • rally
  • rganize
  • b
jects with resp ect to eac h
  • ther.
Time represen tation is clearly the main dierence b et w een m ultim edia authoring en vironmen t and traditional editing en vironmen ts. W e presen t in this pap er
  • ur
rst in v estigational results in
  • ur
exp erimen tal authoring en vironmen t MADEUS. A large part
  • f
these results are based
  • n
the exp erience acquired in implemen ting the MADEUS protot yp e Key-w
  • rds:
authoring en vironmen t , time represen tation, time managemen t (R
  • esum
  • e
: tsvp)

Unit´ e de recherche INRIA Rhˆ

  • ne-Alpes

655, avenue de l’Europe, 38330 MONTBONNOT ST MARTIN (France) T´ el´ ephone : (33) 76 61 52 00 – T´ el´ ecopie : (33) 76 61 52 52

slide-5
SLIDE 5 Services de pr
  • esen
tation dans MADEUS : un en vironnemen t d'
  • edition-
pr
  • esen
tation de do cumen ts m ultim
  • edia
R
  • esum
  • e
: L'explosion des capacit
  • es
de traitemen ts m ultim
  • edia
des
  • rdinateurs
rend le b esoin en en vironnemen ts d'
  • edition
de do cumen ts m ultim
  • edia
de plus en plus agran t. L'un des
  • b
jectifs du pro jet Op
  • era
est de concev
  • ir
un tel en vironnemen t qui concilie les trois exigences suiv an tes : un haut degr
  • e
d'expressivit
  • e
p
  • ur
exprimer l'ordonnancemen t temp
  • rel
et l'organisation spatial d'un do cumen t; une in terface d'
  • edition
in teractiv e et d
  • eclarativ
e (pas d'utilisation de scripts), qui prenne en c harge les fonctions de formatage spatial et temp
  • rel;
un en vironnemen t p
  • rtable
et des do cumen ts p
  • uv
an t etre utilis
  • es
par d'autres applications. P
  • ur
cela, nos eorts
  • n
t tout d'ab
  • rd
p
  • rt
  • e
sur la d
  • enition
d'une b
  • nne
repr
  • esen
tation de la dimension temp
  • relle
d'un do cumen t m ultim
  • edia.
Celle-ci doit
  • a
la fois prendre en compte les di
  • eren
tes carat
  • eristiques
temp
  • relles
des
  • b
jets de base d'un do cumen t m ulti- m
  • edia
et les di
  • eren
ts t yp es de comp
  • sition
p
  • ssibles
en tre ces
  • b
jets. De plus, elle doit servir de supp
  • rt
  • a
des m
  • ecanismes
de pr
  • esen
tation ecaces. Nous pr
  • esen
tons dans ce rapp
  • rt
nos premiers r
  • esultats.
Une grande partie de ceux-ci
  • n
t
  • et
  • e
exp
  • erimen
t
  • es
au sein du protot yp e MADEUS. Mots-cl
  • e
: en vironnemen t d'
  • edition,
do cumen ts m ultim
  • edia,
pr
  • esen
tation m ultim
  • edia
slide-6
SLIDE 6 3 1 In tro duction The recen t adv ances in m ultim edia systems, together with the adv en t
  • f
high sp eed net w
  • rks,
pa v ed the w a y to a new generation
  • f
applications. In particular, authoring en vironmen ts ha v e found in m ultim edia the means
  • f
increasing the ric hness
  • f
information con tained in electronic do cumen ts. One
  • f
the goals
  • f
the Op era team is designing an authoring en vironmen t for m ultim edia do cumen ts, called MADEUS, whic h meets the follo wing requiremen ts:
  • a
high lev el
  • f
expressiv eness for b
  • th
spatial and temp
  • ral
dimensions;
  • a
user-frien dl y in terface allo wing highly in teractiv e design pro cess, scriptless and structured-based editing and automatic pro duction
  • f
spatial and temp
  • ral
la y
  • ut;
  • the
p
  • rtabili
t y and cross-platform in terc han ge
  • f
m ultimedia do cumen ts; T
  • ac
hiev e this researc h goal w e rst fo cus
  • n
nding a go
  • d
represen tation
  • f
time for m ultimedia do cumen ts. This represen tation is required to capture the temp
  • ral
dimension
  • f
media
  • b
jects lik e video, audio, etc. and is also used to temp
  • rally
  • rganize
  • b
jects with resp ect to eac h
  • ther.
Time represen tation is clearly the main dierence b et w een m ultim edia authoring en vironmen t and traditional editing en vironmen ts. W e presen t in this pap er
  • ur
rst in v estigational results in
  • ur
exp erimen tal authoring en vironmen t MADEUS. A large part
  • f
these results are based
  • n
the exp erience acquired in implemen ting the MADEUS protot yp e. More precisely , the rst section is dev
  • ted
to the analysis
  • f
what are the requiremen ts for a go
  • d
represen tation
  • f
time in m ultim edia do cumen ts. A t the end
  • f
this section, w e presen t the approac h used in MADEUS whic h is based
  • n
an extension
  • f
temp
  • ral
constrain t net w
  • rks.
In the second section, w e fo cus
  • n
the design and arc hitecture issues
  • f
the MADEUS application whic h handles b
  • th
authoring and presen tation stages
  • f
m ultim edia do cumen ts. 2 Represen tation
  • f
time in m ultimedia do cumen ts 2.1 Time represen tation requiremen ts In
  • rder
to measure the qualit y
  • f
a giv en time represen tation used to mo del the temp
  • ral
  • rganization
  • f
m ultimedia do cumen ts and to ev aluate comparativ ely those used in dieren t do cumen t systems [2 ][3 ][12 ], w e need to dene some criteria. In what follo ws, w e presen t k ey p
  • in
ts that, in
  • ur
  • pinion,
aect signican tly the qualit y
  • f
the temp
  • ral
represen tation
  • f
m ultimedia do cumen ts. Expressiv eness The time represen tation m ust capture the dieren t temp
  • ral
c haracte- ristics
  • f
basic
  • b
jects . Media
  • b
jects ma y b e static lik e text, still images, etc.,
  • r
dynamic lik e video, audio, in teraction, etc. Dynamicit y is tak en here in the sense that these
  • b
jects RR n2983
slide-7
SLIDE 7 4 ha v e a notion
  • f
a presen tation activit y
  • v
er time. A video
  • r
an audio are activ e when they are pla y ed. An in teraction, lik e a button, is activ e when it is enabled so that it is p
  • ssible
to in teract with it through the user in terface. Dynamic
  • b
jects ha v e an asso ciated duration where t w
  • cases
m ust b e distinguished according to their temp
  • ral
b eha viour:
  • Con
trollabl e
  • b
jects: a range
  • f
p
  • ssible
durations is kno wn at the authoring stage and it is p
  • ssible
to c ho
  • se
an y v alue within this range during the presen tation stage (audio, video, ... ).;
  • Uncon
troll abl e
  • b
jects: their durations are
  • nly
kno wn at the end
  • f
their presen- tation (in teraction
  • b
jects, uncon trollable program executions lik e applets). The time represen tation m ust also capture the dieren t w a ys to temp
  • rally
  • rganize
  • b
jects [16 ][13 ]. Firstly , it should b e p
  • ssible
to distinguish four temp
  • ral
instan ts when dealing with dynamic
  • b
jects: when they are mapp ed (unmapp ed) at the screen (referred in the sequel as spatio-temp
  • ral
instan ts) and when they start (nish) pla ying (referred in the sequel as temp
  • ral
instan ts). Static
  • b
jects are
  • nly
aected b y the spatio-temp
  • ral
instan ts. Secondly ,
  • p
erators whic h express the temp
  • ral
  • rganization
  • f
the do cumen t m ust supp
  • rt
logical time: relativ e p
  • sitioning
  • f
  • b
jects (as expressed b y the thirteen temp
  • ral
relations in tro duced b y Allen [1 ]); quan titativ e time: n umerical constrain ts placed
  • n
  • b
ject's duration and causal time: ho w instan ts causes
  • ther
presen tation action to happ en (the end instan t
  • f
  • ne
  • b
ject causes the termination
  • f
another). F
  • r
instance, the follo wing sp ecication (scenario 1)
  • f
a temp
  • ral
  • rganization
m ust b e supp
  • rted
b y the time represen tation : Map the two vide
  • s
A and B 2 se c
  • nds
after the start
  • f
the do cument; Start playing vide
  • A
2 se c
  • nds
after b eing mapp e d; Play vide
  • B
in p ar al lel to vide
  • A;
Unmap the two vide
  • s
when they ar e nishe d; Start playing audio C when vide
  • B
starts to play; Play vide
  • D
after vide
  • A;
Map D when it is playe d; Unmap it at the end
  • f
the do cument; Start an inter action I when D starts; The end
  • f
vide
  • D
c auses the end
  • f
audio C if not yet nishe d; The rst
  • ne
that nishes b etwe en D and I, kil ls the
  • ther
  • ne.
The dur ation
  • f
the do cument must b e in the r ange b etwe en 20 and 30 se c
  • nds.
The four
  • b
jects A, B, C and D ha v e the follo wing temp
  • ral
c haracteristics, dened b y a tuple (min, d, max) where min and max are resp ectiv ely the minim um and the maxim um durations
  • f
the
  • b
ject, and d is the preferable
  • ne
in the con text
  • f
a giv en do cumen t: A: (4,6,8), B: (2,4,6), C : (10,11,12), D: (4,6,8). Finally , the time represen tation m ust b e able to deal with temp
  • ral
h yp erlinks whic h giv es the do cumen t readers the abilit y to na vigate freely either inside the do cumen t
  • r
  • utside
it. Hyp erlinking breaks the linearit y
  • f
planned scenarios and allo ws seman tical exploration
  • f
the do cumen t con ten t b ey
  • nd
its structure. In terface with a declarativ e and hierarc hic sym b
  • lic
represen tati
  • n
Multimedia do cumen t authors do not necessarily ha v e an y particular skills in computer programming, so the editing phase m ust b e v ery in tuitiv e and as close as p
  • ssible
to the famili ar editing st yle INRIA
slide-8
SLIDE 8 5
  • f
classical do cumen ts. This goal is
  • b
viously hard to
  • btain
when using scripting languages where the do cumen t arc hitecturing sc heme (manipulation
  • f
the do cumen t en tities through a set
  • f
asso ciated attributes) disapp ears in fa v
  • ur
  • f
the programming
  • ne.
Dening global attributes
  • n
comp
  • site
  • b
jects lik e color, fon t, p
  • sition,
etc. and applying inheritance rules
  • n
these attributes reduce the burden
  • n
do cumen t sp ecication, increase the reusabilit y and hide lo w-lev el presen tation details. Existing do cumen t mo dels based
  • n
declarativ e st yle [6 ] and descriptiv e mark-up
  • f
its con ten t are more and more widely used. W e think that this is the b est approac h in
  • rder
to disseminate time-based m ultimedia do cumen ts. Moreo v er, it is also suitable for direct manipulation authoring in terfaces. The sym b
  • lic
represen tation
  • f
time in m ultim edia do cumen ts m ust supp
  • rt
structural comp
  • sition[10
]. This giv es the author the abilit y to
  • rganize
the do cumen t in nested comp
  • nen
ts in
  • rder
to divide its con ten t in logical parts[14]. It helps him also in fo cusing
  • n
an isolated part
  • f
the do cumen t during eac h step
  • f
the authoring pro cess (sub-scenes
  • f
a m ultimedia scenario). Ecien t and incremen tal manipulat ion
  • f
the do cumen t The manipulation
  • f
the temp
  • ral
structure
  • f
a m ultim edia do cumen t in an authoring en vironmen t m ust b e supp
  • r-
ted b y ecien t algorithms and data structures in
  • rder
to b e really in teractiv e. Moreo v er, giv en the progressiv e nature
  • f
the editing pro cess and the accesses across the net w
  • rk
to the do cumen t con ten t [17 ], the temp
  • ral
  • rganization
  • f
a do cumen t m ust b e suitable for an incremen tal building pro cess. The critical parts
  • f
an authoring en vironmen t regarding the time p erformances are:
  • parsing
phase: translation
  • f
the sym b
  • lic
represen tation
  • f
the temp
  • ral
structure
  • f
the do cumen t in to the in ternal and executable
  • ne
used b y the presen tation la y er
  • f
an application engine;
  • consistency
c hec king : as dened in [11 ], three kinds
  • f
temp
  • ral
inconsistencies ma y
  • ccur
in a do cumen t: qualitativ e, quan titativ e and indeterministic. Inconsisten- cies ma y also
  • ccur
from the relativ e p
  • sitioning
  • f
spatio-temp
  • ral
instan ts according to temp
  • ral
  • nes.
F
  • r
instance a dynamic
  • b
ject can not b e started b efore b eing mapp ed
  • n
the screen. All kinds
  • f
inconsistencies ha v e to b e detected at the autho- ring stage and the parsing
  • ne
[11 ]. This c hec king phase prev en ts the author from writing inconsisten t temp
  • ral
sp ecications and the presen tation la y er from executing inconsisten t scenarios.
  • edition/p
resen t ati
  • n
cycle: the transition from the editing phase to the presen ta- tion
  • ne
whic h aects the dev elopmen t
  • f
in tegrated to
  • ls;
  • presen
tation monitorin g: managemen t
  • f
the presen tation phase whic h handles la y
  • ut
  • p
erations lik e graphics rendering, net w
  • rk
access, temp
  • ral
access con trol
  • p
erations lik e pause, resume, stop and fast-forw ard, temp
  • ral
na vigation through h yp erlinks, etc. RR n2983
slide-9
SLIDE 9 6 P
  • rtabili
t y and cross-platform do cumen ts in terc hange The represen tation
  • f
time in m ultim edia do cumen ts m ust not mak e particular assumptions ab
  • ut
the format
  • f
the basic
  • b
jects (video, audio, text, etc.), the
  • p
erating system
  • f
the target mac hine lik e m ulti- threaded pro cessing, the presence
  • f
dedicated m ultim edia hardw are, proprietary media
  • b
ject formats, etc. This is the necessary condition to meet the p
  • rtabilit
y requiremen ts
  • f
m ultimedia do cumen ts in terc hange format [3 ]. Moreo v er, the represen tation
  • f
time m ust b e suc h that do cumen ts in terc hange b et w een dieren t en vironmen ts b ecomes easy . This is the aim
  • f
high-lev el sym b
  • lic
formalism s as Hytime [7 ], MHEG [9 ] and more recen tly PREMO [8 ]. Do cumen ts written in these formats are preserv ed along time and are reusable b y dieren t p eople and applications. W e ha v e seen in this section a set
  • f
requiremen ts whic h serv es as basis and guidelines for the c hoice
  • f
a time represen tation for m ultim edia do cumen ts. In the t w
  • follo
wing sections, w e presen t the solution tak en in
  • ur
pro ject, its adv an tages and an analysis
  • f
particular solutions used in
  • ther
w ell kno wn systems lik e the Timeline editors[5 ], OCPN[12 ] and CMIFED[3]. 2.2 Time represen tation in MADEUS: an extension
  • f
temp
  • ral
constrain t net w
  • rks
The in ternal represen tation
  • f
time in MADEUS is based
  • n
the temp
  • ral
constrain t net- w
  • rks
formalism dened in the articial in telligence area [4 ]. W e use this formalism during the authoring stage in
  • rder
to detect easily inconsistencies in a rst step and to propagate n umerical constrain ts in a second
  • ne
[11 ]. Constrain t net w
  • rks
are p
  • w
erful b ecause they are based
  • n
a data structuring
  • f
the temp
  • ral
constrain ts as a graph. This graph matc hes the data structuring required b y the presen tation stage to monitor the presen tation. An in termediate
  • p
eration is applied b et w een the consistency c hec king and the constrain t propagation stages whic h is called static temp
  • ral
pre-formatting. This
  • p
eration is ne- cessary b ecause constrain t propagation deals
  • nly
with minim um and maxim um v alues
  • f
allo w able durations (a set
  • f
solutions for the scenario). The pre-formatting phase tak es in to accoun t dela ys and
  • b
jects shrinking and stretc hing capabilities in
  • rder
to nd par- ticular solutions. This pro cess uses a constrain t distribution p
  • licy
to balance constrain ts b et w een in terv als taking in to accoun t the most suitable duration
  • f
the
  • b
jects. Because durations
  • f
uncon trollable
  • b
jects are
  • nly
determined during the presen tation stage, a dynamic formatting is also needed [2]. This last p
  • in
t will b e explained in detail in section 3.2.3. 2.2.1 T emp
  • ral
Constrain t net w
  • rks
A temp
  • ral
constrain t net w
  • rk
is a Directed Acyclic Graph (D A G) where no des represen t time p
  • in
ts and an edge (i, [min,max], j) represen ts a temp
  • ral
in terv al with a duration range from min to max b et w een the t w
  • time
p
  • in
ts i and j. The top
  • logy
  • f
this D A G is
  • rganized
as a set
  • f
temp
  • ral
c hains. A temp
  • ral
c hain [e 1 , e 2 , .., e n ] is a sequence
  • f
con tiguous edges where eac h edge e j , 1 < j < n is
  • nly
INRIA
slide-10
SLIDE 10 7

[0, ] 8 8 [1, ] [4,6] [4,6] [0, ] 8 A B [4,8] D

Figure 1: Constrain t net w
  • rk
  • f
the scenario 2 related to
  • ne
successor and
  • ne
predecessor so that b egin (e j ) = end (e j1 ) and end (e j ) = b egin (e j+1 ), ev ery single in terv al con trollable
  • r
uncon trollable. A temp
  • ral
constrain t net w
  • rk
describ es the temp
  • ral
  • rdering
  • f
media
  • b
jects b y cap- turing an information ab
  • ut
their logical and quan titativ e temp
  • ral
dep endencies. The range
  • f
p
  • ssible
durations
  • f
a dynamic
  • b
ject is mo deled b y an edge lab eled b y its appropriate in terv al. F ree
  • r
quan tied dela ys are used to measure the temp
  • ral
metric distance b et w een
  • b
jects
  • r
to enforce precedence relations b et w een them. F
  • rmally
, the class
  • f
problems mo deled b y this formalism is called Simple T emp
  • ral
Problem [15 ]. F
  • r
instance, the fol- lo wing scenario (scenario 2) can b e managed during the authoring stage b y the temp
  • ral
constrain t net w
  • rk
sho wn in gureFig. 1. Play vide
  • B
in p ar al lel to vide
  • A;
Play vide
  • D
after vide
  • A
In this gure in terv al
  • b
jects are represen ted b y b
  • ld
lines and dela y in terv als b y thin
  • nes.
A and B duration ranges ha v e b een mo died to satisfy the sp ecication. 2.2.2 Extended temp
  • ral
constrain t net w
  • rks
In MADEUS, the previous denition
  • f
a constrain t net w
  • rk
has b een extended to meet the requiremen ts in tro duced ab
  • v
e:
  • an
additional lab elling
  • f
in terv al whic h indicates whether the temp
  • ral
distance is con trollable
  • r
not (resp ectiv ely [min,m ax] c and [min,m ax] u ). This is the w a y w e ma- nage indeterministic temp
  • ral
b eha viour as explained latter in the dynamic formatting section.
  • causalit
y relations b et w een temp
  • ral
instan ts
  • n
eac h no de to handle master-sla v e relationships b et w een media streams.
  • a
set
  • f
sp atio-temp
  • r
al actions (map and unmap actions)
  • n
eac h no de to con trol the instan ts when the dieren t
  • b
jects app ear
  • r
disapp ear from the screen indep enden tly from their pla y time. This
  • p
ens the ro
  • m
for the dev elopmen t
  • f
spatio-temp
  • ral
comp
  • sition
and languages whic h are still lac king in m ultimedia authoring systems.
  • basic
  • b
jects
  • f
t yp e h yp erlinks ha v e b een added as temp
  • ral
in terv als corresp
  • nding
to the p erio d
  • f
time where their activ ation is enabled. This represen tation
  • f
h yp erlinks RR n2983
slide-11
SLIDE 11 8

8 [1, ] [4, 6] c u [5, 12] 8 [0, ]c [4,6] c [20, 30] c c [2,2] [2,2] c [0, 8] B A I D [0, 8] u c u C Causality relations endD => endC, endD => endI, endI => endD Spatio-temporal actions mapA, mapB unmapA, unmapB mapD unmapD

Figure 2: The constrain t net w
  • rk
  • f
scenario 1 is homogeneous with the represen tation
  • f
the
  • ther
t yp es
  • f
  • b
jects in the temp
  • ral
constrain t net w
  • rk.
In addition to this constrain t net w
  • rks
structure, w e dene a hierarc hical structure
  • f
nested comp
  • nen
ts where lea v es are basics media
  • b
jects and no des are comp
  • site
  • b
jects. In fact, eac h comp
  • site
  • b
ject is dened b y an extended constrain t net w
  • rk
built from its enclosing comp
  • nen
ts b y a set
  • f
relational
  • p
erators. It is w
  • rth
noting that the hierarc hi- cal
  • rganization
  • f
the do cumen t do es not restrict the degree
  • f
expressiv eness
  • f
temp
  • ral
scenarios. As a result
  • f
the
  • rthogonalit
y
  • f
the temp
  • ral
dimension with resp ect to the logical
  • rganization,
media
  • b
jects can b e in v
  • lv
ed in sev eral relations. Because
  • ne
can not predict ho w
  • b
jects are inserted in the scenario, the hierarc hical structure denes temp
  • ral
b
  • undaries
to sub-comp
  • nen
ts serving as an
  • v
erall b egin and end
  • f
the sub-scene. The net w
  • rk
asso ciated with comp
  • site
  • b
jects constitutes the main building blo c k
  • f
  • ur
in ter- nal represen tation
  • f
a m ultim edia do cumen t. Eac h comp
  • site
represen ting a lev el
  • f
the hierarc h y denes a new lev el
  • f
constrain t net w
  • rk.
Through this h yp er-graph, temp
  • ral
constrain ts can b e propagated up-stream
  • r
do wn-stream dep ending
  • n
whether the editing
  • p
erations mo dify the temp
  • ral
parameters
  • f
a basic
  • b
ject
  • r
a comp
  • site
  • b
ject as a whole. The extended constrain t net w
  • rks
asso ciated with the scenario 1 during the authoring stage is giv en in Fig. 2. As describ ed in the example, some in terv als
  • f
this scenario b ecome uncon trollable b y inheritance (lik e C and D due to their in teraction with I). This scenario is temp
  • rally
consisten t since it is p
  • ssible
to nd particular duration v alues
  • f
con trollable in terv als, suc h that, whatev er v alue the duration
  • f
the uncon trollable in terv al I tak es at the presen tation stage, the last con trollable in terv al can b e dynamically adjusted to comp ensate the
  • bserv
ed indeterminism [11 ]. INRIA
slide-12
SLIDE 12 9 2.2.3 Ev aluation In this section, w e presen t the adv an tages
  • f
the extended temp
  • ral
constrain t net w
  • rks
as a c hoice for a b etter time represen tation. These adv an tages are ev aluated regarding the requiremen ts that w e ha v e in tro duced earlier. Expressiv eness Our time represen tation is able to deal with static and dynamic
  • b
jects, con trollable and uncon trollable
  • nes.
It tak es in to accoun t the spatio-temp
  • ral
comp
  • sition
b y lab elling the net w
  • rk
no des b y spatio-temp
  • ral
mapping actions. The set
  • f
the thir- teen primitiv e relations in tro duced b y Allen are fully supp
  • rted
and extended to tak e in to accoun t the quan titativ e asp ect
  • f
the relations. The p
  • in
tizable time algebra [15 ] is fully implemen ted to cop e with arbitrary complex temp
  • ral
presen tations, and the set
  • f
temp
  • ral
  • p
erators can b e redened without mo difying the temp
  • ral
managemen t la y er [16 ]. It is also p
  • ssible
to express causalit y relations b et w een
  • b
jects lik e parallel presen tation
  • f
t w
  • b
jects where the rst to nish terminates the
  • ther
  • ne.
Moreo v er, h yp erlinks (na vigation supp
  • rt)
and user in teractions are in tegrated in the temp
  • ral
language in a homogeneous w a y as in terv als. In terface with a declarativ e and hierarc hi c sym b
  • lic
represen tati
  • n
Our time represen tation is in terfaced with a declarativ e and hierarc hic sym b
  • lic
relation based
  • n
a set
  • f
relational
  • p
erators (start, meet, equal, par-min, kills, ...) as illustrated b y Fig. 3. A comp
  • site
  • b
ject is dened b y three parts: a declaration part describing the enclosed
  • b
jects together with their attributes lik e colors, fon ts, motion st yle and video eects. A se- cond part describing temp
  • ral
relations whic h expresses the
  • rdering
b et w een these
  • b
jects. Finally , a third part describ es the spatio-temp
  • ral
actions
  • f
the dieren t
  • b
jects. All these sp ecications can b e easily p erformed through the user in terface b y selecting
  • b
jects and the temp
  • ral
  • r
spatio-temp
  • ral
relations b et w een them. Ecien t and incremen tal algorithms Our time represen tation is handled b y an incre- men tal editing pro cess [11 ]. The consistency c hec king is ac hiev ed b y three dieren t metho ds dep ending
  • n
the kind
  • f
temp
  • ral
inconsistencies:
  • static:
a top
  • logical
sort
  • f
the constrain t net w
  • rk
is main tained in
  • rder
to a v
  • id
this kind
  • f
inconsistency b y detecting cycles.
  • quan
titativ e : b y using constrain ts propagation tec hniques, eac h time the net w
  • rk
is mo died, w e c hec k that all c hains parallel to eac h
  • ther
still ha v e non-empt y in tersec- tion
  • f
their allo w able duration ranges. Algorithms used for the constrain t propagation are p
  • lynomial
  • nes
[15 ]. Moreo v er, they tak e adv an tage
  • f
the top
  • logy
  • f
the graph
  • rganized
as temp
  • ral
c hains to ac hiev e lo cal propagation and fast up dating tec h- niques.
  • indetermi
ni sti c: a c hec k is applied statically to see if it is p
  • ssible
to reco v er dy- namically , at the presen tation stage, the indeterministic b eha viour
  • f
uncon trollable RR n2983
slide-13
SLIDE 13 10

A equals B D kills C Scenario1 <DOC> <COMP>Scene1 <DECL> <TEMP_REL> <SPATIO_TEMP_REL> ... ... ... ... ... <OBJECT> <NAME> <MOTIONSTYLE> <POS> <FORMAT> A 252,108 1 <URL> <TEMP_REL> ... <SPATIO_TEMP_REL> http://opera.inrialpes.fr/Logo.Mpeg1 </OBJECT> </DECL> </TEMP_REL> </SPATIO_TEMP_REL> </COMP> </TEMP_REL> </SPATIO_TEMP_REL> </DOC> {right_to_left : 150 pt,0} A.start maps B

Figure 3: MADEUS graphic user in terface and sym b
  • lic
do cumen t represen tation in terv als. This
  • p
eration is p erformed to ensure the dynamic formatting whic h is based
  • n
an
  • bserv
e and repair approac h
  • f
the indeterminism (see section 3.2.3). Another adv an tage w e pro vide with
  • ur
in ternal represen tation
  • f
the do cumen t is that it is suitable as a supp
  • rt
for b
  • th
edition and presen tation supp
  • rt.
Th us, the MADEUS application engine do es not require an y structural translation b et w een the edition phase and the presen tation
  • ne.
P
  • rtabili
t y and cross-platform do cumen t in terc han ge One
  • f
the main adv an tages
  • f
  • ur
time represen tation is that it is in terfaced with a sym b
  • lic
declarativ e represen tation whic h can b e directly used as an in terc hange format. Also, it can b e easily translated in an y
  • ther
format used b y another application giv en its higher lev el
  • f
expressiv eness. 2.3 Analysis
  • f
  • ther
time represen tations Timeline editors lik e Director [5 ], Ob ject Comp
  • sition
P etri Nets [12 ] and CMIFED [3 ] time represen tations are analyzed regarding ho w they meet the requiremen ts presen ted in the previous section. First
  • f
all, let us note that none
  • f
them distinguishes clearly the spatio- temp
  • ral
asp ects
  • f
a m ultimedia presen tation from the temp
  • ral
  • nes.
Th us the kind
  • f
p
  • ssible
written scenarios is limited b y comparison with
  • ur
approac h. INRIA
slide-14
SLIDE 14 11
  • Timeline
editors: the expressiv eness
  • f
timeline is not v ery ric h since it do es not cap- ture the shrink and stretc h capabilities
  • f
dynamic
  • b
jects, uncon trollable
  • b
jects and h yp erlinks are not easy to in tegrate (use
  • f
a scripting language) and
  • nly
quan titativ e dating sc heme is supp
  • rted.
In terfaces based
  • n
timeline are rather in tuitiv e but not scalable for large do cumen ts. The lac k
  • f
structure is also
  • ne
  • f
the dra wbac ks
  • f
timeline editors. It puts a hea vy burden
  • n
the authoring pro cess and is error prone since an y mo dication
  • f
the scenario ha v e to b e propagated b y hand to the rest
  • f
the do cumen t. Ho w ev er, the simplicit y
  • f
the time represen tation eases the application construction. The do cumen t in terc hange requiremen ts are not satised since an ex- haustiv e dating
  • f
the
  • b
jects presen tation do es not capture their m utual relationships (there is no logical time).
  • OCPN:
this mo del handles logical and quan titativ e time but do es not supp
  • rt
the causal
  • ne.
Ho w ev er, from
  • ur
understanding it is not stated ho w n umerical constrain ts are managed and propagated. Uncon trollable
  • b
jects and h yp erlinks are not supp
  • r-
ted. Graphical authoring in terface based
  • n
p etri-nets allo wing hierarc hical structuring can b e dened. Ho w ev er, this temp
  • ral
mo delling
  • f
m ultimedia do cumen ts is not ap- propriate as an authoring paradigm. The temp
  • ral
alignmen t
  • f
media
  • b
jects and the do cumen t paradigm (en tit y attribute editing) disapp ear in fa v
  • ur
  • f
the transitional asp ect
  • f
p etri-nets.
  • CMIF
ed represen tation
  • f
time:
  • nly
t w
  • causal
time
  • p
erators are dened in CMI- F ed, the sequence
  • p
erator and the parallel
  • ne
where the last to nish ends the temp
  • ral
construction. The CMIF ed editor has a user friendly in terface based
  • n
a lo- gical structuring
  • f
the do cumen t as a hierarc h y . Ho w ev er, this hierarc h y is mixed with the temp
  • ral
structure
  • f
the do cumen t where eac h comp
  • site
elemen t is dened b y a temp
  • ral
  • p
erator. Therefore, the temp
  • ral
  • rganization
  • f
the do cumen t is restricted to a tree structure, whic h together with the exclusiv ely causal represen tation
  • f
time, limits the n um b er
  • f
expressible scenarios. T
  • v
ercome this problem, sync hronization arcs ha v e b een added but inconsistencies can b e easily in tro duced. A recen t eort has b een undertak en to meet the p
  • rtabilit
y requiremen ts and a translation
  • f
CMIF ed do cumen ts in to HyTime has b een successfully ac hiev ed. 3 Presen tation la y er in MADEUS In this section, w e fo cus
  • n
the design and arc hitecture issues
  • f
the MADEUS application whic h handles b
  • th
authoring and presen tation stages
  • f
m ultim edia do cumen ts. P arti- cularly , w e detail the presen tation services related to the sc heduling issues, namely the sc heduler and the execution mediator lev els, whic h are based
  • n
the hierarc h y
  • f
extended temp
  • ral
constrain t net w
  • rks.
RR n2983
slide-15
SLIDE 15 12

Parser Translator Consistency checker Static formatter Scheduler Context manager Execution mediator Protocol manager Layer Specification Layer Presentation Level of Abstraction Internal Symbolic Representation Hierarchical structure + set of extended constraint networks Representation GUI Hardware device Object manager

Figure 4: General arc hitecture
  • f
MADEUS system 3.1 Arc hitecture
  • v
erview
  • f
MADEUS The general arc hitecture
  • f
the MADEUS system as sho wn in Fig. 4, dep ends
  • n
dieren t lev els
  • f
abstraction. The highest lev el is the sp ecication lev el, whic h furnishes the system with a simple declarativ e sp ecication language
  • f
high p
  • w
er
  • f
expressiv eness to sp ecify the required temp
  • ral
relationships b et w een dieren t m ultim edia
  • b
jects. The resulting set
  • f
declared m ultimedia
  • b
jects together with the temp
  • ral
relations b et w een them, constitute a sym b
  • lic
represen tation
  • f
the m ultim edia do cumen t. A parser is used to transform the sym b
  • lic
represen tation sho wn in Fig. 3 in
  • rder
to
  • btain
the in ternal represen tation starting from whic h w e can c hec k consistency and monitor the presen tation
  • f
the m ultimedia do cumen t. The author can incremen tally edit and mo dify the sym b
  • lic
represen tation
  • r
the in ternal represen tation
  • f
the do cumen t via a Graphical User In terface (GUI). In addition, there is atranslator whose role is to con v ert the in ternal represen tation in to the sym b
  • lic
  • ne
whic h is the format in whic h the do cumen t will b e stored
  • n
the disk. The sc heduler lev el (furtherly describ ed in section 3.2) handles the sync hronization b et- w een temp
  • rally
related instan ts
  • f
dieren t m ultim edia
  • b
jects, lik e for example the start INRIA
slide-16
SLIDE 16 13
  • f
the presen tation
  • f
the whole do cumen t, the co-starting
  • f
a n um b er
  • f
in terv als sim ul- taneously , the causalit y relationships b et w een concurren t
  • b
jects, user na vigation through the h yp erlinks, etc. The execution mediator lev el (furtherly describ ed in section 3.3) handles the state transitions
  • f
the m ultim edia
  • b
jects as needed b y the sc heduler and requests the cor- resp
  • nding
actions to b e done b y the executional lo w er lev el. This lev el guaran tees the indep endence
  • f
the implemen tatio n la y er. Also, it giv es a feedbac k to the sc heduler ab
  • ut
the curren t state
  • f
the m ultim edia
  • b
jects during their presen tation life time. The
  • b
ject lev el dieren tiates b et w een the dieren t t yp es
  • f
  • b
jects while they are hidden in all
  • f
the ab
  • v
e lev els. If a certain action is to b e done
  • n
an
  • b
ject, then the w a y
  • f
ac hieving it will b e dep enden t
  • n
the t yp e
  • f
this
  • b
ject and its implem en tation, so the
  • b
ject lev el helps in pro v
  • king
the righ t pro cedure to execute this action based
  • n
the
  • b
ject's t yp e. Also, this lev el manages the la y
  • ut
  • f
the m ultim edia
  • b
jects
  • n
their suitable presen tation devices, suc h as audio c hannels, screen, etc. The net w
  • rk
proto col lev el handles the data access
  • f
m ultimedia
  • b
jects. Firstly , it is to b e sp ecied if the
  • b
ject to b e accessed is lo cal
  • r
remote. This is ac hiev ed b y a URL (Uniform Ressource Lo cator) lik e
  • b
ject naming sc heme where the proto col app ears as a prex. As some media
  • b
jects can tolerate loss
  • f
data lik e video and audio VDP , R TP and UDP based proto cols are more adapted, while
  • thers
cannot tolerate this loss lik e the source do cumen t represen tation, text, image and graphics so HTTP whic h is a TCP/IP based proto col is used. Whether the m ultim edia
  • b
ject is lo cal
  • r
remote, in b
  • th
cases the
  • b
ject is accessed via a stream
  • f
data managed indep enden tly from the
  • b
ject lev el. The net w
  • rk
lev el manages
  • p
erations lik e
  • p
ennig, closing, buering the net w
  • rk
connections and callbac ks for activities in the
  • b
ject lev el. 3.2 Sc heduler The sc heduler can b e considered as the maestro
  • f
the presen tation phase
  • f
a m ultim edia do cumen t. It do es not
  • nly
handle the temp
  • ral
sync hronization b et w een m ultimedia
  • b-
jects, but it handles the dynamic comp ensation for indeterminism as w ell. It also handles temp
  • ral
h yp erlinks and references b y main taining con texts
  • f
the do cumen t's presen tation. 3.2.1 Presen tation con text and presen tati
  • n
stac k A presen tation con text, at a giv en instan t, is the information ab
  • ut
the curren t activities
  • f
a running presen tation. The presen tation con text is main tained in the form
  • f
a list
  • f
activit y iden tiers. If w e imagine that w e ha v e a presen tation as sho wn b y the graph in Fig. 5, and w e mak e a temp
  • ral
logical cut in this presen tation represen ted b y the curv ed line (sa y at time t). Then the presen tation con text will con tain information ab
  • ut
the activities
  • f
the curren tly activ e
  • b
jects (A, B, C, and D). Also w e can use a presen tation stac k in
  • rder
to switc h b et w een dieren t presen tation con texts b y storing the
  • ne
to b e susp ended in the stac k (after susp ending all its activities). This facilit y enables us to free the resources allo cated b y a susp ended presen tation con text as w e ha v e all the information RR n2983
slide-17
SLIDE 17 14

A B C D t t X (composite)

Figure 5: A temp
  • ral
logical cut in a m ultim edia presen tation needed to reallo cate resources in
  • rder
to resume this con text at an y time (e.g. the curren t frame n um b er for the video). This kind
  • f
simple c hec k p
  • in
ts co v ers most
  • f
the cases except for program executions (applets) where it is dened b y the system run-time stac k itself. 3.2.2 Sc hedulin g principl es The sc heduler uses the in ternal represen tation to monitor the do cumen t's presen tation. A t ev ery no de in the constrain t net w
  • rk
b eing reac hed b y an ending in terv al, the sc heduler ap- plies a simple and straigh tforw ard rule ha ving the follo wing statemen t: "if all the incoming in terv als
  • f
this no de ha v e reac hed their end, then execute all the spatio-temp
  • ral
actions and start all the
  • utgoing
in terv als". This kind
  • f
sc heduling is done b y the help
  • f
sync hro- nization coun ters (at eac h no de) for (1) the n um b er
  • f
incoming and
  • utgoing
in terv als at this no de, and (2) the n um b er
  • f
in terv als remaining to terminate at this no de during the presen tation. The v alues
  • f
these coun ters and the related in terv als are stored in the no de structure as sho wn in Fig. 6. Also in this structure, w e main tain tables represen ting the kind
  • f
dep endencies (equalit y and causalit y) b et w een instan ts
  • f
the incoming and
  • utgoing
in- terv als at this no de. The equalit y dep endency
  • f
instan ts implies their temp
  • ral
coincidence whic h m ust b e v eried b y the temp
  • ral
consistency c hec king phase. F
  • r
example in Fig. 6, the end instan t
  • f
B coincides with start instan t
  • f
in terv al D, and the end instan t
  • f
A coincides with that
  • f
C. The causalit y dep endency
  • f
instan ts implies that the
  • ccurrence
  • f
an instan t leads to the
  • ccurrence
  • f
the
  • ther
instan t. The eld called timestamp and its role in dynamic comp ensation for indeterminism, will b e explained in a latter section. The do cumen t's presen tation starts from the starting no de
  • f
the graph at the ro
  • t
  • f
the hierarc hical structure. A comp
  • site
  • b
ject is started b y recursivly starting its comp
  • nen
ts un til basic media
  • b
jects. It is ended when it receiv es end signals from all its comp
  • nen
ts. INRIA
slide-18
SLIDE 18 15

A B C D Synchronization counters B + A + B +

  • D

+ A + C = = Incoming Outgoing Remains 3 {A,B,C} 1 {D} 3 {A,B,C} Equality Causality table table Timestamp <Flx,Ind,Date> Spatio- temporal actions Unmap A ; Map D ; .....

Figure 6: The structure
  • f
a no de in the constrain t net w
  • rk
3.2.3 Dynamic formatting One problem that arises in the presen tation
  • f
a m ultim edia do cumen t is the abilit y to handle the temp
  • ral
indeterminism that
  • ccurs
during the presen tation. In
  • rder
to deal with these situations and to meet the constrain ts sp ecied in the scenarios, certain information ab
  • ut
temp
  • ral
exibilit y , indeterminism and ev en t dates, m ust b e a v ailable at ev ery end-of-pla y instan t for ev ery in terv al.This information is called the time stamping
  • f
instan ts. These instan ts corresp
  • nd
to the no des
  • f
the temp
  • ral
constrain t net w
  • rks.
The time stamp information at eac h no de can b e represen ted in the form
  • f
a tuple <FLX, IND, D A TE>, where D A TE is the date
  • f
termination
  • f
the last incoming in terv al at this no de, FLX is the amoun t
  • f
exibilit y a v ailable that can b e pro vided b y the
  • utgoing
in terv al
  • f
this no de and along a particular temp
  • ral
c hain, and IND is the amoun t
  • f
uncomp ensated indeterminism in tercepted in the curren t c hain un til the instan t D A TE. The v alue
  • f
indeterminism
  • f
an uncon trollable
  • b
ject is the dierence b et w een its eectiv e duration and its static
  • ne
determined b y the static pre-formatter. The eld D A TE is calculated with resp ect to either a global do cumen t's presen tation clo c k that measures time from the starting instan t
  • f
the presen tation
  • f
the do cumen t
  • r
simply a time-of-da y clo c k. This clo c k is examined
  • nly
when an ev en t happ ens in the scenario so that the corresp
  • nding
c hain's clo c k can b e c hec k ed against the clo c ks
  • f
the concurren t c hains. This particular p
  • in
t
  • f
time represen ted b y
  • in
Fig. 7 is a p
  • in
t
  • f
decision where eectiv e duration
  • f
x can b e determined b eing the dierence b et w een the D A TE elds at this no de and that
  • f
the starting no de
  • f
x. A t time instan t
  • ,
a table
  • f
c hain information is handled where w e ha v e for eac h concur- ren t c hain t w
  • en
tries: RR n2983
slide-19
SLIDE 19 16

IF δ > f1

Configuration of concurrent chains at logical time τ

Chain C1 C2 C3 C4 f2 f3 f4 δ δ δ CInd CFlx δ f3 f1 f4 C1 C2 C3 C4 δ δ δ

τ

f2

x y z w

S E

Figure 7: Dynamic formatting 1. the v alue
  • f
uncomp ensated indeterminism CInd resulting from x. 2. the a v ailable exibilit y CFlx at the nearest future no de
  • n
the c hain. Initially , the CInd eld
  • f
eac h c hain has a zero v alue and CFlx eld is initialized with the amoun t
  • f
a v ailable exibilit y
  • n
eac h c hain un til the next sync hronisation p
  • in
t E. This information is dened also statically b efore starting the presen tation, but it is up dated dynamically during the presen tation when used to comp ensate for indeterminism. It is essen tial to mark the amoun t
  • f
exibilit y used for comp ensation as to b e reserv ed in
  • rder
not to b e used again b y another detected indeterminism in the future. In theFig. 7 an amoun t
  • f
indeterminism
  • has
b een detected along the c hain C1. Since there is not enough exiblit y f1 along this c hain ( >f1), the
  • ther
c hains are ask ed via the table
  • f
concurren t c hains to reco v er from this dela y . The c hain C2 notices this request at the end
  • f
the
  • b
ject y where its end is dela y ed (static
  • b
ject) while the
  • b
jects z and w are stretc hed b y the amoun t
  • pro
vided b y their exibilt y f3 and f4 resp ectiv ely . W e can conclude that the MADEUS's extended temp
  • ral
constrain t net w
  • rk
pro vides a self adaptativ e mecanism handling unpredictable temp
  • ral
b eha viour. As w e ha v e seen in the example, this mecanism is based mainly
  • n
the
  • rganization
  • f
the temp
  • ral
structure as a graph whic h in turn is
  • rganized
as c hains. 3.2.4 T A C managemen t By T A C (Time Access Con trol) managem en t w e mean all actions that ma y b e requested b y the user and lead to a c hange
  • f
state
  • f
a curren tly presen ted m ultim edia do cumen t. Examples
  • f
these actions are start, restart, pause, resume, kill, fast forw ard, and fast rewind
  • f
a m ultim edia do cumen t's presen tation. T
  • handle
all these actions ecien tly , the sc he- duler handles presen tation con texts holding information ab
  • ut
curren tly activ e m ultim edia
  • b
jects and their asso ciated system activities (thread, pro cess, timer, etc.). INRIA
slide-20
SLIDE 20 17

A Activity B Activity C Activity Pres 1 Pres 2 Resume of Presentation 1 (3) Activation of link to Presentation 2 (1) End of Presentation 2 (2) Stop_Activities ( Pres 1 ) ( Push Stack ) Start_Activities ( Pres 2 ) ( Pop Stack ) Resume_Activities ( Pres 1 ) A B C Presentation Context Presentation 1

Figure 8: Managemen t
  • f
presen tation con text F
  • r
example, when the user requests a pause
  • f
the presen tation, the system susp ends all
  • f
the concurren tly running activities. It stores the v alue
  • f
their in ternal clo c k represen ted b y the curren t presen tation step and pushes the curren t presen tation con text in the presen- tation stac k. When the user requests a resume
  • f
the presen tation, the presen tation con text is p
  • pp
ed
  • ut
from the stac k and all susp ended activities b elonging to this con text are to b e resumed
  • nce
again. Also, the
  • p
erations
  • f
fast forw ard and fast rewind are ac hiev ed similarly b y mo difying the sp eed and direction
  • f
the activities in the curren t con text. Ob- jects with hidden in ternal clo c ks mo delled b y uncon trollable in terv als are simply ignored in the last t w
  • p
erations. 3.2.5 T emp
  • ral
h yp erlinks T emp
  • ral
h yp erlinks means linking b et w een dieren t presen tations in a con text sensitiv e w a y dep ending
  • n
the seman tics asso ciated with this link. The link ed presen tations ma y b e in the same m ultim edia do cumen t (in ternal references)
  • r
in dieren t m ultim edia do cumen ts (external references). The sc heduler handles this kind
  • f
referencing b y managing the stac k
  • f
presen tation in a v ery simple w a y . Once a link is activ ated, the curren t presen tation con text (i.e. the presen tation con text
  • f
the source anc hor) is pushed in the stac k
  • f
presen tation, and the presen tation con text
  • f
the destination b ecomes the curren t
  • ne.
Once the presen tation
  • f
the destination nishes, then the presen tation con text
  • f
the source is p
  • pp
ed
  • ut
from the stac k
  • f
presen tation in
  • rder
to resume starting from the time p
  • in
t
  • f
link activ ation. RR n2983
slide-21
SLIDE 21 18

Mapped Unmapped Terminated Active Suspended Stored Initially Map Unmap End/Kill Restart Pause Resume Restore Store Start Play_one_step

Figure 9: State transition diagram
  • f
m ultimedia
  • b
jects 3.3 Execution mediator The execution mediator main tains the states
  • f
the
  • b
jects and handles their transitions b et w een these states as sho wn in Fig. 9. The mediator treats all kinds
  • f
m ultim edia
  • b
jects in the same w a y . The main states
  • f
the
  • b
jects are: unmapp ed, mapp ed, activ e, terminated, susp ended, and stored. The unmapp ed state is the initial state
  • f
all
  • b
jects, i.e. the
  • b
ject has not y et allo cated its required resources. In the mapp ed state, the
  • b
ject has allo cated its required resources but its presen tation has not b een y et started. In the activ e state the system activities necessary for the
  • b
ject's presen tation, are started. In the terminated state, the
  • b
ject's presen tation has come to an end and it is ready to free its allo cated resources. The susp ended state means that the system activities asso ciated to the
  • b
ject are susp ended, i.e. its presen tation is susp ended but not y et ended. Actually , w e ha v e dieren t kinds
  • f
resources that can b e classied as follo ws:
  • spatial
resources suc h as windo ws, pixmaps whic h are giv en unique iden tiers b y the windo w manager and the color resource.
  • hardw
are resources suc h as video decompression cards and audio devices.
  • system
resources suc h as pro cesses, threads and timers. Once a h yp erlink is activ ated, the curren tly running
  • b
jects
  • f
the anc hor presen tation en ters in the stored state where their curren t status information and clo c k are stored, for example the mapp ed state
  • f
an
  • b
ject
  • r
the curren t frame n um b er
  • f
a video. As a result
  • f
storing this information, the system can free the resources allo cated b y these
  • b
jects, INRIA
slide-22
SLIDE 22 19 so that when a return from the link tak es place, the system can reallo cate the necessary resources in
  • rder
to pla y the
  • b
jects starting from their status and in ternal clo c k at the time
  • f
link activ ation. The pla y
  • ne
step transition sho wn in Fig. 9, represen ts the case
  • f
taking in to accoun t the gran ularit y
  • f
a m ultim edia
  • b
ject (for example frames
  • f
a video
  • b
ject) where with eac h unit
  • f
gran ularit y
  • f
the
  • b
ject the system can mak e sev eral actions, as for example mo ving the enclosing b
  • x
  • f
a video
  • n
the screen, c hanging the degree
  • f
brigh tness
  • f
the displa y ed video, displa ying a frame
  • f
this video, and visual eects lik e stippling and bluring. This state transition can b e also used to exp
  • rt
in ternal ev en ts
  • f
an activ e
  • b
ject to the sc heduler. These ev en ts can b e sync hronized with
  • ther
media
  • b
jects
  • f
the do cumen t (temp
  • ral
pro jections [8 ]) if their description is pro vided b y indexation to
  • ls.
In this section, w e presen ted the dieren t la y ers handling a m ultimedia presen tation. The MADEUS protot yp e has b een implem en ted
  • n
Sun w
  • rkstations
with C language. It handles media streams
  • f
v arious format lik e Mp eg video, Mp eg Audio, still pictures (GIF, JPEG, ..) and formatted text. W e ha v e created and tested a v ariet y
  • f
testing and demonstating do cumen ts and p erformance enhancemen ts are curren tly undetak en. 4 Conclusion W e ha v e presen ted in this pap er a framew
  • rk
for temp
  • ral
represen tation and sync hroniza- tion
  • f
m ultim edia do cumen ts. W e ha v e
  • utlined
some criteria to compare
  • ther
prop
  • sed
time represen tations and their limitations. W e ha v e also sho wn that the extended temp
  • ral
constrain t net w
  • rks
implem en ted in MADEUS are p
  • w
erful and suitable for b
  • th
m ultim edia authoring and presen tation. Among the c haracteristics
  • f
the MADEUS protot yp e, w e ha v e addressed the follo wing asp ects:
  • the
expressiv eness
  • f
the temp
  • ral
represen tation whic h captures the qualitativ e, quan- titativ e, causal and indeterministic asp ects
  • f
m ultimedia
  • b
jects and presen tations.
  • a
la y ered application arc hitecture whic h allo ws the pro duction
  • f
temp
  • ral
la y
  • ut,
temp
  • ral
na vigation through h yp erlinks and run-time monitoring
  • f
the presen tation b y a dynamic formatter. F rom the implemen tation exp erience
  • f
MADEUS, w e came to the conclusion that the b
  • ttlenec
k
  • f
p erformance is due to presen tation la y
  • ut.
Op erations lik e video decompres- sion, graphics rendering and color quan tization are the main source
  • f
temp
  • ral
deviation from a planned scenario. This fact sho ws that treating temp
  • ral
uncertain t y b y a self- adaptativ e managemen t
  • f
temp
  • ral
sync hronization is a cen tral issue. This w
  • rk
is curren tly exp erimen ted in the Op era pro ject. The goal
  • f
this pro ject is to dev elop an editorial en vironmen t for the construction, manipulatio n and storage
  • f
complex m ultimedia do cumen ts. RR n2983
slide-23
SLIDE 23 20 References [1] ALLEN J.F., \Main taining Kno wledge ab
  • ut
T emp
  • ral
In terv als", Communic ations
  • f
the A CM, v
  • l.
V
  • l.
26, n um. No. 11, pp. 832-843, no v em b er 1983. [2] BUCHANAN M.C., ZELL WEGER P .T., \Automatic T emp
  • ral
La y
  • ut
Mec hanisms", Pr
  • c
e e dings
  • f
the First A CM International Confer enc e
  • n
Multime dia, pp. 341-350, Anaheim, California, august 1993. [3] BUL TERMAN D.C.A., V AN R OSSUM G., V AN LIERE R., \A Structure for T ransp
  • r-
table, Dynamic Multimedia Do cumen ts", USENIX Confer enc e, pp. 137-1559, Nash ville, TN, june 1991. [4] DECHTER R., MEIRI I., PEARL J., \T emp
  • ral
Constrain t net w
  • rks",
A rticial In- tel ligenc e, v
  • l.
49, , pp. 61-95, 1991. [5] DIRECTOR 4.0, \User's Guide", Macromedia. [6] GOLDF ARB C. F., The SGML Handb
  • k,
Oxford Univ ersit y Press, Oxford, 1990. [7] ISO/IEC 10743, HyTime Information T e chnolo gy Hyp erme dia/Time-b ase d Structuring L anguage (HyTime), no v em b er 1992. [8] ISO/IEC 14478-1, Information Pr
  • c
essing Systems { Computer Gr aphics and Image Pr
  • c
essing { Pr esentation Envir
  • nments
for multime dia Obje cts (PREMO), n um. 1, ma y 1996. [9] ISO/IEC 13552-1, Information T ehnolo gy Co de d R epr esentation
  • f
Multime dia and Hy- p erme dia Information Obje ct (MHEG) , n um. 1,
  • ctob
er 1994. [10] KHALF ALLAH H., KARMOUCH A., \An arc hitecture and a data mo del for in tegrated m ultim edia do cumen ts and presen tational applications", Multime dia Systems, pp. 238- 250, V
  • l.
3, 1995. [11] LA Y A
  • ID
A N., SABR Y-ISMAIL L., \Main taining T emp
  • ral
Consistency
  • f
Multimedia Do cumen ts Using Constrain t Net w
  • rks",
Multime dia Computing and Networking 1996, M. F r e eman, P. Jar detzky, H. M. Vin, e d., pp. 124-135, SPIE 2667, San-Jos
  • e,
USA, february 1996. [12] LITTLE T.D.C., GHAF OOR A, \In terv al-Based Conceptual Mo dels for Time- Dep enden t Multimedia Data", IEEE T r ansactions
  • n
Know le dge and Data Engine e- ring (Sp e cial Issue: Multime dia Information Systems), v
  • l.
V
  • l.
5, n um. 4, pp. 551-563, august 1993. [13] PEREZ-LUQUE , LITTLE T.D.C., \A T emp
  • ral
Reference F ramew
  • rk
for Multimedia Sync hronization", Pr
  • c
e e dings
  • f
Sync'95, ma y 19, Virginia, 1995. INRIA
slide-24
SLIDE 24 21 [14] QUINT V., V A TTON I., \Grif: An In teractiv e System for Structured Do cumen t Ma- nipulation", T ext Pr
  • c
essing and Do cument Manipulation, Pr
  • c
e e dings
  • f
the Interna- tional Confer enc e, pp. 200-213, Cam bridge, 1986. [15] VILAIN M., KA UTZ H. A., \Constrain t Propagation Algorithms for T emp
  • ral
Reaso- ning", Pr
  • c
e e dings
  • f
AAAI, pp. 377-382, Philadelphia, august 1986. [16] W AHL T., R OTHERMEL K., \Represen ting Time in m ultim edia", Pr
  • c
e e dings
  • f
the International Confer enc e
  • n
Multime dia Computing and Systems, pp. 538-543, Boston, 1994. [17] WEITZMAN L., WITTENBUR G K., \Automatic Presen tation
  • f
Multimedia Do- cumen ts Using Relational Gramm ars", Pr
  • c
e e dings
  • f
the Se c
  • nd
A CM International Confer enc e
  • n
Multime dia, pp. 443-451, San F rancisco, California,
  • ctob
er 1994. RR n2983
slide-25
SLIDE 25 22 Con ten ts 1 In tro ducti
  • n
3 2 Represen tati
  • n
  • f
time in m ultim edi a do cumen ts 3 2.1 Time represen tation requiremen ts . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Time represen tation in MADEUS: an extension
  • f
temp
  • ral
constrain t net w
  • rks
6 2.2.1 T emp
  • ral
Constrain t net w
  • rks
. . . . . . . . . . . . . . . . . . . . . . 6 2.2.2 Extended temp
  • ral
constrain t net w
  • rks
. . . . . . . . . . . . . . . . . 7 2.2.3 Ev aluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Analysis
  • f
  • ther
time represen tations . . . . . . . . . . . . . . . . . . . . . . 10 3 Presen tati
  • n
la y er in MADEUS 11 3.1 Arc hitecture
  • v
erview
  • f
MADEUS . . . . . . . . . . . . . . . . . . . . . . . . 12 3.2 Sc heduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2.1 Presen tation con text and presen tation stac k . . . . . . . . . . . . . . . 13 3.2.2 Sc heduling principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.2.3 Dynamic formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.4 T A C managemen t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.5 T emp
  • ral
h yp erlinks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.3 Execution mediator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4 Conclusion 19 INRIA
slide-26
SLIDE 26

Unit´ e de recherche INRIA Lorraine, Technopˆ

  • le de Nancy-Brabois, Campus scientifique,

615 rue du Jardin Botanique, BP 101, 54600 VILLERS L` ES NANCY Unit´ e de recherche INRIA Rennes, Irisa, Campus universitaire de Beaulieu, 35042 RENNES Cedex Unit´ e de recherche INRIA Rhˆ

  • ne-Alpes, 655, avenue de l’Europe, 38330 MONTBONNOT ST MARTIN

Unit´ e de recherche INRIA Rocquencourt, Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY Cedex Unit´ e de recherche INRIA Sophia-Antipolis, 2004 route des Lucioles, BP 93, 06902 SOPHIA-ANTIPOLIS Cedex

´ Editeur INRIA, Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY Cedex (France) ISSN 0249-6399

View publication stats View publication stats