An Integrated Authoring And Presentation Environment For Interactive - - PDF document

an integrated authoring and presentation environment for
SMART_READER_LITE
LIVE PREVIEW

An Integrated Authoring And Presentation Environment For Interactive - - PDF document

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/2362328 An Integrated Authoring And Presentation Environment For Interactive Multimedia Documents Article June 2001 Source: CiteSeer


slide-1
SLIDE 1

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/2362328

An Integrated Authoring And Presentation Environment For Interactive Multimedia Documents

Article · June 2001

Source: CiteSeer

CITATIONS

11

READS

28

4 authors, including: Some of the authors of this publication are also working on these related projects: Structural Health monitoring using Wireless Sensor Networks View project IoT-Based Fire Alarm System View project Cécile Roisin University of Grenoble

78 PUBLICATIONS 865 CITATIONS

SEE PROFILE

Loay Sabry Ismail Qatar University

31 PUBLICATIONS 344 CITATIONS

SEE PROFILE

All content following this page was uploaded by Cécile Roisin on 06 February 2013.

The user has requested enhancement of the downloaded file.

slide-2
SLIDE 2 AN INTEGRA TED A UTHORING AND PRESENT A TION ENVIR ONMENT F OR INTERA CTIVE MUL TIMEDIA DOCUMENTS M. JOURD AN, N. LA Y AID A, C. R OISIN, L. SABR Y-ISMAIL OPERA pr
  • je
ct, INRIA R h^
  • ne-A
lp e s 655 A venue de l'Eur
  • p
e, 38330 Montb
  • nnot,
F r anc e email: fMuriel.Jour dan, Nabil.L ayai da, Ce cile.R
  • isin,
L
  • ay.Sabryg@inrial
p es .fr Abstract In this pap er, w e presen t
  • ur
con tribution to meet the main require- men ts for pro viding an authoring en vironmen t for in teractiv e m ultimedia do cumen ts. The particularit y
  • f
  • ur
approac h is to consider m ultimedia authoring as a whole, from sp ecication issues to execution requiremen ts. The results
  • f
these theoretical w
  • rks
are v alidated in to a real running application called Madeus. 1 In tro ducti
  • n
The recen t adv ances in m ultim edia systems, together with the adv en t
  • f
high sp eed net w
  • rks,
ha v e lead to the emergence
  • f
a new generation
  • f
applica- tions and will considerably aect the existing
  • nes.
In particular, h yp ermedia applications lik e the WWW
  • riginally
designed for text and simple graphics are mo ving to w ards a seamless in tegration
  • f
data lik e audio, video, plugins and legacy programs (applets). In con trast to the static nature
  • f
text-orien ted do cumen ts, the addition
  • f
time-based media
  • b
jects to do cumen ts requires the consideration
  • f
another dimension
  • f
their underlying arc hitecture: the do cumen t dynamics
  • r
tem- p
  • ral
dimension. Basic media
  • b
jects, lik e video, ha v e in trinsic durations and they can b e temp
  • rally
  • rganized
b y the author. This temp
  • ral
  • rganization,
so-called the temp
  • r
al sc enario
  • f
the do cumen t, raises a n um b er
  • f
problems and in spite
  • f
the big amoun t
  • f
researc h w
  • rk
ac hiev ed in this area, these is- sues are still far from b eing mastered. W e ha v e iden tied four main asp ects to b e considered when designing a m ultim edia do cumen t authoring en vironmen t:
  • Expressiv
eness: It measures the abilit y
  • f
the temp
  • ral
language to co v er all the p
  • ssible
scenarios.
  • Consistency
c hec king: It ensures that the scenario sp ecied b y the author is consisten t during the edition phase.
slide-3
SLIDE 3 2
  • Simplicit
y: It measures the abilit y
  • f
naiv e users in using the authoring to
  • l.
  • Run-time
monitoring
  • f
the presen tation: This requiremen t is related to the presen tation la y er and its abilit y to la y
  • ut
the do cumen t in compli- ance with the author's sp ecication. This pap er addresses these dieren t issues, from the do cumen t sp ecica- tion, construction and manipulation to the application arc hitecture and
  • rga-
nization. The w
  • rk
ac hiev ed is presen ted through Madeus: an exp erimen tal authoring and presen tation to
  • l
dev elop ed in
  • ur
pro ject. The pap er is
  • rganized
in t w
  • parts.
In the rst
  • ne,
w e explain the ab
  • v
e-men tioned issues,
  • utlining
the problems raised b y eac h
  • f
them. In the second part
  • f
the pap er, w e presen t
  • ur
approac h and the design c hoices that ha v e b een made: rstly , pro viding the author with a declarativ e sp ecication language
  • f
m ultim edia do cumen ts; secondly , using an extension
  • f
temp
  • ral
constrain t net w
  • rks
to manage the temp
  • ral
scenario
  • f
a do cumen t. Moreo v er, w e dene a general arc hitecture
  • f
the Madeus application to demonstrate the v alidit y
  • f
the t w
  • ab
  • v
e-men tioned c hoices. 2 Multimedi a authoring issues 2.1 Example
  • f
a F amily Pr esentation Let us consider the example
  • f
a m ultim edia do cumen t presen ting a family
  • f
four mem b ers. Eac h p erson can b e presen ted b y a sequence comp
  • sed
  • f
a still picture
  • f
him when he w as a bab y , a short video ab
  • ut
his life (its duration is ab
  • ut
  • ne
min ute) and a recen t photo. Then eac h
  • f
these three elemen ts is asso ciated with textual elemen ts whic h giv es some commen ts to the pictures. A t an y time, the reader
  • f
the do cumen t can in terrupt the video in
  • rder
to get in to the presen tation
  • f
the last still picture. The sequence (named S in the rest
  • f
the pap er) is comp
  • sed
  • f
the three elemen ts previously men tioned, the three asso ciated textual parts and the button
  • f
in teraction. In addition, eac h family mem b er can add a p ersonal audio message (its duration is ab
  • ut
1 min ute and 30 sec.). This audio message m ust start with the sequence S and m ust end b efore it. Fig. 1 presen ts a p
  • ssible
execution
  • f
the scenario for
  • ne
mem b er
  • f
the family . In this execution, the reader (b y clic king
  • n
the but ton) in terrupts the video 40 seconds after its b eginning. The rst screen
  • f
the do cumen t is comp
  • sed
  • f
a still picture
  • f
all the mem b ers
  • f
the family . The reader can c ho
  • se
to pla y the presen tation
  • f
  • ne
p erson. Once this presen tation ends, the rst screen sho ws up again.
slide-4
SLIDE 4 3

95

Time(sec)

40 80

Button SP1 Video SP2 T3 T2 T1 Audio

Figure 1: P
  • ssible
execution for the presen tation
  • f
  • ne
p erson 2.2 Expr essiveness The rst requiremen t is related to the kind
  • f
temp
  • ral
scenario that an au- thoring system allo ws an author to express. W e presen t in the follo wing the four basic features that w e consider necessary to build realistic scenarios.
  • A
wide v ariet y
  • f
basic
  • b
jects: basic elemen ts lik e text, video, audio, still pictures, virtual animation, JA V A applets should b e supp
  • rted.
It is necessary to distinguish b et w een t w
  • kinds
  • f
  • b
jects: con trollable
  • b
jects whose duration v alues can b e c hosen b y the system from a range
  • f
p
  • ssible
v alues, and uncon trollable
  • b
jects whic h tak e a duration v alue at the presen tation time. F
  • r
instance, a video duration can ha v e a range
  • f
p
  • ssible
v alues dep ending
  • n
its acceptable pla ying rates. W e kno w the lo w er and upp er b
  • und
v alues, but w e cannot predict its eectiv e v alue during the presen tation.
  • Global
and lo cal in teractions: The idea is to pro vide the reader with t w
  • kinds
  • f
in teractions: global
  • nes
whose scop e is to in terrupt all the curren t activities in the do cumen t. In
  • ur
example, the rst screen is comp
  • sed
  • f
a set
  • f
global in teractions. A t the
  • pp
  • site,
the button whic h in terrupts the video in the sequence is a lo cal in teraction since it do es not in terrupt the audio message whic h is activ e at the same time
  • f
the video. A user-in teraction is a t ypical uncon trollable
  • b
ject whith no upp er b
  • und.
  • Causal
relations b et w een ev en ts: Some b eha viors
  • f
a scenario can b e expressed as causal relationships b et w een temp
  • ral
ev en ts (start
  • r
end
  • f
  • b
jects). In
  • ur
example, this is the case with the button asso ciated to the video: if the reader clic ks
  • n
it, the video is in terrupted.
slide-5
SLIDE 5 4
  • Relativ
e p
  • sitioning
  • f
ev en ts (sync hronization): They are used to ex- press the equalit y
  • r
the precedence
  • f
temp
  • ral
ev en ts. In
  • ur
example, suc h sync hronization is frequen tly used: textual commen tary related to comp
  • nen
ts
  • f
the sequence m ust start (resp. end) when the comp
  • nen
t related to it starts (resp. ends). This is also the case with the audio message whic h m ust start at the same time as the sequence S and nish b efore it. Some authoring systems lik e CMIF ed 14 , are restricted to causal relations. Because relativ e p
  • sitioning
  • f
ev en ts ma y lead to inconsistencies, it a v
  • ids
the dicult task to v alidate the scenario. On the
  • ther
hand,
  • ther
systems lik e ISIS 13 ,
  • nly
consider relativ e p
  • sitioning.
In this approac h the scenario sp ecication is mo delled b y a set
  • f
linear inequalities to b e solv ed. Duration v alues
  • f
  • b
jects are c hosen b y the system from an in terv al
  • f
lo w er and upp er b
  • und
v alues. The system, with the help
  • f
a temp
  • ral
formatter, statically computes a time-map whic h satises all the constrain ts. The t w
  • ma
jor dra wbac ks
  • f
this approac h are:
  • Lo
cal user in teractions cannot b e mo delled.
  • Ob
jects lik e video, audio, JA V A applets are mo delled b y static durations and therefore ma y b e in terrupted (resp. dela y ed) if to
  • long
(resp. to short) at presen tation time. In fact, merging the t w
  • t
yp es
  • f
sp ecication: causal and relativ e p
  • si-
tioning is a dicult task. As an example, the sequence S
  • f
Figure 1 con tains a causal relation, since a user-in teraction can in terrupt the video, together with a relativ e
  • ne:
S m ust start at the same time as an audio message and m ust end after it. Causal relations in tro duce the distinction
  • f
t w
  • kinds
  • f
ending ev en ts for eac h
  • b
ject: the normal end and the in terrupted
  • ne.The
dicult y is to dene clearly whic h end is in v
  • lv
ed when using relativ e expressions. 2.3 Static A nalysis
  • f
Do cument Sp e cic ation The temp
  • ral
scenario
  • f
m ultim edia do cumen t m ust b e v alidated during the authoring phase and therefore b efore the presen tation phase. It is
  • b
vious that the presence and the nature
  • f
temp
  • ral
errors are strongly related to (1) the w a y in whic h the author sp ecies the do cumen t and (2) the expressiv eness p
  • w
er
  • f
the sp ecication language itself. Approac hes limited to causal
  • p
erators (lik e CMIF ed 3 ) ha v e no problems
  • f
temp
  • ral
consistency . A t the
  • pp
  • site,
relativ e p
  • sitioning
  • f
ev en ts can lead to sp ecify erroneous scenarios. The trivial case
  • f
inconsistency consists
slide-6
SLIDE 6 5 in sp ecifying that t w
  • b
jects ha v e the same duration, although their in terv al
  • f
p
  • ssible
duration ha v e an empt y in tersection. Another example is to sp ecify that the end
  • f
a video is sync hronized with the end
  • f
an audio. If the video and the audio are uncon trollable
  • b
jects, their duration can
  • nly
b e kno wn when they end, it is therefore not p
  • ssible
to guaran tee that the sp ecication is v alid. In all these cases the system ha v e to w arn the author
  • f
the existence
  • f
inconsistencies and to help him in prev en ting them. Inconsistencies in the case
  • f
relativ e p
  • sitioning
  • f
temp
  • ral
ev en ts and con trollable
  • b
jects can b e easily iden tied. In this case, it is a classical prob- lem
  • f
constrain t satisfaction: a sp ecication is consisten t if and
  • nly
if there exists a v alue for eac h starting and ending p
  • in
t whic h satises all the con- strain ts. Uncon trollable
  • b
jects are far more complex to handle. The notion
  • f
consistency b ecomes more complex 10 since all the p
  • ssible
durations m ust b e tak en in to accoun t for the uncon trollable
  • b
jects together with the
  • rdering
b et w een ev en ts. This situation arises in
  • ur
w
  • rking
example: in
  • rder
to b e sure that the sequence S will nish after the audio message, the duration
  • f
the second still picture
  • f
the sequence dep ends
  • n
the real duration
  • f
the video, whic h can b e in terrupted b y a user-in teraction. The constrain t satisfaction problem with uncon trollable v ariables in a con- strain t satisfaction problem is still an
  • p
en issue and is curren tly studied in another area: planning in rob
  • tics
15 . 2.4 User-F riend ly Interfac e T
  • da
y , most p
  • pular
commercial authoring to
  • ls
lik e Director 5 use a propri- etary script language (lik e Lingo) and, as so
  • n
as
  • ne
needs adv anced features, programming is still the
  • nly
solution for sp ecifying scenarios. Ho w ev er, build- ing an in teractiv e m ultim edia do cumen t should b e p erformed b y non-sp ecialists with higher lev el user in terface facilities 7 . W e claim that the authoring ac- tivit y should b e a cyclic "sp ecify , test and mo dify" pro cess with the follo wing features:
  • Spatial
and temp
  • ral
p
  • sitioning
  • f
do cumen t comp
  • nen
ts m ust b e ex- pressed in a relativ e w a y for man y reasons : rst it is easier to use a declarativ e st yle
  • f
sp ecication than to learn a new programmi ng lan- guage; second a spatial
  • r
temp
  • ral
  • rganization
describ ed with relativ e p
  • sitionning
  • p
erators is easier to mo dify than absolute p
  • sitionning
as in timeline approac hes; third this is the system (the formatter part) and not the author whic h computes the v alues asso ciated with eac h
  • b
ject (for instance in the time dimension: its starting p
  • in
t and its duration). Ho w ev er, suc h relativ e approac hes m ust b e asso ciated with p
  • w
erful vi- sual in terface in
  • rder
to mak e the author's task easier. Esp ecially in
slide-7
SLIDE 7 6 the time dimension, it is necessary to help the author to understand the global b eha vior implied b y the set
  • f
lo cal constrain ts. Let us not conclude that temp
  • ral
formatting is confron ted with the uncertain t y
  • f
the duration
  • f
uncon trollable
  • b
jects, lo cal user-in teractions and causal
  • p
erators.
  • Edition
and presen tation phases m ust b e in tegrated. This allo ws fast toggling b et w een the t w
  • phases,
so that the author can in teract dur- ing the presen tation phase to mo dify the do cumen t. This is a w a y to approac h the Wysiwyg paradigm successfully used for static do cumen ts but that cannot b e extended to m ultimedia do cumen ts due to their tem- p
  • ral
dimension.
  • Hierarc
hic decomp
  • sition
  • f
the do cumen t m ust b e supp
  • rted.
Ho w ev er, this hierarc h y m ust not b ecome to
  • constraining
for the author. Some approac hes (SR T, CMIF ed) mo del the do cumen t b y a tree structure with a temp
  • ral
  • p
erator asso ciated to eac h no de. This solution has t w
  • ma
jor dra wbac ks: some temp
  • ral
comp
  • sitions
can not b e sp ecied and the mo dication
  • f
an existing scenario b ecomes dicult. 2.5 The A r chite ctur e
  • f
the Pr esentation L ayer Sev eral functionalities for the presen tation la y er ha v e b een iden tied in 11 : (1) pro viding the reader with an accurate rendition
  • f
what the author has sp ec- ied, (2) supp
  • rting
T emp
  • ral
Access Con trol
  • p
erations lik e: stop, resume, fast-forw ard, fast-rewind, as w ell as h yp erlinks to allo w the user to manipulate the m ultim edia do cumen t presen tation and (3) supp
  • rting
the distribution
  • f
media
  • b
jects across the net w
  • rk.
Multimedia applications are real time systems and CPU time consuming, so the applications ha v e to gh t against limitati
  • ns
  • f
resources (suc h as CPU a v ailabili t y , disks bandwidth, net w
  • rks
bandwidth, net w
  • rk
accessing dela ys, a v ailabili t y
  • f
media c hannels, data buers sizes, etc.). F
  • r
example, media
  • b
jects lik e video and still pictures need a considerable CPU pro cessing time for image decompression, while media
  • b
jects lik e audio and video, are really sensitiv e to the system load as they ha v e deadline times that m ust b e resp ected. So, Qualit y
  • f
Service (QoS) issues m ust b e tak en in to consideration to guaran- tee minim um requiremen ts
  • f
p erformance. Examples
  • f
situations that m ust b e a v
  • ided
are:
  • The
author sp ecies that t w
  • videos
pla y in sequence. During the presen- tation, the rst nishes but a considerable dela y b et w een the t w
  • videos
is encoun tered (due to the time needed b y decompression
  • r
the net w
  • rk
accessing dela y), in a w a y that it aects the qualit y
  • f
the presen tation.
slide-8
SLIDE 8 7
  • The
author sp ecies that t w
  • videos
pla y in parallel together with an audio and the CPU is
  • v
erloaded b y the decompression
  • f
the videos at the cost
  • f
the qualit y
  • f
the audio. Resource limitati
  • ns
are unpredictable in nature. So, the task
  • f
the pre- sen tation system can b e simplied if dela y v alues (of net w
  • rk
and disk access as w ell as buer's queue w aiting time) and CPU a v ailabil it y can b e appro ximately predicted. As a result, it is imp
  • rtan
t to ha v e an ecien t presen tation la y er that can mak e a trade-o b et w een the qualit y
  • f
presen tation and the a v ailabilit y
  • f
resources, in suc h a w a y that it giv es the reader with an acceptable rendering
  • f
the m ultim edia do cumen t. 2.6 Synthesis In
  • rder
to summarize these requiremen ts, w e can
  • utline
the v e follo wing p
  • in
ts whic h are considered as cen tral issues in m ultim edia authoring:
  • Relativ
e p
  • sitioning
  • f
  • b
jects are necessary due to the user-in terface requiremen ts but it implies b
  • th
the presence
  • f
a temp
  • ral
formatter nd a static c hec k er allo wing the detection
  • f
temp
  • ral
inconsistencies.
  • Causal
  • p
erators are necessary to pro vide the reader with lo cal in terac- tions. They in tro duce for eac h
  • b
ject, the distinction b et w een t w
  • kinds
  • f
ending: normal and in terrupted.
  • Some
  • b
jects ha v e no static con trollable duration and their presen tation should not b e in terrupted. This increases the complexit y
  • f
the notion
  • f
temp
  • ral
inconsistencies and temp
  • ral
formatting.
  • Hierarc
hic decomp
  • sition
is necessary to represen t the logical structure whic h do es not necessarily corresp
  • nd
to the temp
  • ral
  • rganization
  • f
the do cumen t.
  • The
presen tation phase m ust b e in tegrated with the editing phase in
  • rder
to reduce the editing cycle. 3 Madeus Approac h Madeus has prop
  • sed
t w
  • imp
  • rtan
t c hoices: rstly , pro viding the author with a declarativ e and hierarc hic sp ecication language for m ultim edia do cumen ts based
  • n
an extension
  • f
Allen's in terv al
  • p
erators; secondly , using an ex- tension
  • f
temp
  • ral
constrain t net w
  • rks
to manage the temp
  • ral
scenario
  • f
a do cumen t during its static analysis, and also to supp
  • rt
the presen tation
slide-9
SLIDE 9 8 phase. W e start b y presen ting the sp ecication language used, then the in ter- nal structure whic h serv es as the basic data structure to manage b
  • th
edition and presen tation phases. 3.1 A Mark-up F
  • rmat
for Multime dia Do cuments Basic Ob jects Madeus basic
  • b
jects are video, audio, still picture, text, temp
  • ral
dela y and button. Buttons are used in the scenario to in tegrate lo cal user-in teractions. All
  • b
jects are abstracted as temp
  • ral
in terv als. The in terv al asso ciated to a button starts at the instan t when the button can b e activ ated and ends when it is really activ ated. In terv als are t yp ed as con trollable (text, still pic ture, temp
  • ral
dela y)
  • r
uncon trollable (video, audio, button). A lo w er b
  • und
and an upp er b
  • und
  • f
duration are asso ciated with eac h
  • b
ject. Upp er b
  • und
duration can b e innite, this is the case
  • f
a button. In case
  • f
a con trollable in terv al, the temp
  • ral
formatter can c ho
  • se
an y v alue within these b
  • unds.
In case
  • f
an uncon trollable in terv al, the temp
  • ral
formatter is informed that the real duration
  • f
the
  • b
ject will b e within these b
  • unds.
Allen's In terv al Algebra and Our Extensions Allen's algebra 1 expresses relativ e p
  • sitioning
  • f
in terv als. W e ha v e extended it with causal
  • p
erators to express causal relationships b et w een ending ev en ts:
  • A
par minB: the t w
  • in
terv als start at the same time and the shorter causes the termination
  • f
the
  • ther
  • ne;
  • A
par master(A)B: the t w
  • in
terv als start at the same time and the end
  • f
A (whic h is the master) causes the termination
  • f
B, if and
  • nly
if B is still pla ying. W e ha v e, also, extended the algebra with the quan tied dela ys b et w een in terv als. Hierarc hic Decomp
  • sition
A do cumen t is comp
  • sed
  • f
a set
  • f
  • b
jects (basic and comp
  • site)
related in time. A hierarc hic decomp
  • sition
giv es the logical structure
  • f
the do cumen t, where the siblings at eac h hierarc hical lev el can b e temp
  • rally
related. Fig. 2 illustrates the hierarc hic decomp
  • sition
and the sym b
  • lic
represen tation
  • f
the w
  • rking
example. The principal do cumen t has a ro
  • t
  • b
ject called Men u whic h has four c hildren (Option1,...., Option4)
  • f
t yp e button. These c hildren are temp
  • rally
related b y the par min
  • p
erator, and eac h
  • ne
  • f
them is asso ciated
slide-10
SLIDE 10 9

Menu Document Symbolic Representation Logical Structure Hyperlinks

. . . . . . . . . . . . . . .

Biography Documents Menu Option1 Optioni Option4 Biographyi

S Audio SP1 T1 Video T2 Button SP2 T3

Menu { LIST-PARMIN (Option1 , ..., Option i, ..., Option 4 ) . . . Optioni { . . . . . . <href> Biography i } } Biography i Audio STARTS { S S { SP1 EQUALS T1 SP1 Video Button Video SP2 MEETS Video EQUALS T2 PARMIN Video MEETS SP2 EQUALS T3 } } Biography Biography4 1

Figure 2: Sym b
  • lic
and hierarc hic sp ecication
  • f
the w
  • rking
example with a h yp erlink to another corresp
  • nding
do cumen t. It is imp
  • rtan
t to notice that the hierarc hic decomp
  • sition
denes b
  • undaries
for the scop e
  • f
temp
  • ral
relations. Hyp erlinks are managed lik e in HTML b y asso ciating sp ecic attributes to
  • b
jects. This attribute denes a comp
  • nen
t
  • f
the do cumen t as the target
  • f
the link whic h is accessed when the user clic ks
  • n
the
  • b
ject. 3.2 Extende d T emp
  • r
al Constr aint Networks The temp
  • ral
scenario
  • f
the do cumen t is built incremen tally (temp
  • ral
rela- tions are analyzed
  • ne
b y
  • ne)
in to a hierarc h y
  • f
extended temp
  • ral
constrain t net w
  • rks.
Eac h time a relation is analyzed, the consistency
  • f
the scenario is c hec k ed. W e rst presen t the constrain t temp
  • ral
net w
  • rks
formalism 4 and then presen t the extended form w e prop
  • se.
Moreo v er, w e p
  • in
t
  • ut
in the second part ho w to c hec k the consistency
  • f
a scenario. T emp
  • ral
Constrain t Net w
  • rks
F
  • rmalism
A temp
  • ral
constrain t net w
  • rk
is a Directed Acyclic Graph (D A G) where no des represen t time p
  • in
ts and eac h edge (i, [min,max], j) represen ts a temp
  • ral
slide-11
SLIDE 11 10

Button Video SP2 T3 delay T2 Audio SP1 T1

Figure 3: An example
  • f
Madeus constrain t net w
  • rk
in terv al with a duration range from min to max b et w een the t w
  • time
p
  • in
ts i and j. Dieren t kinds
  • f
algorithms exist to answ er questions lik e: do es a solution exist? Can a particular set
  • f
v alues b e considered as a solution for the problem? In
  • ur
con text, a temp
  • ral
constrain t net w
  • rk
describ es the temp
  • ral
  • rdering
  • f
media
  • b
jects. The range
  • f
p
  • ssible
durations
  • f
a dynamic
  • b
ject is mo delled b y an edge lab elled b y its appropriate in terv al. Before going further, w e in tro duce the follo wing denition: "A temp
  • ral
c hain [i,j,.. .s] is a sequence
  • f
con tiguous edges where eac h edge is
  • nly
related to
  • ne
successor and
  • ne
predecessor". Madeus T emp
  • ral
Constrain t Net w
  • rks
In MADEUS, the previous denition
  • f
a constrain t net w
  • rk
has b een extended to capture t w
  • t
yp es
  • f
information: the con trollabilit y
  • f
the duration and the causalit y relationships. An additional lab elling
  • f
in terv als is added to distinguish whether the temp
  • ral
distance is con trollable
  • r
uncon trollable (resp ectiv ely [min,max]c and [min,max]u). In
  • rder
to express causalit y in temp
  • ral
scenarios, w e add causal rela- tions b et w een temp
  • ral
instan ts, i.e. no des
  • f
the net w
  • rk.
F
  • r
instance, Fig. 3 sho ws the extended net w
  • rks
represen ting the presen tation for
  • ne
p er- son in
  • ur
w
  • rking
example. The causal
  • p
erator par min is represen ted b y a bidirectional arro w. A p
  • sitiv
e dela y is in tro duced since the audio m ust nish b efore the sequence S. Static Analysis Giv en suc h a temp
  • ral
net w
  • rk,
the v alidation phase is equiv alen t to c hec k the existence
  • f
at least
  • ne
solution
  • f
the sp ecication. The three kinds
  • f
inconsistencies that can
  • ccur
in the sp ecication
  • f
a scenario ha v e b een
slide-12
SLIDE 12 11 iden tied previously in 10 : qualitativ e, quan titativ e and indeterministic. These dieren t t yp es
  • f
consistency c hec king can b e p erformed as follo ws:
  • qualitativ
e: a top
  • logical
sort
  • f
the constrain t net w
  • rk
is main tained in
  • rder
to detect the existence
  • f
cycles.
  • quan
titativ e: constrain t propagation tec hniques 16 can b e applied eac h time the net w
  • rk
is mo died. W e c hec k for all parallel c hains if they ha v e non-empt y in tersection
  • f
their allo w able duration ranges. Algorithms used for the constrain t propagation are p
  • lynomial
in time and space and supp
  • rt
incremen tal mo dications
  • indeterministic:
the idea here is to p erform w
  • rst
case analysis to
  • btain
dates
  • f
the form (inferior, sup erior)
  • n
eac h no de
  • f
the net w
  • rk.
The idea here is to see if it is p
  • ssible
to reco v er dynamically , at the pre- sen tation phase, the indeterministic b eha vior
  • f
uncon trollable in terv als b y using the exibilit y
  • f
  • ther
con trollable in terv als to b e pla y ed in the future 10 . 3.3 Pr esentation phase Once the temp
  • ral
consistency
  • f
the scenario is c hec k ed, the dicult y is to compute the solution dynamically at the presen tation stage. It cannot b e done completely statically due to the uncon trollable durations. The role
  • f
the presen tation phase is t w
  • fold
: rst to p erform the sc hedul- ing
  • f
the presen tation
  • p
erations
  • f
ev ery basic
  • b
jects (starting/ nishing, mapping/unm apping, ...); second to handle na vigation inside the doumen t as requested b y the reader (h yp erlink activ ation, jump inside the do cumen t, rewind, ...). Starting from this p
  • in
t, the temp
  • ral
constrain t net w
  • rk
is used b y the presen tation la y er as a task description
  • f
the presen tation: it is a h yp ergraph structure
  • f
nested comp
  • nen
ts where lea v es are basic media
  • b
jects and no des are comp
  • site
  • b
jects (see Fig. 4). This h yp ergraph con tains the follo wing information that ha v e b een statically computed : durations and dates
  • f
basic and comp
  • site
  • b
jects. During this computation, uncon trolable
  • b
jects ha v e b een handle as con trolable
  • nes
so they ha v e also an asso ciated duration. This duration will b e dynamically adjusted during the presen tation phase, as briey explained in the follo wing section. The adv an tage
  • f
sharing the same structure for editing and presen ting a m ultim edia do cumen t is the rapidit y b y whic h w e can toggle b et w een these t w
  • phases.
W e briey describ e ho w temp
  • ral
information is managed in Madeus in
  • rder
to driv e the presen tation and to handle na vigation through the do cumen t.
slide-13
SLIDE 13 12

Biography i Audio

S

delay Button SP1 Video SP2 T1 T2 T3

Figure 4: An example
  • f
the h yp ergraph
  • f
a do cumen t ready to b e presen ted Time managing during presen tation phase T emp
  • ral
informati
  • n
managemen t Eac h
  • b
ject in Madeus is asso ciated with a set
  • f
v ariables and metho ds for its managemen t b y the system. Among them, the v ariables used for time managemen t are the follo wing:
  • State:
suc h as activ e, susp ended and terminated.
  • Nature:
BASIC
  • r
COMPOSITE.
  • StartTime:
instan t at whic h the
  • b
ject starts pla ying.
  • EndTime:
instan t at whic h the
  • b
ject ends pla ying.
  • Exp
ected duration (statically computed). T
  • handle
presen tation
  • f
m ultimedia do cumen ts together with na vigation
  • p
erations, w e need to dene a pr esentation c
  • ntext
that holds the temp
  • ral
information ab
  • ut
a presen tation at a certain time instan t called a temp
  • r
al cut. If w e consider a temp
  • ral
cut at a time instan t t for example, w e are in terested in kno wing whic h
  • b
jects are concurren tly active at that precise instan t
  • f
time and what p
  • in
t in their presen tation lifetimes they ha v e reac hed (an
  • set
v alue is calculated as the dierence b et w een time instan t t and the
  • b
ject's start time). So, the temp
  • ral
cut
  • f
scenario S Ro
  • t
at instan t t for a do cumen t iden- tied b y its Ro
  • t
  • b
ject is dened as the set
  • f
  • rdered
pairs
  • f
all active basic
  • b
jects together with their asso ciated
  • set
v alues. It is dened b y the function k t as follo ws:
slide-14
SLIDE 14 13 k t (Ro
  • t)
= f(Ob ject, Oset (Ob ject)) / (Ob ject 2 S Ro
  • t
) and (Nature (Ob ject) = BASIC) and (StartTime (Ob ject)
  • t
< EndTime (Ob ject)) and (Oset (Ob ject) = t
  • StartTime
(Ob ject)) g Based
  • n
this information, elapsed p erio d
  • f
presen tation
  • f
  • b
jects at an y cut instan t can b e dynamically calculated b y the presen tation manager: the k t function is recursiv ely applied for eac h activ e comp
  • site
  • b
ject un til the lo w est lev els are reac hed (basic
  • b
jects) in
  • rder
to co v er the whole h yp ergraph. Time scaling for m ultimedi a presen tation Another imp
  • rtan
t notion to iden tify for a m ultimedia presen tation system is the time scaling, i.e. what are the relev an t instan ts at whic h the presen tation manager has to p erform some actions. F
  • r
that purp
  • se,
w e ha v e in tro duced t w
  • notions
  • f
presen tation instan ts: the Tics and the T acs. Denitio n
  • f
presen tati
  • n
Tics: A Tic is a time instan t in the lifetime
  • f
the presen tation
  • f
a scenario at whic h a set
  • f
presen tation actions m ust b e carried
  • ut.
So the presen tation
  • f
a scenario can b e represen ted as a sequence
  • f
Tics (as clo c k tics), suc h as the Tic at t= 100 in Fig. 4. Eac h t w
  • consequen
t Tics are separated b y a time step that is the minim al in terv al needed for the presen tation
  • f
the
  • b
jects (for example 1/30 sec
  • r
1/25 sec is required for pla ying video frames). Denitio n
  • f
presen tati
  • n
T acs: T acs are the Tics corresp
  • nding
to the time instan ts at whic h basic (and comp
  • site)
  • b
jects start and nish: these instan ts corresp
  • nd
to the no des
  • f
the h yp ergraph represen ting the do cumen t. Driving the presen tati
  • n
The presen tation progresses as follo ws: at ev ery Tic instan t, the system p erforms the
  • p
erations required as they are dened b y the h yp ergraph structure. Firing a no de consists in starting all the
  • b
jects (text, video, dela y , audio, ...) asso ciated with its
  • utgoing
edges. A no de can b e red if and
  • nly
if all the
  • b
jects asso ciated with its incoming edges ha v e nished. Causal relations
  • n
eac h no de are used to propagate ending information. Duration
  • f
uncon trollable
  • b
jects are
  • bserv
ed. After computing the dierence b et w een the exp ected duration (statically computed) and the
  • bserv
ed
  • ne,
some con trollable durations
  • f
  • b
jects pla y ed in the future are mo died in
  • rder
to resp ect the sync hronization conditions 10 . This in ternal structure whic h is the supp
  • rt
  • f
the presen tation phase is ric h enough to an ticipate the executional state
  • f
the whole do cumen t at dif- feren t future instan ts in the presen tation life time. This abilit y
  • f
an ticipation will allo w us to b etter manage the media
  • b
jects distribution
  • n
the W
  • rld
Wide W eb (pre-fetc hing for example).
slide-15
SLIDE 15 14 Figure 5: In terface
  • f
the Madeus protot yp e Na vigation in Madeus Tw
  • classes
  • f
na vigation can b e iden tied: con text- dep enden t na vigation dened b y explicit
  • b
jects declared in the do cumen t at some precise p
  • in
ts (buttons), and con text-indep enden t na vigation including generic presen tation manipulation
  • p
erations lik e stop, resume,
  • r
fast for- w arding and rewinding. Con text-indep enden t na vigation in m ultim edia do cumen ts is a na vigation not
  • nly
in space within do cumen t con ten ts, but in time as w ell, based
  • n
Tics and T acs presen tation instan ts. Na vigation using T acs can ha v e sev eral forms dep ending
  • n
the c hoice
  • f
whic h T acs p
  • in
ts to b e used as time mark ers. Three classes
  • f
na vigation using T acs ha v e b een stated:
  • Step
b y step na vigation, i. e. to go from
  • ne
instan t to the nearest T ac instan t (whether in the past
  • r
in the future).
  • Structural
na vigation, na vigation b et w een
  • b
jects lo cated at the same hierarc hical lev el
  • f
the h yp ergraph
  • User
dened na vigation among T acs dened b y the author.
slide-16
SLIDE 16 15 In terface
  • f
Madeus The in terface
  • f
  • ur
application is sho wn gure Fig. 5. Curren tly , the editing phase is mainly based
  • n
an in tegrated textual editor, although it is p
  • ssible
to mo dify the temp
  • ral
and spatial scenario b y selecting
  • b
jects during the presen tation phase and b y then setting the relation holding b et w een them with a palette
  • f
  • p
erators. Pro viding an in teractiv e authoring graphical user in terface is still a main issue. 4 Conclusion The aim
  • f
this pap er is t w
  • fold:
rstly , w e w
  • uld
lik e to con vince the m ultime- dia comm unit y
  • f
the in terest and the div ersit y
  • f
the authoring
  • f
m ultim edia do cumen ts issue. This is an activ e researc h eld that is strongly in terrelated with
  • ther
researc h areas (planning in rob
  • tics,
formal v erication, Human Computer In terface, etc.). The second aim
  • f
this pap er is to sho w the Madeus protot yp e con tributes to the m ultimedia authoring researc h domain b y bring- ing an in tegrated solution: using a declarativ e and hierarc hic sp ecication language
  • f
m ultim edia do cumen ts based
  • n
an extension
  • f
Allen's in terv al
  • p
erators and managing the temp
  • ral
part b y an extended form
  • f
temp
  • ral
constrain t net w
  • rks.
Finally , this ma y help in the recen t w
  • rk
undergoing at the W
  • rld
Wide W eb Consortium (W3C) to extend the curren t HTML format to the temp
  • ral
dimension 9 . References 1. ALLEN J.F., \Main taining Kno wledge ab
  • ut
T emp
  • ral
In terv als", Com- munic ations
  • f
the A CM, v
  • l.
V
  • l.
26, n um. No. 11, pp. 832-843, no v em b er 1983. 2. BUCHANAN M.C., ZELL WEGER P .T., \Automatic T emp
  • ral
La y
  • ut
Mec hanisms", Pr
  • c
e e dings
  • f
the First A CM International Confer enc e
  • n
Multime dia, pp. 341-350, Anaheim, California, august 1993. 3. BUL TERMAN D.C.A., V AN R OSSUM G., V AN LIERE R., \A Struc- ture for T ransp
  • rtable,
Dynamic Multimedia Do cumen ts", USENIX Confer enc e, pp. 137-1559, Nash ville, TN, june 1991. 4. DECHTER R., MEIRI I., PEARL J., \T emp
  • ral
Constrain t net w
  • rks",
A rticial Intel ligenc e, v
  • l.
49, , pp. 61-95, 1991. 5. DIRECTOR 4.0, \User's Guide", Macromedia. 6. JOURD AN M., LA Y A
  • ID
A N., SABR Y-ISMAIL L., \Time Represen ta- tion and Managemen t in Madeus : an authoring en vironmen t for Multi- media do cumen ts ", in Pr
  • c
e e dings
  • f
Multime dia Computing and Net-
slide-17
SLIDE 17 16 working 1997 , SPIE 3020, San-Jose, F ebruary 1997. 7. K OEGEL BUF ORD J. F. , Multime dia Systems, v
  • l.
A CM Press, Ad- dison W esley , , 1994. 8. KIM W., KENCHAMMANA-HOSEK OTE D. et SRIV AST A J. ., \Syn- c hronization Relation T ree : A mo del for T emp
  • ral
Sync hronization in Multimedia Presen tation", T e chnic al R ep
  • rt
TR92-42, Dept.
  • f
Com- puter Scienc e, Univ.
  • f
Minnesota, 1992 . 9. LA Y A
  • ID
A N, \Issues
  • n
T emp
  • ral
Represen tation
  • f
Multimedia Do c- umen ts", International Workshop
  • n
R e al-Time Multime dia and the World-Wide-Web (R TMW'96), Sophia-An tip
  • lis,
Octob er 1996. 10. LA Y A
  • ID
A N., SABR Y-ISMAIL L., \Main taining T emp
  • ral
Consistency
  • f
Multimedia Do cumen ts Using Constrain t Net w
  • rks",
Multime dia Computing and Networking 1996, M. F r e eman, P. Jar detzky, H. M. Vin, e d., pp. 124-135, SPIE 2667, San-Jos
  • e,
USA, february 1996. 11. LIA O W. and LI V., \Sync hronization
  • f
distributed m ultim edia sys- tems with user in teractions", International Confer enc e
  • n
Multime dia Mo deling , T
  • ulouse
1996. 12. SANNELLA M., MALONEY J., FREEMAN-BENSON B. and BORN- ING A., \Multi-w a y v ersus
  • ne
w a y constrain ts in user in terfaces: Ex- p eriences with the Deltablue algorithm", Softwar es Pr actic es and Exp e- rienc es, v
  • l.
23, n um. 5, 1993. 13. SONG J., DOGANA T A Y, KIM M. and T ANT A WI A., \Mo deling Timed User-In teractions in Multimedia Do cumen ts", Pr
  • c
e e dings
  • f
the IEEE International Confer enc e
  • n
Multime dia Computing and Systems, 1996. 14. V AN R OSSUM G., JANSEN J. , MULLENDER K. and BUL TERMAN D., \CMIF ed : a presen tation En vironmen t for P
  • rtable
Hyp ermedia Do cumen ts", Pr
  • c
e e dings
  • f
the A CM Multime dia Confer enc e, California (USA) 1993. 15. VID AL T and H.F AR GIER, \Con tingen t duration in temp
  • ral
CSP : from consistency to con trollabilities", 4th Int. Workshop
  • n
T emp
  • r
al R epr esentation and R e asoning (TIME97), Da ytona Beac h, Florida, USA, Ma y 10-11, 1997. 16. VILAIN M., KA UTZ H. A., \Constrain t Propagation Algorithms for T emp
  • ral
Reasoning", Pr
  • c
e e dings
  • f
AAAI, pp. 377-382, Philadelphia, august 1986. 17. WILLRICH R., de SA QUI-SANNES P ., SENA C P . and DIAZ M., \Hy- p ermedia Do cumen t Design Using the HTSPN Mo del", International Confer enc e
  • n
Multime dia Mo deling, T
  • ulouse,
1996.

View publication stats View publication stats