[PDF] - An Integrated Authoring And Presentation Environment For Interactive PDF Document

SLIDE 1

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/2362328

An Integrated Authoring And Presentation Environment For Interactive Multimedia Documents

Article · June 2001

Source: CiteSeer

CITATIONS

11

READS

28 4 authors, including: Some of the authors of this publication are also working on these related projects: Structural Health monitoring using Wireless Sensor Networks View project IoT-Based Fire Alarm System View project Cécile Roisin University of Grenoble

78 PUBLICATIONS 865 CITATIONS

SEE PROFILE

Loay Sabry Ismail Qatar University

31 PUBLICATIONS 344 CITATIONS

SEE PROFILE

All content following this page was uploaded by Cécile Roisin on 06 February 2013.

The user has requested enhancement of the downloaded file.

SLIDE 2 AN INTEGRA TED A UTHORING AND PRESENT A TION ENVIR ONMENT F OR INTERA CTIVE MUL TIMEDIA DOCUMENTS M. JOURD AN, N. LA Y AID A, C. R OISIN, L. SABR Y-ISMAIL OPERA pr

je

ct, INRIA R h^

ne-A

lp e s 655 A venue de l'Eur

p

e, 38330 Montb

nnot,

F r anc e email: fMuriel.Jour dan, Nabil.L ayai da, Ce cile.R

isin,

L

ay.Sabryg@inrial

p es .fr Abstract In this pap er, w e presen t

ur

con tribution to meet the main requiremen ts for pro viding an authoring en vironmen t for in teractiv e m ultimedia do cumen ts. The particularit y

f
ur

approac h is to consider m ultimedia authoring as a whole, from sp ecication issues to execution requiremen ts. The results

f

these theoretical w

rks

are v alidated in to a real running application called Madeus. 1 In tro ducti

n

The recen t adv ances in m ultim edia systems, together with the adv en t

f

high sp eed net w

rks,

ha v e lead to the emergence

f

a new generation

f

applications and will considerably aect the existing

nes.

In particular, h yp ermedia applications lik e the WWW

riginally

designed for text and simple graphics are mo ving to w ards a seamless in tegration

f

data lik e audio, video, plugins and legacy programs (applets). In con trast to the static nature

f

text-orien ted do cumen ts, the addition

f

time-based media

b

jects to do cumen ts requires the consideration

f

another dimension

f

their underlying arc hitecture: the do cumen t dynamics

r

tem- p

ral

dimension. Basic media

b

jects, lik e video, ha v e in trinsic durations and they can b e temp

rally
rganized

b y the author. This temp

ral
rganization,

so-called the temp

r

al sc enario

f

the do cumen t, raises a n um b er

f

problems and in spite

f

the big amoun t

f

researc h w

rk

ac hiev ed in this area, these issues are still far from b eing mastered. W e ha v e iden tied four main asp ects to b e considered when designing a m ultim edia do cumen t authoring en vironmen t:

Expressiv

eness: It measures the abilit y

f

the temp

ral

language to co v er all the p

ssible

scenarios.

Consistency

c hec king: It ensures that the scenario sp ecied b y the author is consisten t during the edition phase.

SLIDE 3 2

Simplicit

y: It measures the abilit y

f

naiv e users in using the authoring to

l.
Run-time

monitoring

f

the presen tation: This requiremen t is related to the presen tation la y er and its abilit y to la y

ut

the do cumen t in compli- ance with the author's sp ecication. This pap er addresses these dieren t issues, from the do cumen t sp ecication, construction and manipulation to the application arc hitecture and

rga-

nization. The w

rk

ac hiev ed is presen ted through Madeus: an exp erimen tal authoring and presen tation to

l

dev elop ed in

ur

pro ject. The pap er is

rganized

in t w

parts.

In the rst

ne,

w e explain the ab

v

e-men tioned issues,

utlining

the problems raised b y eac h

f

them. In the second part

f

the pap er, w e presen t

ur

approac h and the design c hoices that ha v e b een made: rstly , pro viding the author with a declarativ e sp ecication language

f

m ultim edia do cumen ts; secondly , using an extension

f

temp

ral

constrain t net w

rks

to manage the temp

ral

scenario

f

a do cumen t. Moreo v er, w e dene a general arc hitecture

f

the Madeus application to demonstrate the v alidit y

f

the t w

ab
v

e-men tioned c hoices. 2 Multimedi a authoring issues 2.1 Example

f

a F amily Pr esentation Let us consider the example

f

a m ultim edia do cumen t presen ting a family

f

four mem b ers. Eac h p erson can b e presen ted b y a sequence comp

sed
f

a still picture

f

him when he w as a bab y , a short video ab

ut

his life (its duration is ab

ut
ne

min ute) and a recen t photo. Then eac h

f

these three elemen ts is asso ciated with textual elemen ts whic h giv es some commen ts to the pictures. A t an y time, the reader

f

the do cumen t can in terrupt the video in

rder

to get in to the presen tation

f

the last still picture. The sequence (named S in the rest

f

the pap er) is comp

sed
f

the three elemen ts previously men tioned, the three asso ciated textual parts and the button

f

in teraction. In addition, eac h family mem b er can add a p ersonal audio message (its duration is ab

ut

1 min ute and 30 sec.). This audio message m ust start with the sequence S and m ust end b efore it. Fig. 1 presen ts a p

ssible

execution

f

the scenario for

ne

mem b er

f

the family . In this execution, the reader (b y clic king

n

the but ton) in terrupts the video 40 seconds after its b eginning. The rst screen

f

the do cumen t is comp

sed
f

a still picture

f

all the mem b ers

f

the family . The reader can c ho

se

to pla y the presen tation

f
ne

p erson. Once this presen tation ends, the rst screen sho ws up again.

SLIDE 4 3

95

Time(sec)

40 80

Button SP1 Video SP2 T3 T2 T1 Audio

Figure 1: P

ssible

execution for the presen tation

f
ne

p erson 2.2 Expr essiveness The rst requiremen t is related to the kind

f

temp

ral

scenario that an authoring system allo ws an author to express. W e presen t in the follo wing the four basic features that w e consider necessary to build realistic scenarios.

A

wide v ariet y

f

basic

b

jects: basic elemen ts lik e text, video, audio, still pictures, virtual animation, JA V A applets should b e supp

rted.

It is necessary to distinguish b et w een t w

kinds
f
b

jects: con trollable

b

jects whose duration v alues can b e c hosen b y the system from a range

f

p

ssible

v alues, and uncon trollable

b

jects whic h tak e a duration v alue at the presen tation time. F

r

instance, a video duration can ha v e a range

f

p

ssible

v alues dep ending

n

its acceptable pla ying rates. W e kno w the lo w er and upp er b

und

v alues, but w e cannot predict its eectiv e v alue during the presen tation.

Global

and lo cal in teractions: The idea is to pro vide the reader with t w

kinds
f

in teractions: global

nes

whose scop e is to in terrupt all the curren t activities in the do cumen t. In

ur

example, the rst screen is comp

sed
f

a set

f

global in teractions. A t the

pp
site,

the button whic h in terrupts the video in the sequence is a lo cal in teraction since it do es not in terrupt the audio message whic h is activ e at the same time

f

the video. A user-in teraction is a t ypical uncon trollable

b

ject whith no upp er b

und.
Causal

relations b et w een ev en ts: Some b eha viors

f

a scenario can b e expressed as causal relationships b et w een temp

ral

ev en ts (start

r

end

f
b

jects). In

ur

example, this is the case with the button asso ciated to the video: if the reader clic ks

n

it, the video is in terrupted.

SLIDE 5 4

Relativ

e p

sitioning
f

ev en ts (sync hronization): They are used to express the equalit y

r

the precedence

f

temp

ral

ev en ts. In

ur

example, suc h sync hronization is frequen tly used: textual commen tary related to comp

nen

ts

f

the sequence m ust start (resp. end) when the comp

nen

t related to it starts (resp. ends). This is also the case with the audio message whic h m ust start at the same time as the sequence S and nish b efore it. Some authoring systems lik e CMIF ed 14 , are restricted to causal relations. Because relativ e p

sitioning
f

ev en ts ma y lead to inconsistencies, it a v

ids

the dicult task to v alidate the scenario. On the

ther

hand,

ther

systems lik e ISIS 13 ,

nly

consider relativ e p

sitioning.

In this approac h the scenario sp ecication is mo delled b y a set

f

linear inequalities to b e solv ed. Duration v alues

f
b

jects are c hosen b y the system from an in terv al

f

lo w er and upp er b

und

v alues. The system, with the help

f

a temp

ral

formatter, statically computes a time-map whic h satises all the constrain ts. The t w

ma

jor dra wbac ks

f

this approac h are:

Lo

cal user in teractions cannot b e mo delled.

Ob

jects lik e video, audio, JA V A applets are mo delled b y static durations and therefore ma y b e in terrupted (resp. dela y ed) if to

long

(resp. to short) at presen tation time. In fact, merging the t w

t

yp es

f

sp ecication: causal and relativ e p

si-

tioning is a dicult task. As an example, the sequence S

f

Figure 1 con tains a causal relation, since a user-in teraction can in terrupt the video, together with a relativ e

ne:

S m ust start at the same time as an audio message and m ust end after it. Causal relations in tro duce the distinction

f

t w

kinds
f

ending ev en ts for eac h

b

ject: the normal end and the in terrupted

ne.The

dicult y is to dene clearly whic h end is in v

lv

ed when using relativ e expressions. 2.3 Static A nalysis

f

Do cument Sp e cic ation The temp

ral

scenario

f

m ultim edia do cumen t m ust b e v alidated during the authoring phase and therefore b efore the presen tation phase. It is

b

vious that the presence and the nature

f

temp

ral

errors are strongly related to (1) the w a y in whic h the author sp ecies the do cumen t and (2) the expressiv eness p

w

er

f

the sp ecication language itself. Approac hes limited to causal

p

erators (lik e CMIF ed 3 ) ha v e no problems

f

temp

ral

consistency . A t the

pp
site,

relativ e p

sitioning
f

ev en ts can lead to sp ecify erroneous scenarios. The trivial case

f

inconsistency consists

SLIDE 6 5 in sp ecifying that t w

b

jects ha v e the same duration, although their in terv al

f

p

ssible

duration ha v e an empt y in tersection. Another example is to sp ecify that the end

f

a video is sync hronized with the end

f

an audio. If the video and the audio are uncon trollable

b

jects, their duration can

nly

b e kno wn when they end, it is therefore not p

ssible

to guaran tee that the sp ecication is v alid. In all these cases the system ha v e to w arn the author

f

the existence

f

inconsistencies and to help him in prev en ting them. Inconsistencies in the case

f

relativ e p

sitioning
f

temp

ral

ev en ts and con trollable

b

jects can b e easily iden tied. In this case, it is a classical problem

f

constrain t satisfaction: a sp ecication is consisten t if and

nly

if there exists a v alue for eac h starting and ending p

in

t whic h satises all the constrain ts. Uncon trollable

b

jects are far more complex to handle. The notion

f

consistency b ecomes more complex 10 since all the p

ssible

durations m ust b e tak en in to accoun t for the uncon trollable

b

jects together with the

rdering

b et w een ev en ts. This situation arises in

ur

w

rking

example: in

rder

to b e sure that the sequence S will nish after the audio message, the duration

f

the second still picture

f

the sequence dep ends

n

the real duration

f

the video, whic h can b e in terrupted b y a user-in teraction. The constrain t satisfaction problem with uncon trollable v ariables in a constrain t satisfaction problem is still an

p

en issue and is curren tly studied in another area: planning in rob

tics

15 . 2.4 User-F riend ly Interfac e T

da

y , most p

pular

commercial authoring to

ls

lik e Director 5 use a propri- etary script language (lik e Lingo) and, as so

n

as

ne

needs adv anced features, programming is still the

nly

solution for sp ecifying scenarios. Ho w ev er, build- ing an in teractiv e m ultim edia do cumen t should b e p erformed b y non-sp ecialists with higher lev el user in terface facilities 7 . W e claim that the authoring ac- tivit y should b e a cyclic "sp ecify , test and mo dify" pro cess with the follo wing features:

Spatial

and temp

ral

p

sitioning
f

do cumen t comp

nen

ts m ust b e expressed in a relativ e w a y for man y reasons : rst it is easier to use a declarativ e st yle

f

sp ecication than to learn a new programmi ng language; second a spatial

r

temp

ral
rganization

describ ed with relativ e p

sitionning
p

erators is easier to mo dify than absolute p

sitionning

as in timeline approac hes; third this is the system (the formatter part) and not the author whic h computes the v alues asso ciated with eac h

b

ject (for instance in the time dimension: its starting p

in

t and its duration). Ho w ev er, suc h relativ e approac hes m ust b e asso ciated with p

w

erful vi- sual in terface in

rder

to mak e the author's task easier. Esp ecially in

SLIDE 7 6 the time dimension, it is necessary to help the author to understand the global b eha vior implied b y the set

f

lo cal constrain ts. Let us not conclude that temp

ral

formatting is confron ted with the uncertain t y

f

the duration

f

uncon trollable

b

jects, lo cal user-in teractions and causal

p

erators.

Edition

and presen tation phases m ust b e in tegrated. This allo ws fast toggling b et w een the t w

phases,

so that the author can in teract during the presen tation phase to mo dify the do cumen t. This is a w a y to approac h the Wysiwyg paradigm successfully used for static do cumen ts but that cannot b e extended to m ultimedia do cumen ts due to their tem- p

ral

dimension.

Hierarc

hic decomp

sition
f

the do cumen t m ust b e supp

rted.

Ho w ev er, this hierarc h y m ust not b ecome to

constraining

for the author. Some approac hes (SR T, CMIF ed) mo del the do cumen t b y a tree structure with a temp

ral
p

erator asso ciated to eac h no de. This solution has t w

ma

jor dra wbac ks: some temp

ral

comp

sitions

can not b e sp ecied and the mo dication

f

an existing scenario b ecomes dicult. 2.5 The A r chite ctur e

f

the Pr esentation L ayer Sev eral functionalities for the presen tation la y er ha v e b een iden tied in 11 : (1) pro viding the reader with an accurate rendition

f

what the author has sp ecied, (2) supp

rting

T emp

ral

Access Con trol

p

erations lik e: stop, resume, fast-forw ard, fast-rewind, as w ell as h yp erlinks to allo w the user to manipulate the m ultim edia do cumen t presen tation and (3) supp

rting

the distribution

f

media

b

jects across the net w

rk.

Multimedia applications are real time systems and CPU time consuming, so the applications ha v e to gh t against limitati

ns
f

resources (suc h as CPU a v ailabili t y , disks bandwidth, net w

rks

bandwidth, net w

rk

accessing dela ys, a v ailabili t y

f

media c hannels, data buers sizes, etc.). F

r

example, media

b

jects lik e video and still pictures need a considerable CPU pro cessing time for image decompression, while media

b

jects lik e audio and video, are really sensitiv e to the system load as they ha v e deadline times that m ust b e resp ected. So, Qualit y

f

Service (QoS) issues m ust b e tak en in to consideration to guaran- tee minim um requiremen ts

f

p erformance. Examples

f

situations that m ust b e a v

ided

are:

The

author sp ecies that t w

videos

pla y in sequence. During the presentation, the rst nishes but a considerable dela y b et w een the t w

videos

is encoun tered (due to the time needed b y decompression

r

the net w

rk

accessing dela y), in a w a y that it aects the qualit y

f

the presen tation.

SLIDE 8 7

The

author sp ecies that t w

videos

pla y in parallel together with an audio and the CPU is

v

erloaded b y the decompression

f

the videos at the cost

f

the qualit y

f

the audio. Resource limitati

ns

are unpredictable in nature. So, the task

f

the presen tation system can b e simplied if dela y v alues (of net w

rk

and disk access as w ell as buer's queue w aiting time) and CPU a v ailabil it y can b e appro ximately predicted. As a result, it is imp

rtan

t to ha v e an ecien t presen tation la y er that can mak e a trade-o b et w een the qualit y

f

presen tation and the a v ailabilit y

f

resources, in suc h a w a y that it giv es the reader with an acceptable rendering

f

the m ultim edia do cumen t. 2.6 Synthesis In

rder

to summarize these requiremen ts, w e can

utline

the v e follo wing p

in

ts whic h are considered as cen tral issues in m ultim edia authoring:

Relativ

e p

sitioning
f
b

jects are necessary due to the user-in terface requiremen ts but it implies b

th

the presence

f

a temp

ral

formatter nd a static c hec k er allo wing the detection

f

temp

ral

inconsistencies.

Causal
p

erators are necessary to pro vide the reader with lo cal in teractions. They in tro duce for eac h

b

ject, the distinction b et w een t w

kinds
f

ending: normal and in terrupted.

Some
b

jects ha v e no static con trollable duration and their presen tation should not b e in terrupted. This increases the complexit y

f

the notion

f

temp

ral

inconsistencies and temp

ral

formatting.

Hierarc

hic decomp

sition

is necessary to represen t the logical structure whic h do es not necessarily corresp

nd

to the temp

ral
rganization
f

the do cumen t.

The

presen tation phase m ust b e in tegrated with the editing phase in

rder

to reduce the editing cycle. 3 Madeus Approac h Madeus has prop

sed

t w

imp
rtan

t c hoices: rstly , pro viding the author with a declarativ e and hierarc hic sp ecication language for m ultim edia do cumen ts based

n

an extension

f

Allen's in terv al

p

erators; secondly , using an extension

f

temp

ral

constrain t net w

rks

to manage the temp

ral

scenario

f

a do cumen t during its static analysis, and also to supp

rt

the presen tation

SLIDE 9 8 phase. W e start b y presen ting the sp ecication language used, then the in ternal structure whic h serv es as the basic data structure to manage b

th

edition and presen tation phases. 3.1 A Mark-up F

rmat

for Multime dia Do cuments Basic Ob jects Madeus basic

b

jects are video, audio, still picture, text, temp

ral

dela y and button. Buttons are used in the scenario to in tegrate lo cal user-in teractions. All

b

jects are abstracted as temp

ral

in terv als. The in terv al asso ciated to a button starts at the instan t when the button can b e activ ated and ends when it is really activ ated. In terv als are t yp ed as con trollable (text, still pic ture, temp

ral

dela y)

r

uncon trollable (video, audio, button). A lo w er b

und

and an upp er b

und
f

duration are asso ciated with eac h

b

ject. Upp er b

und

duration can b e innite, this is the case

f

a button. In case

f

a con trollable in terv al, the temp

ral

formatter can c ho

se

an y v alue within these b

unds.

In case

f

an uncon trollable in terv al, the temp

ral

formatter is informed that the real duration

f

the

b

ject will b e within these b

unds.

Allen's In terv al Algebra and Our Extensions Allen's algebra 1 expresses relativ e p

sitioning
f

in terv als. W e ha v e extended it with causal

p

erators to express causal relationships b et w een ending ev en ts:

A

par minB: the t w

in

terv als start at the same time and the shorter causes the termination

f

the

ther
ne;
A

par master(A)B: the t w

in

terv als start at the same time and the end

f

A (whic h is the master) causes the termination

f

B, if and

nly

if B is still pla ying. W e ha v e, also, extended the algebra with the quan tied dela ys b et w een in terv als. Hierarc hic Decomp

sition

A do cumen t is comp

sed
f

a set

f
b

jects (basic and comp

site)

related in time. A hierarc hic decomp

sition

giv es the logical structure

f

the do cumen t, where the siblings at eac h hierarc hical lev el can b e temp

rally

related. Fig. 2 illustrates the hierarc hic decomp

sition

and the sym b

lic

represen tation

f

the w

rking

example. The principal do cumen t has a ro

t
b

ject called Men u whic h has four c hildren (Option1,...., Option4)

f

t yp e button. These c hildren are temp

rally

related b y the par min

p

erator, and eac h

ne
f

them is asso ciated

SLIDE 10 9

Menu Document Symbolic Representation Logical Structure Hyperlinks

. . . . . . . . . . . . . . .

Biography Documents Menu Option1 Optioni Option4 Biographyi

S Audio SP1 T1 Video T2 Button SP2 T3

Menu { LIST-PARMIN (Option1 , ..., Option i, ..., Option 4 ) . . . Optioni { . . . . . . <href> Biography i } } Biography i Audio STARTS { S S { SP1 EQUALS T1 SP1 Video Button Video SP2 MEETS Video EQUALS T2 PARMIN Video MEETS SP2 EQUALS T3 } } Biography Biography4 1

Figure 2: Sym b

lic

and hierarc hic sp ecication

f

the w

rking

example with a h yp erlink to another corresp

nding

do cumen t. It is imp

rtan

t to notice that the hierarc hic decomp

sition

denes b

undaries

for the scop e

f

temp

ral

relations. Hyp erlinks are managed lik e in HTML b y asso ciating sp ecic attributes to

b

jects. This attribute denes a comp

nen

t

f

the do cumen t as the target

f

the link whic h is accessed when the user clic ks

n

the

b

ject. 3.2 Extende d T emp

r

al Constr aint Networks The temp

ral

scenario

f

the do cumen t is built incremen tally (temp

ral

relations are analyzed

ne

b y

ne)

in to a hierarc h y

f

extended temp

ral

constrain t net w

rks.

Eac h time a relation is analyzed, the consistency

f

the scenario is c hec k ed. W e rst presen t the constrain t temp

ral

net w

rks

formalism 4 and then presen t the extended form w e prop

se.

Moreo v er, w e p

in

t

ut

in the second part ho w to c hec k the consistency

f

a scenario. T emp

ral

Constrain t Net w

rks

F

rmalism

A temp

ral

constrain t net w

rk

is a Directed Acyclic Graph (D A G) where no des represen t time p

in

ts and eac h edge (i, [min,max], j) represen ts a temp

ral

SLIDE 11 10

Button Video SP2 T3 delay T2 Audio SP1 T1

Figure 3: An example

f

Madeus constrain t net w

rk

in terv al with a duration range from min to max b et w een the t w

time

p

in

ts i and j. Dieren t kinds

f

algorithms exist to answ er questions lik e: do es a solution exist? Can a particular set

f

v alues b e considered as a solution for the problem? In

ur

con text, a temp

ral

constrain t net w

rk

describ es the temp

ral
rdering
f

media

b

jects. The range

f

p

ssible

durations

f

a dynamic

b

ject is mo delled b y an edge lab elled b y its appropriate in terv al. Before going further, w e in tro duce the follo wing denition: "A temp

ral

c hain [i,j,.. .s] is a sequence

f

con tiguous edges where eac h edge is

nly

related to

ne

successor and

ne

predecessor". Madeus T emp

ral

Constrain t Net w

rks

In MADEUS, the previous denition

f

a constrain t net w

rk

has b een extended to capture t w

t

yp es

f

information: the con trollabilit y

f

the duration and the causalit y relationships. An additional lab elling

f

in terv als is added to distinguish whether the temp

ral

distance is con trollable

r

uncon trollable (resp ectiv ely [min,max]c and [min,max]u). In

rder

to express causalit y in temp

ral

scenarios, w e add causal relations b et w een temp

ral

instan ts, i.e. no des

f

the net w

rk.

F

r

instance, Fig. 3 sho ws the extended net w

rks

represen ting the presen tation for

ne

p erson in

ur

w

rking

example. The causal

p

erator par min is represen ted b y a bidirectional arro w. A p

sitiv

e dela y is in tro duced since the audio m ust nish b efore the sequence S. Static Analysis Giv en suc h a temp

ral

net w

rk,

the v alidation phase is equiv alen t to c hec k the existence

f

at least

ne

solution

f

the sp ecication. The three kinds

f

inconsistencies that can

ccur

in the sp ecication

f

a scenario ha v e b een

SLIDE 12 11 iden tied previously in 10 : qualitativ e, quan titativ e and indeterministic. These dieren t t yp es

f

consistency c hec king can b e p erformed as follo ws:

qualitativ

e: a top

logical

sort

f

the constrain t net w

rk

is main tained in

rder

to detect the existence

f

cycles.

quan

titativ e: constrain t propagation tec hniques 16 can b e applied eac h time the net w

rk

is mo died. W e c hec k for all parallel c hains if they ha v e non-empt y in tersection

f

their allo w able duration ranges. Algorithms used for the constrain t propagation are p

lynomial

in time and space and supp

rt

incremen tal mo dications

indeterministic:

the idea here is to p erform w

rst

case analysis to

btain

dates

f

the form (inferior, sup erior)

n

eac h no de

f

the net w

rk.

The idea here is to see if it is p

ssible

to reco v er dynamically , at the presen tation phase, the indeterministic b eha vior

f

uncon trollable in terv als b y using the exibilit y

f
ther

con trollable in terv als to b e pla y ed in the future 10 . 3.3 Pr esentation phase Once the temp

ral

consistency

f

the scenario is c hec k ed, the dicult y is to compute the solution dynamically at the presen tation stage. It cannot b e done completely statically due to the uncon trollable durations. The role

f

the presen tation phase is t w

fold

: rst to p erform the sc hedul- ing

f

the presen tation

p

erations

f

ev ery basic

b

jects (starting/ nishing, mapping/unm apping, ...); second to handle na vigation inside the doumen t as requested b y the reader (h yp erlink activ ation, jump inside the do cumen t, rewind, ...). Starting from this p

in

t, the temp

ral

constrain t net w

rk

is used b y the presen tation la y er as a task description

f

the presen tation: it is a h yp ergraph structure

f

nested comp

nen

ts where lea v es are basic media

b

jects and no des are comp

site
b

jects (see Fig. 4). This h yp ergraph con tains the follo wing information that ha v e b een statically computed : durations and dates

f

basic and comp

site
b

jects. During this computation, uncon trolable

b

jects ha v e b een handle as con trolable

nes

so they ha v e also an asso ciated duration. This duration will b e dynamically adjusted during the presen tation phase, as briey explained in the follo wing section. The adv an tage

f

sharing the same structure for editing and presen ting a m ultim edia do cumen t is the rapidit y b y whic h w e can toggle b et w een these t w

phases.

W e briey describ e ho w temp

ral

information is managed in Madeus in

rder

to driv e the presen tation and to handle na vigation through the do cumen t.

SLIDE 13 12

Biography i Audio

S

delay Button SP1 Video SP2 T1 T2 T3

Figure 4: An example

f

the h yp ergraph

f

a do cumen t ready to b e presen ted Time managing during presen tation phase T emp

ral

informati

n

managemen t Eac h

b

ject in Madeus is asso ciated with a set

f

v ariables and metho ds for its managemen t b y the system. Among them, the v ariables used for time managemen t are the follo wing:

State:

suc h as activ e, susp ended and terminated.

Nature:

BASIC

r

COMPOSITE.

StartTime:

instan t at whic h the

b

ject starts pla ying.

EndTime:

instan t at whic h the

b

ject ends pla ying.

Exp

ected duration (statically computed). T

handle

presen tation

f

m ultimedia do cumen ts together with na vigation

p

erations, w e need to dene a pr esentation c

ntext

that holds the temp

ral

information ab

ut

a presen tation at a certain time instan t called a temp

r

al cut. If w e consider a temp

ral

cut at a time instan t t for example, w e are in terested in kno wing whic h

b

jects are concurren tly active at that precise instan t

f

time and what p

in

t in their presen tation lifetimes they ha v e reac hed (an

set

v alue is calculated as the dierence b et w een time instan t t and the

b

ject's start time). So, the temp

ral

cut

f

scenario S Ro

t

at instan t t for a do cumen t iden- tied b y its Ro

t
b

ject is dened as the set

f
rdered

pairs

f

all active basic

b

jects together with their asso ciated

set

v alues. It is dened b y the function k t as follo ws:

SLIDE 14 13 k t (Ro

t)

= f(Ob ject, Oset (Ob ject)) / (Ob ject 2 S Ro

t

) and (Nature (Ob ject) = BASIC) and (StartTime (Ob ject)

t

< EndTime (Ob ject)) and (Oset (Ob ject) = t

StartTime

(Ob ject)) g Based

n

this information, elapsed p erio d

f

presen tation

f
b

jects at an y cut instan t can b e dynamically calculated b y the presen tation manager: the k t function is recursiv ely applied for eac h activ e comp

site
b

ject un til the lo w est lev els are reac hed (basic

b

jects) in

rder

to co v er the whole h yp ergraph. Time scaling for m ultimedi a presen tation Another imp

rtan

t notion to iden tify for a m ultimedia presen tation system is the time scaling, i.e. what are the relev an t instan ts at whic h the presen tation manager has to p erform some actions. F

r

that purp

se,

w e ha v e in tro duced t w

notions
f

presen tation instan ts: the Tics and the T acs. Denitio n

f

presen tati

n

Tics: A Tic is a time instan t in the lifetime

f

the presen tation

f

a scenario at whic h a set

f

presen tation actions m ust b e carried

ut.

So the presen tation

f

a scenario can b e represen ted as a sequence

f

Tics (as clo c k tics), suc h as the Tic at t= 100 in Fig. 4. Eac h t w

consequen

t Tics are separated b y a time step that is the minim al in terv al needed for the presen tation

f

the

b

jects (for example 1/30 sec

r

1/25 sec is required for pla ying video frames). Denitio n

f

presen tati

n

T acs: T acs are the Tics corresp

nding

to the time instan ts at whic h basic (and comp

site)
b

jects start and nish: these instan ts corresp

nd

to the no des

f

the h yp ergraph represen ting the do cumen t. Driving the presen tati

n

The presen tation progresses as follo ws: at ev ery Tic instan t, the system p erforms the

p

erations required as they are dened b y the h yp ergraph structure. Firing a no de consists in starting all the

b

jects (text, video, dela y , audio, ...) asso ciated with its

utgoing

edges. A no de can b e red if and

nly

if all the

b

jects asso ciated with its incoming edges ha v e nished. Causal relations

n

eac h no de are used to propagate ending information. Duration

f

uncon trollable

b

jects are

bserv

ed. After computing the dierence b et w een the exp ected duration (statically computed) and the

bserv

ed

ne,

some con trollable durations

f
b

jects pla y ed in the future are mo died in

rder

to resp ect the sync hronization conditions 10 . This in ternal structure whic h is the supp

rt
f

the presen tation phase is ric h enough to an ticipate the executional state

f

the whole do cumen t at dif- feren t future instan ts in the presen tation life time. This abilit y

f

an ticipation will allo w us to b etter manage the media

b

jects distribution

n

the W

rld

Wide W eb (pre-fetc hing for example).

SLIDE 15 14 Figure 5: In terface

f

the Madeus protot yp e Na vigation in Madeus Tw

classes
f

na vigation can b e iden tied: con text- dep enden t na vigation dened b y explicit

b

jects declared in the do cumen t at some precise p

in

ts (buttons), and con text-indep enden t na vigation including generic presen tation manipulation

p

erations lik e stop, resume,

r

fast for- w arding and rewinding. Con text-indep enden t na vigation in m ultim edia do cumen ts is a na vigation not

nly

in space within do cumen t con ten ts, but in time as w ell, based

n

Tics and T acs presen tation instan ts. Na vigation using T acs can ha v e sev eral forms dep ending

n

the c hoice

f

whic h T acs p

in

ts to b e used as time mark ers. Three classes

f

na vigation using T acs ha v e b een stated:

Step

b y step na vigation, i. e. to go from

ne

instan t to the nearest T ac instan t (whether in the past

r

in the future).

Structural

na vigation, na vigation b et w een

b

jects lo cated at the same hierarc hical lev el

f

the h yp ergraph

User

dened na vigation among T acs dened b y the author.

SLIDE 16 15 In terface

f

Madeus The in terface

f
ur

application is sho wn gure Fig. 5. Curren tly , the editing phase is mainly based

n

an in tegrated textual editor, although it is p

ssible

to mo dify the temp

ral

and spatial scenario b y selecting

b

jects during the presen tation phase and b y then setting the relation holding b et w een them with a palette

f
p

erators. Pro viding an in teractiv e authoring graphical user in terface is still a main issue. 4 Conclusion The aim

f

this pap er is t w

fold:

rstly , w e w

uld

lik e to con vince the m ultimedia comm unit y

f

the in terest and the div ersit y

f

the authoring

f

m ultim edia do cumen ts issue. This is an activ e researc h eld that is strongly in terrelated with

ther

researc h areas (planning in rob

tics,

formal v erication, Human Computer In terface, etc.). The second aim

f

this pap er is to sho w the Madeus protot yp e con tributes to the m ultimedia authoring researc h domain b y bring- ing an in tegrated solution: using a declarativ e and hierarc hic sp ecication language

f

m ultim edia do cumen ts based

n

an extension

f

Allen's in terv al

p

erators and managing the temp

ral

part b y an extended form

f

temp

ral

constrain t net w

rks.

Finally , this ma y help in the recen t w

rk

undergoing at the W

rld

Wide W eb Consortium (W3C) to extend the curren t HTML format to the temp

ral

dimension 9 . References 1. ALLEN J.F., \Main taining Kno wledge ab

ut

T emp

ral

In terv als", Com- munic ations

f

the A CM, v

l.

V

l.

26, n um. No. 11, pp. 832-843, no v em b er 1983. 2. BUCHANAN M.C., ZELL WEGER P .T., \Automatic T emp

ral

La y

ut

Mec hanisms", Pr

c

e e dings

f

the First A CM International Confer enc e

n

Multime dia, pp. 341-350, Anaheim, California, august 1993. 3. BUL TERMAN D.C.A., V AN R OSSUM G., V AN LIERE R., \A Struc- ture for T ransp

rtable,

Dynamic Multimedia Do cumen ts", USENIX Confer enc e, pp. 137-1559, Nash ville, TN, june 1991. 4. DECHTER R., MEIRI I., PEARL J., \T emp

ral

Constrain t net w

rks",

A rticial Intel ligenc e, v

l.

49, , pp. 61-95, 1991. 5. DIRECTOR 4.0, \User's Guide", Macromedia. 6. JOURD AN M., LA Y A

ID

A N., SABR Y-ISMAIL L., \Time Represen tation and Managemen t in Madeus : an authoring en vironmen t for Multi- media do cumen ts ", in Pr

c

e e dings

f

Multime dia Computing and Net-

SLIDE 17 16 working 1997 , SPIE 3020, San-Jose, F ebruary 1997. 7. K OEGEL BUF ORD J. F. , Multime dia Systems, v

l.

A CM Press, Ad- dison W esley , , 1994. 8. KIM W., KENCHAMMANA-HOSEK OTE D. et SRIV AST A J. ., \Syn- c hronization Relation T ree : A mo del for T emp

ral

Sync hronization in Multimedia Presen tation", T e chnic al R ep

rt

TR92-42, Dept.

f

Com- puter Scienc e, Univ.

f

Minnesota, 1992 . 9. LA Y A

ID

A N, \Issues

n

T emp

ral

Represen tation

f

Multimedia Do c- umen ts", International Workshop

n

R e al-Time Multime dia and the World-Wide-Web (R TMW'96), Sophia-An tip

lis,

Octob er 1996. 10. LA Y A

ID

A N., SABR Y-ISMAIL L., \Main taining T emp

ral

Consistency

f

Multimedia Do cumen ts Using Constrain t Net w

rks",

Multime dia Computing and Networking 1996, M. F r e eman, P. Jar detzky, H. M. Vin, e d., pp. 124-135, SPIE 2667, San-Jos

e,

USA, february 1996. 11. LIA O W. and LI V., \Sync hronization

f

distributed m ultim edia systems with user in teractions", International Confer enc e

n

Multime dia Mo deling , T

ulouse

1996. 12. SANNELLA M., MALONEY J., FREEMAN-BENSON B. and BORN- ING A., \Multi-w a y v ersus

ne

w a y constrain ts in user in terfaces: Ex- p eriences with the Deltablue algorithm", Softwar es Pr actic es and Exp e- rienc es, v

l.

23, n um. 5, 1993. 13. SONG J., DOGANA T A Y, KIM M. and T ANT A WI A., \Mo deling Timed User-In teractions in Multimedia Do cumen ts", Pr

c

e e dings

f

the IEEE International Confer enc e

n

Multime dia Computing and Systems, 1996. 14. V AN R OSSUM G., JANSEN J. , MULLENDER K. and BUL TERMAN D., \CMIF ed : a presen tation En vironmen t for P

rtable

Hyp ermedia Do cumen ts", Pr

c

e e dings

f

the A CM Multime dia Confer enc e, California (USA) 1993. 15. VID AL T and H.F AR GIER, \Con tingen t duration in temp

ral

CSP : from consistency to con trollabilities", 4th Int. Workshop

n

T emp

r

al R epr esentation and R e asoning (TIME97), Da ytona Beac h, Florida, USA, Ma y 10-11, 1997. 16. VILAIN M., KA UTZ H. A., \Constrain t Propagation Algorithms for T emp

ral

Reasoning", Pr

c

e e dings

f

AAAI, pp. 377-382, Philadelphia, august 1986. 17. WILLRICH R., de SA QUI-SANNES P ., SENA C P . and DIAZ M., \Hy- p ermedia Do cumen t Design Using the HTSPN Mo del", International Confer enc e

n

Multime dia Mo deling, T

ulouse,

1996.

View publication stats View publication stats