SLIDE 1
ACADI: Automatic Character (in Audiovisual Document) Indexing - - PDF document
ACADI: Automatic Character (in Audiovisual Document) Indexing - - PDF document
ACADI: Automatic Character (in Audiovisual Document) Indexing Frdric GIANNI and Julien PINQUIER Ewa KIJAK IRIT - Universit Paul Sabatier IRISA 118, route de Narbonne Campus de Beaulieu 31062 Toulouse Cedex 9, FRANCE 35042 Rennes
SLIDE 2
SLIDE 3
2.4 Audio/video association method
❙♦♠❡ r❡❝❡♥t ♠❡t❤♦❞s ✉s❡ ❜♦t❤ ❛✉❞✐♦ ❛♥❞ ✈✐❞❡♦ ❝✉❡s ❢♦r ✐♠✲ ♣r♦✈✐♥❣ ♣❡rs♦♥ ✐❞❡♥t✐✜❝❛t✐♦♥ ✭❬✶❪✱ ❢♦r ❡①❛♠♣❧❡✮✳ ❚❤❡✐r ❣♦❛❧ ✐s t♦ ✐❞❡♥t✐❢② ♣❡rs♦♥s ✉s✐♥❣ ❜♦t❤ ✈✐s✉❛❧ ❢❡❛t✉r❡s ❛♥❞ s♣❡❡❝❤ r❡❝♦❣♥✐t✐♦♥ ✇✐t❤ t❤❡ ❛ss✉♠♣t✐♦♥ t❤❛t t❤❡ ❝✉rr❡♥t ✈♦✐❝❡ ❝♦r✲ r❡s♣♦♥❞s t♦ ❛ ✈✐s✉❛❧ ❢❡❛t✉r❡ ✐♥ t❤❡ ❢r❛♠❡ ✐s ♠❛❞❡✳ ■♥ r❡❛❧ s❡q✉❡♥❝❡s✱ t❤✐s ❤②♣♦t❤❡s✐s ✐s ♦❢t❡♥ ✈✐♦❧❛t❡❞✳ ■t ✐s ✈❡r② ❝♦♠✲ ♠♦♥ t♦ ✜♥❞ s❡q✉❡♥❝❡s ✇❤❡r❡ t❤❡ ❛♣♣❡❛r✐♥❣ ♣❡rs♦♥s ❞♦ ♥♦t s♣❡❛❦ ❞✉r✐♥❣ ♠❛♥② ❢r❛♠❡s ✭♦r ♠❛♥② s❤♦ts✮✳ ▼♦r❡♦✈❡r✱ ✐t ✐s ❛❧s♦ ✉s✉❛❧ t❤❛t t❤❡ ❝✉rr❡♥t ✈♦✐❝❡ ❜❡❧♦♥❣s t♦ ❛ ♣❡rs♦♥ ✇❤♦s❡ ✈✐s✉❛❧ ❢❡❛t✉r❡ ✐s ♥♦t ✐♥ t❤❡ ❝✉rr❡♥t ❢r❛♠❡✳ ■♥ ♦✉r ❛♣♣❧✐❝❛t✐♦♥✱ ✇❡ ♣r♦♣♦s❡ t♦ ❝♦♠♣✉t❡ ❝♦✲♦❝❝✉rr❡♥❝❡s ❜❡t✇❡❡♥ ❛✉❞✐♦ ❛♥❞ ✈✐❞❡♦ ✐♥❞❡①❡s✱ ✐✳❡✳ ✇❡ ❞❡t❡r♠✐♥❡ t❤❡ ♠❛t❝❤✐♥❣ ❜❡t✇❡❡♥ ✈♦✐❝❡s ❛♥❞ ✐♠❛❣❡s✳ ❚❤✐s ❛♣♣r♦❛❝❤ ✐s s✉✐t✲ ❛❜❧❡ t♦ t❛❦❡ ✐♥t♦ ❛❝❝♦✉♥t t❤❡ ❝❛s❡s ✇❤❡r❡ t❤❡ ✉s✉❛❧ ❛ss✉♠♣✲ t✐♦♥s ❛r❡ ♥♦t ✈❡r✐✜❡❞✳ ❚❤❡ s❝❛❧❡ ♦❢ ❛✉❞✐♦ ❛♥❞ ✈✐❞❡♦ ✐♥❞❡①❡s ❛r❡ ❞✐✛❡r❡♥t✱ ♠❛❦✐♥❣ ✐♠♣♦ss✐❜❧❡ ❛ ❞✐r❡❝t ❝♦♠♣❛r✐s♦♥ ♦❢ t❤❡ t✇♦ ✐♥❞❡①❡s✳ ❚❤❛t ✐s ✇❤② ✇❡ ♣r♦♣♦s❡ t♦ ✉s❡ ❛ ❝♦♠♠♦♥ s❝❛❧❡ ❢♦r ❛✉❞✐♦ ❛♥❞ ✈✐❞❡♦ ✐♥❞❡①❡s ✐♥ ♦r❞❡r t♦ ❜❡ ❛❜❧❡ t♦ ❞✐r❡❝t❧② ❝♦♠♣❛r❡ t❤❡♠✳ ❯s✐♥❣ t❤❡ ✈✐❞❡♦ ❢r❡q✉❡♥❝②✱ ❛ ❢r❛♠❡ ❜② ❢r❛♠❡ ❞❡❝♦♠♣♦s✐t✐♦♥ ♦❢ ❡❛❝❤ ✐♥❞❡① ✐s ♠❛❞❡✳ ▲❡t na ❜❡ t❤❡ ♥✉♠❜❡r ♦❢ ❞✐✛❡r❡♥t ✈♦✐❝❡s ✐♥ t❤❡ ❛✉❞✐♦ ✐♥❞❡①✱ ❛♥❞ nv t❤❡ ♥✉♠❜❡r ♦❢ ❞✐✛❡r❡♥t ✈✐s✉❛❧ ♣❡rs♦♥s ✐♥ t❤❡ ✈✐❞❡♦ ✐♥❞❡①✳ {Ai}i=1...na ❛♥❞ {Vj}j=1...nv ❛r❡ r❡s♣❡❝t✐✈❡❧② t❤❡ s❡t ♦❢ ✈♦✐❝❡s ❛♥❞ ✈✐s✉❛❧ ❢❡❛t✉r❡s ♦❢ ❛❧❧ ♣❡rs♦♥s✳ ❚♦ ❝♦♠♣✉t❡ t❤❡ ✐♥t❡rs❡❝t✐♦♥ ♠❛tr✐① ❜❡t✇❡❡♥ ❛✉❞✐♦ ❛♥❞ ✈✐❞❡♦ ✐♥❞❡①❡s✱ ✇❡ ❣♦ t❤r♦✉❣❤ t❤❡ t✇♦ ✐♥❞❡①❡s✱ ❢r❛♠❡ ❜② ❢r❛♠❡✳ ❋♦r ❡❛❝❤ ❢r❛♠❡✱ ✐❢ t❤❡ ✈♦✐❝❡ Ai ✐s ❤❡❛r❞ ❛♥❞ t❤❡ ✈✐s✉❛❧ ♣❡rs♦♥ Vj ✐s ♣r❡s❡♥t✱ t❤❡ ♥✉♠❜❡r ♦❢ ♦❝❝✉rr❡♥❝❡s mij ♦❢ t❤❡ ♣❛✐r (Ai, Vj) ✐s ✐♥❝r❡♠❡♥t❡❞✳ ❚❤❡♥✱ ✇❡ ♦❜t❛✐♥ t❤❡ ❢♦❧❧♦✇✐♥❣ ♠❛tr✐①✿ m = V1 V2 . . . Vnv A1 A2 ✳ ✳ ✳ Ana B B B @ m11 m12 . . . m1nv m21 m22 . . . m2nv ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ mna1 mna2 . . . mnanv 1 C C C A ✭✶✮ ■♥ t❤✐s ♠❛tr✐①✱ t❤❡ ✈❛❧✉❡ mij ♠❡❛♥s t❤❛t ✐♥ ❛❧❧ t❤❡ ❢r❛♠❡s ✇❤❡r❡ t❤❡ ✈♦✐❝❡ Ai ✐s ❤❡❛r❞✱ t❤❡ ✈✐s✉❛❧ ♣❡rs♦♥ Vj ❛♣♣❡❛rs mij t✐♠❡s✳ ❈♦♥✈❡rs❡❧②✱ ✐♥ ❛❧❧ t❤❡ ❢r❛♠❡s ✇❤❡r❡ t❤❡ ♣❡rs♦♥ Vj ✐s ♣r❡s❡♥t✱ t❤❡ ✈♦✐❝❡ Ai ✐s ❤❡❛r❞ mij t✐♠❡s✳ ❚♦ ❝❛rr② ♦✉t t❤❡ ❢✉s✐♦♥✱ ✇❡ ❝♦♠♣✉t❡ t✇♦ ♥❡✇ ♠❛tr✐❝❡s✱ ma ❛♥❞ mv✱ ✇❤❡r❡ t❤❡ ❢r❛♠❡ ♥✉♠❜❡rs ❛r❡ r❡♣❧❛❝❡❞ ✇✐t❤ ♣❡r❝❡♥t❛❣❡ ❜② r♦✇s ❛♥❞ ❜② ❝♦❧✉♠♥s✿ ma = V1 V2 . . . Vnv A1 A2 ✳ ✳ ✳ Ana B B @ f a
11
f a
12
. . . f a
1nv
f a
21
f a
22
. . . f a
2nv
. . . . . . . . . . . . f a
na1
f a
na2
. . . f a
nanv
1 C C A 100 % ✭✷✮ mv = V1 V2 . . . Vnv A1 A2 ✳ ✳ ✳ Ana B B @ f v
11
f v
12
. . . f v
1nv
f v
21
f v
22
. . . f v
2nv
. . . . . . . . . . . . f v
na1
f v
na2
. . . f v
nanv
1 C C A 100 % ✭✸✮ ▼❛tr✐① ma ❣✐✈❡s t❤❡ ♣r♦❜❛❜✐❧✐t② ❞❡♥s✐t② ♦❢ ❡❛❝❤ ✈♦✐❝❡ Ai✱ ✇❤❡r❡❛s mv ❣✐✈❡s t❤❡ ♦♥❡ ♦❢ ❡❛❝❤ ✈✐s✉❛❧ ❢❡❛t✉r❡ Vj✳ ❋r♦♠ t❤❡s❡ ♠❛tr✐❝❡s✱ ✇❡ ❞❡✜♥❡ t❤❡ ❢✉s✐♦♥ ♠❛tr✐① F✱ ❜② ❝♦♠♣✉t✲ ✐♥❣✱ ❢♦r ❡❛❝❤ ♣❛✐r ✭i, j✮✱ ❛ ❢✉s✐♦♥ ❜❡t✇❡❡♥ f a
ij ❛♥❞ f v ij ✇✐t❤ ❛
❢✉s✐♦♥ ♦♣❡r❛t♦r ❧✐❦❡ ♠❛①✐♠✉♠✱ ♠❡❛♥ ♦r ♣r♦❞✉❝t✳ ■❢ ✇❡ ♥♦t❡ C(Ai, Vj) t❤❡ ❢✉s✐♦♥ ❝♦❡✣❝✐❡♥t ❜❡t✇❡❡♥ Ai ❛♥❞ Vj✱ ❡①♣r❡s✲ s✐♦♥ ♦❢ ♠❛tr✐① F ✐s ❣✐✈❡♥ ❜②✿ F = B B @ C(A1, V1) . . . C(A1, Vnv) C(A2, V1) . . . C(A2, Vnv) . . . . . . . . . C(Ana, V1) . . . C(Ana, Vnv) 1 C C A ✭✹✮ ❚❤✐s ♠❛tr✐① F ❝❛♥ ❜❡ ❞✐r❡❝t❧② ✉s❡❞ t♦ r❡❛❧✐③❡ t❤❡ ❛ss♦❝✐❛✲ t✐♦♥✳ ❲❤❡♥ t❤❡ ♥✉♠❜❡r ♦❢ ✈♦✐❝❡s ❛♥❞ ✈✐s✉❛❧ ❢❡❛t✉r❡s ✐s t❤❡ s❛♠❡ ✭na = nv✮✱ ✐t ✐s r❡❛❞ ❡q✉❛❧❧② ❜② r♦✇s ♦r ❜② ❝♦❧✉♠♥s✳ ❋♦r ✐♥st❛♥❝❡✱ ❢♦r ❡❛❝❤ r♦✇ i✱ ✇❡ s❡❛r❝❤ t❤❡ ❝♦❧✉♠♥ j ✇❤✐❝❤ ♣r♦✈✐❞❡s t❤❡ ♠❛①✐♠✉♠ ✈❛❧✉❡✿ t❤❡♥ t❤❡ ✈♦✐❝❡ Ai ✐s ❛✉t♦♠❛t✲ ✐❝❛❧❧② ❛ss♦❝✐❛t❡❞ t♦ t❤❡ ✈✐s✉❛❧ ❢❡❛t✉r❡ Vj✳ ❋♦r ♠♦r❡ ❞❡t❛✐❧s ❛❜♦✉t t❤✐s ❛ss♦❝✐❛t✐♦♥✱ s❡❡ ❬✹❪✳
3. ENHANCING THE USE OF DETECTION TOOL WITH AN INTERFACE
❲❡ ♣r♦♣♦s❡ ❛ ❣r❛♣❤✐❝❛❧ ✉s❡r ✐♥t❡r❢❛❝❡✱ ❝❛❧❧❡❞ ❆❈❆❉■ ❛♥❞ ✐♠♣❧❡♠❡♥t❡❞ ❞✉r✐♥❣ ❛ ▼❯❙❈▲❊ ❬✶✶❪ s❤♦✇❝❛s❡✱ t♦ ♣r♦✈✐❞❡ ❛ t♦♦❧ ✐♥ ❛ ✈❡r✐✜❝❛t✐♦♥✲❛✐❞❡❞ ❢❛s❤✐♦♥ ♦❢ t❤❡ s❡❣♠❡♥t❛t✐♦♥ r❡✲ s✉❧ts✳ ❚❤❡ s❡❣♠❡♥t❛t✐♦♥ t♦♦❧ ♣r♦❞✉❝❡ ❳▼▲ ✜❧❡s ❛s r❡s✉❧t ✭s❡❡ t❛❜❧❡ ✶✮✳ ❊✈❡♥ ✐❢ ❳▼▲ ✜❧❡s ❛r❡ ❤✉♠❛♥ r❡❛❞❛❜❧❡✱ t❤❡ ✈❡r✐✜❝❛t✐♦♥ ♣r♦❝❡ss ✐s ❛ ♣❛✐♥❢✉❧ t❛s❦ ✐❢ ❞♦♥❡ ♠❛♥✉❛❧❧②✱ ❛s t❤❡s❡ ✜❧❡s ❝♦✉❧❞ ❝♦♥t❛✐♥ ❧♦t ♦❢ ❞❛t❛✳ ❆ ♣♦st✲♣r♦❝❡ss t♦ s✉♠✲ ♠❛r✐③❡ r❡s✉❧ts ✐s ♥❡❡❞❡❞ ❛♥❞ ❛❧s♦ ❛ ❝♦♥✈❡♥✐❡♥t ✈✐❞❡♦✴❛✉❞✐♦ ♣❧❛②❡r ✐♥ ♦r❞❡r t♦ ❝❤❡❝❦ t❤❡ s❡❣♠❡♥t❛t✐♦♥ ❝♦❤❡r❡♥❝②✳ ❁❄①♠❧ ✈❡rs✐♦♥❂✧✶✳✵✧❄❃ ❁❙❡❣♠❡♥t❛t✐♦♥❃ ❁❍❡❛❞❡r❃ ❁❱✐❞❡♦❃✴❧♦❝❛❧✴❈♦r♣✉s❱✐❞❡♦✴✶✵✵✺✷✵✵✺✳♠♣❣❁✴❱✐❞❡♦❃ ❁✴❍❡❛❞❡r❃ ❁❇♦❞②❃ ❁❙❊●▼❊◆❚ ■❉❂✧✶✵✸✵✧❃ ❁❈❖❙❚❯▼❊ ■❉❂✧❈❤❛r❛❝t❡r ✵✵✶✧❃ ❁❊❈❍❆◆❈❘❯❘❊❃❡❝❤❛♥❝r❡❁✴❊❈❍❆◆❈❘❯❘❊❃ ❁▼❆◆❈❍❊❙❃✐♥❝♦♥♥✉❁✴▼❆◆❈❍❊❙❃ ❁❚❊❳❚❯❘❊❃✐♥❝♦♥♥✉❁✴❚❊❳❚❯❘❊❃ ❁✴❈❖❙❚❯▼❊❃ ❁✴❙❊●▼❊◆❚❃ ❁✴❇♦❞②❃ ❁✴❙❡❣♠❡♥t❛t✐♦♥❃ ❚❛❜❧❡ ✶✿ ❆♥♥♦t❛t✐♦♥s r❡s✉❧ts ❢♦r ❛ ✈✐❞❡♦ ✭♥❛♠❡❞ ✶✵✵✺✷✵✵✺✳♠♣❣✮✱ t❤❡ ❛♥♥♦t❛t✐♦♥s ❛r❡ ❣✐✈❡♥ ❢♦r ❡❛❝❤ ❢r❛♠❡✱ ❤❡r❡ ✐s s❤♦✇♥ ♦♥❡ ❢r❛♠❡ ✭♥✉♠❜❡r❡❞ ✶✵✸✵✮✳ ❚❤❡ ❝❤❛r❛❝t❡r ❧✐st ✐s ❞✐s♣❧❛②❡❞ ✐♥ s♦rt❡❞ ❛♣♣❡❛r❛♥❝❡ ❞✉r❛✲ t✐♦♥ ♦r❞❡r ✭t❤❡ ♠❛✐♥ ❝❤❛r❛❝t❡r ✜rst✮✳❚❤❡ ✉s❡r ❝❛♥ s❡❧❡❝t ❛ ❝❤❛r❛❝t❡r ❛♥❞ s❡❡ t❤❡ ✈✐❞❡♦ ❛♥❞ ❛✉❞✐♦ s❡❣♠❡♥t❛t✐♦♥ st❛t✐s✲ t✐❝s r❡❧❛t❡❞ t♦ t❤✐s ❝❤❛r❛❝t❡r ✭❝❢✳ ✜❣✉r❡ ✶✮✳ ❚❤❡ s❡❣♠❡♥✲ t❛t✐♦♥ ✐s ❛❧s♦ ❞✐s♣❧❛②❡❞ ❛♥❞ ❛✉❞✐♦✴✈✐❞❡♦ s❡❣♠❡♥ts ❝❛♥ ❜❡ ♣❧❛②❡❞✳ ❚❤♦s❡ st❛t✐st✐❝s ♣r❡s❡♥t t❤❡ ❛♣♣❡❛r❛♥❝❡ ❞✉r❛t✐♦♥ ❛♥❞ t❤❡ s♣❡❛❦✐♥❣ ❞✉r❛t✐♦♥ ❢♦r ❛ ❝❤❛r❛❝t❡r ❞✉r✐♥❣ t❤❡ s❡✲ q✉❡♥❝❡✳ ❚♦ ♣r♦✈✐❞❡ ♣♦rt❛❜✐❧✐t② ❛♠♦♥❣ ❞✐✛❡r❡♥t s②st❡♠s ✇❡ ✉s❡❞ t❤❡ ●t❦♠♠ ❧✐❜r❛r② ❢♦r t❤❡ ●❯■✱ ❛♥❞ ❢r❛♠❡ ❛❝❝❡ss ✐s ♠❛♥❛❣❡❞ ❜② ❛♥ ❡♠❜❡❞❞❡❞ ✛♠♣❡❣ ♣❧❛②❡r✳ ❚❤✐s ♦♥❡ ❛❧s♦ ♣r♦✲ ✈✐❞❡s r❛♥❞♦♠ ❢r❛♠❡ ❛❝❝❡ss✳ ❋♦r ♥♦✇ ✇❡ ♦♥❧② s✉♣♣♦rt ♠♣❡❣ P❙ ✜❧❡ ❢♦r♠❛t✳
SLIDE 4