Project Presentation June 2008 Presentation Outline 1. - - PowerPoint PPT Presentation

Computer-Aided Semantic Annotation of Multimedia

Project Presentation

June 2008

June 2008

CASAM Project Presentation

2

Presentation Outline

• 1. Introduction
• 2. Consortium
• 3. Motivation
• 4. Project Main Target
• 5. Project Objectives
• 6. Approach
• 7. Partner Roles
• 8. Research Areas
• 9. Project Expected Results
• 10. Potential Impact
• 11. Contact Information

June 2008

CASAM Project Presentation

3

Introduction

Project Acronym / Number: CASAM / FP7 – 217061 Project Title: Computer-Aided Semantic Annotation of Multimedia Project Website: www.casam-project.eu Project Logo: Total Cost / EC Funding: 4.24 m€ / 3.03 m€ Duration: 1 April 2008 – 31 March 2011

June 2008

CASAM Project Presentation

4

Consortium

2

National Centre of Scientific Research “Demokritos” (NCSR)

3

University of Birmingham (UNI-BHAM)

4

Technische Universität Hamburg (TUHH)

5

Athens Technology Center SA (ATC)

6

Deutsche Welle (DW)

7

Agência de Notícias de Portugal (LUSA)

8

European Journalism Centre (EJC)

1

INTRASOFT International SA (INTRASOFT) – Project Coordinator

June 2008

CASAM Project Presentation

5

Motivation

Pressing need for annotated multimedia content

Immature machine-only (fully automated) annotation Cost-prohibitive manual (human) annotation

Computer-Aided Semantic Annotation of Multimedia

User effort optimization through human-machine synergy Human and machine intelligence combination

Annotation of multimedia by employing human-machine synergy that optimizes user effort.

Multimedia Content Annotation

Machine Annotation

(fully automated)

Manual Annotation CASAM

June 2008

CASAM Project Presentation

6

Project Main Target

To design, implement and validate an Annotation Toolkit that will augment machine knowledge with human input, towards the target of minimizing user effort and bridging the gap between machine-derived and human annotation 3-phases Platform Validation in terms of easiness and quality of annotation:

Initial System Architecture First Prototype Second – Final Prototype

June 2008

CASAM Project Presentation

7

Project Objectives

To provide the technology for Multimedia Analysis

Knowledge-driven analysis Reassessment of context and concepts based on user feedback and new knowledge

To develop novel methods of Reasoning for Multimedia Interpretation

Built around information exchange between multimedia analysis and human

To design unique intelligent Human-Computer Interaction methods

Maximization of the expected information gain from the user’s input Cooperative environment to guide the knowledge aggregation process

June 2008

CASAM Project Presentation

8

Approach (I)

Introduction of the effort-optimized knowledge aggregation concept Annotation toolkit featuring a human-machine interaction loop for fast convergence of human and machine knowledge Annotation task into 3 main operations:

Reasoning for Multimedia Interpretation (RMI) Knowledge-Driven Multimedia Analysis (KDMA) Human-Computer Interaction (HCI)

Demonstration Domain: News Production in News Agencies and Broadcasters

Methods also applicable to other domains

June 2008

CASAM Project Presentation

9

Approach (II)

June 2008

CASAM Project Presentation

10

Approach (III)

CASAM information flow (interaction loop)

1. KDMA analyzes the multimedia content (video, image, natural language) and extracts the low level information. 2. User enters a small number of keywords that describe the high-level concepts present in the multimedia content. 3. RMI augments the KDMA-derived and user-provided information instantiating an ontology, infers new concept instances and reassesses the context and previous input from KDMA. Results are fed back to the KDMA for multimedia analysis driven by the renewed information until nothing more can be inferred by RMI or recognized by KDMA. 4. RMI reasons about what is needed in order to add missing instances or resolve any ambiguities. If the annotation target has been achieved the loop exits, otherwise the information requirements are fed to HCI. 5. HCI transforms the information requirements to input requests towards the user, optimizing the user interaction. The user interface also provides the user with the opportunity to alter the knowledge acquisition path devised by the system. 6. HCI input is passed to the RMI and the loop continues from step 3.

June 2008

CASAM Project Presentation

11

Partner Roles

End Users

RMI HCI KDMA

User Requirements & Knowledge Representation Platform Evaluation Platform Integration

ATC UNI-BHAM NCSR DW, LUSA EJC INTRASOFT TUHH Project Management (INTRASOFT) Dissemination & Exploitation (ATC)

June 2008

CASAM Project Presentation

12

Research Areas (I)

Knowledge-Driven Multimedia Analysis

Text Processing Image Processing Video Processing and Video-OCR Audio Processing Ontology-based Information Extraction

June 2008

CASAM Project Presentation

13

Research Areas (II)

Knowledge Representation and Reasoning for Multimedia Interpretation

Probabilistic Reasoning Machine Learning Multimedia Interpretation Generation of Requests for Information

June 2008

CASAM Project Presentation

14

Research Areas (III)

Interactive Interface Design, Personalization and Optimization

Effective User Interfaces for Multimedia Tagging Adaptive Interfaces Domain Models User Models

June 2008

CASAM Project Presentation

15

Project Expected Results

Unified representation of related domain knowledge Methodology for knowledge-driven multimedia analysis Methodology for reasoning for multimedia interpretation Methodology for intelligent human-computer interface The final toolkit for computer-aided semantic annotation of multimedia content

June 2008

CASAM Project Presentation

16

Potential Impact (I)

Commercial Benefits

Creators to be able to design more participative and communicative forms of content Publishers in creative industries, enterprises and professional sectors to increase their productivity with innovative content

• f greater complexity and ease of repurposing

Organizations to be able to automate the collection and distribution of digital content and machine traceable knowledge and share them with partner organizations

June 2008

CASAM Project Presentation

17

Potential Impact (II)

Scientific Impact Deployment of:

Self-adaptive, self-aware, non-isolated and knowledge- guided Multimedia Analysis methods Intelligent Reasoning methods, guided by multiple targets Human-Computer Interaction methods, maintaining the user model and customize the queries

June 2008

CASAM Project Presentation

18

Contact Information

Coordinator contact details:

• Dr. Antonis Ramfos

INTRASOFT International S.A.

E-Mail: Antonis.Ramfos@intrasoft-intl.com Tel: +30 210 68 76 482 Fax: +30 210 68 59 166

Dissemination responsible:

• Dr. George Kountourakis

Athens Technology Center S.A.

E-Mail: g.kountourakis@atc.gr Tel: +30 210 68 74 300 Fax: +30 210 68 55 564

Project Website:

www.casam-project.eu

June 2008

CASAM Project Presentation

19