VK Multimedia Information Systems Mathias Lux, - PowerPoint PPT Presentation

VK Multimedia Information Systems Mathias Lux, mlux@itec.uni-klu.ac.at Dienstags, 16.oo Uhr c.t., E.2.69 This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 2.0 License. See http://creativecommons.org/licenses/by-nc-sa/2.0/at/ Department for Information Technology, Klagenfurt University, Austria

Präsentationen 03.06. http:// www.uni-klu.ac.at ● Video Summary - Pörtsch ● Flickr Related Tags – Pitman ● MP3 LSA – Waltl & Jonke ● YouTube Usage – Bartha 2 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Präsentationen 24.06. http:// www.uni-klu.ac.at ● LSH - Kofler & Pasterk ● Image Features – Finke & Unterzaucher ● Seam Carving – Wanschou ● Retrieval von Intentionen – Doliner & Irrasch 3 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Contents http:// www.uni-klu.ac.at ● MPEG-7 ● MPEG-21 ● Metadata Generation & Annotation ● Social Software & Metadata 4 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 http:// www.uni-klu.ac.at ● ISO/IEC Standard: Multimedia Content Description Interface ● Moving Pictures Expert Group  Specification goes on ... ● It’s based on XML (Schema)  Binary representations possible (BiM) ● Allows differing granularity of descriptions  Extensive to very simple 5 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 History http:// www.uni-klu.ac.at ● Call for Proposals: October 1998 ● Evaluation: February 1999 ● First version of Working Draft (WD): December 1999 ● Committee Draft (CD): October 2000 ● Final Committee Draft (FCD): February 2001 ● Final Draft International Standard (FDIS): July 2001 ● International Standard (IS): September 2001 6 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 Basics http:// www.uni-klu.ac.at ● Descriptors  Syntax and semantics of exactly one (low or high level) elementary feature  Also base data types are defined ● Description Schemes Defines structures within a  framework ● Description Definition Language (DDL)  Extension of XML Schemes ● Coding Schemes  Create and interpret descriptions in BiM 7 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 Parts http:// www.uni-klu.ac.at 1. MPEG-7 Systems ● Tools needed to prepare MPEG-7 descriptions for efficient transport and storage and the terminal architecture. 2. MPEG-7 Description Definition Language ● Language for defining the syntax of the MPEG-7 Description Tools and for defining new Description Schemes. 3. MPEG-7 Visual ● Description Tools dealing with (only) visual descriptions. 4. MPEG-7 Audio ● Description Tools dealing with (only) audio descriptions. 5. MPEG-7 Multimedia Description Schemes ● Description Tools dealing with generic features and multimedia descriptions. 8 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 Parts http:// www.uni-klu.ac.at 6. MPEG-7 Reference Software Implementation of relevant parts of the MPEG-7 Standard with ● normative status. 7. MPEG-7 Conformance Testing ● Guidelines and procedures for testing conformance of MPEG-7 implementations 8. MPEG-7 Extraction and Use of Descriptions ● Informative material about the extraction and use of some of the Description Tools. 9. MPEG-7 Profiles and levels Provides guidelines and standard profiles. ● 10. MPEG-7 Schema Definition Specifies the schema using the Description Definition ● Language 9 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Scope of MPEG-7 http:// www.uni-klu.ac.at from: http://www.chiariglione.org/mpeg/standards/mpeg-7/mpeg-7.htm 10 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Basic Elements http:// www.uni-klu.ac.at Basic elements are fundamental constructs and used throughout the whole MPEG-7 description ● Basic datatypes  Time and date, relative and absolute  Numeric datatypes like matrices and vectors ● Links & Media Localization  Interconnections and content linking 11 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Navigation & Access http:// www.uni-klu.ac.at ● Descriptors for Browsing & Retrieval  Summaries  Partitions (time, space & frequency)  Decompositions (time, space & frequency)  Variations 12 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

User Interaction http:// www.uni-klu.ac.at ● Pertaining consumption of AV data  user preferences  usage history ● Meant to facilitate personalization  Matching User Interaction DS with content description  Is research topic @ ITEC 13 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Content Organization http:// www.uni-klu.ac.at ● Organization & modelling of collections  Audio-visual content, segments, events, and/or objects • E.g. pictures, scenes, music files, etc.  Allows collection description as a whole • E.g. “Pictures of my holiday in Ebonia” 14 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Content Management http:// www.uni-klu.ac.at ● Creation & Classification  Title, textual annotation, creators, creation locations, and dates.  Categories such as genre, subject, purpose or language.  Review and guidance information: Age classification, parental guidance, and subjective review.  Related material information. ● Media coding, storage & file formats  Media profiles & master media ● Content Usage  Usage rights, usage record, and financial information 15 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Content Description: Structural vs. Conceptual Aspects http:// www.uni-klu.ac.at ● Program DS (in sense of TV program) ● Analogy to  Table of content – Region tree (linear partitioning)  Index – Object tree (non-linear structure) Index Table of Contents Topic 1 ............. Chapter 1 ............. Item 1.…… Section 1.1.... Item 2.…… Section 1.2.... Topic 2 ............. Chapter 2 ............. Item 1.……. Section 2.1.... Item 2.……. Section 2.2.... 16 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Content Description: Structural Aspects http:// www.uni-klu.ac.at ● Divide a video stream into physical and logical video segments ● The higher the level of a physical video unit, the more semantic information is necessary ● Logical units are based on semantic content 17 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Region and Object Trees http:// www.uni-klu.ac.at 18 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Content Description: Semantic Aspects http:// www.uni-klu.ac.at ● Low Level Features  Extraction from Content  Descriptors for • Shape, color, texture (visual) • Timbre, rhythm (audio) ● High Level Features  Annotation  So called semantic descriptors • Textual information • Conceptual information 19 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 High Level Descriptors http:// www.uni-klu.ac.at ● Textual Descriptions  Text to describe temporal / spatial regions ● The W’s  Structured way of textual descriptions • Who, Where, What Object, When, Why, How & Where ● Instead of textual descriptions  Controlled Terms • Dictionaries, Taxonomies, Classifications Schemes  Semantic Description Scheme 20 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-7 Semantic Description Scheme http:// www.uni-klu.ac.at 21 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Actual Description in MPEG-7 http:// www.uni-klu.ac.at 22 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

Contents http:// www.uni-klu.ac.at ● MPEG-7 ● MPEG-21 ● Metadata Generation & Annotation ● Social Software & Metadata 23 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-21 – motivation and scope http:// www.uni-klu.ac.at 24 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-21 Objectives http:// www.uni-klu.ac.at MPEG- 21’s goal is to create an interoperable and integrated multimedia framework in three steps: 1. Develop “big picture” : understand how the components of the framework are related and identify where gaps in the framework exist 2. Fill the gaps : develop new standard specifications where needed 3. Integrate : achieve the integration of standards to support harmonized technologies for the management of multimedia content 25 ITEC, Klagenfurt University, Austria – Multimedia Information Systems

MPEG-21 Digital Item http:// www.uni-klu.ac.at A Digital Item (DI) is a structured digital object with a standard representation, identification, and metadata within the MPEG-21 framework ● Digital Items are “the content” ● DIs consist of  Resources (individual assets, distributed content),  Metadata (data about or pertaining the DI) and  Structure (relationships between parts of the DI) 26 ITEC, Klagenfurt University, Austria – Multimedia Information Systems