META-FORUM 2016 04-July-2016 LiMe Motivation Knowledge in the EU - - PowerPoint PPT Presentation

▶

Dec 26, 2022 411 likes •530 views

Ronald Denaux rdenaux@expertsystem.com META-FORUM 2016 04-July-2016 LiMe Motivation Knowledge in the EU is fragmented So far, information can only be TV, social analysed videos video, visual photos, social independently for each

SLIDE 1

Ronald Denaux rdenaux@expertsystem.com

META-FORUM 2016

04-July-2016

SLIDE 2

LiMe

Motivation

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 2 mainstream/ professionally produced social/ user generated

social video, social photos audio from TV audio from social media tweets, blogs, comments, reviews TV, videos photos, images news, annotations

audio/video

modalities channels

visual auditiv textual

Knowledge in the EU is fragmented… So far, information can only be analysed independently for each dimension. This restricts the extractable knowledge and keeps it fragmented.

SLIDE 3

LiMe

Approach

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 3

Multi- lingual Annotati

visual auditiv textual

mainstream/ commercial/ professional social/ user generated

Social Video, Social Photos

Audio from TV

Podcasts, Audio from Social Video Twitter, Blogs, Comment

Natural Languag e

Cross- lingual text annotations

Multi- lingual text anno

Multi-lingual text Multi-lingual text annotations TV, Video Clips, Photos, Images News, Descriptions

f video or

audio

SLIDE 4

LiMe

Approach

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 4

SLIDE 5

LiMe

Roles of Partners

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 5

Iberia

Research & Development of general functionality Integration Data & Use Cases & Product Specific Development

SLIDE 6

LiMe

Approach

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 6

SLIDE 7

LiMe

Use Case: Zattoo „mini embed“

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 7

For a given online article, find related TV programs from what’s currently airing

SLIDE 8

LiMe

Use Case: Econda product recommendation

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 8

For a given product catalog, recommend products based

n recent mentions in

supported media.

Products/categories/brands to DBpedia
Identify mapped entities in messages

Mappings of domain entities

Include domain entities in annotation knowledge base
Use names and descriptions as surface forms

Text/ASR annotations using domain entities

Visual index of all products in shop
Detect similar products in video frames

Video annotations using domain entities

SLIDE 9

LiMe

Use Case: VICO Brand Monitoring

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 9

SLIDE 10

LiMe

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 10

Start: November 2013; End: October 2016
Project Coordinator: Achim Rettinger rettinger@kit.edu
Project Manager: valentina.pavlova@kit.edu
Web: http://xLiMe.eu

SLIDE 11

LiMe

Y1 Benchmarks

04-July-16 Ronald Denaux (Expert System Iberia) – META-FORUM-16 11

Entity Linking for Social Media: 220% CPU, 27% main memory, ~150,000 microposts per day, 12

languages supported, throughput ~2.1kb/s

Speech to Text: single 40s audio chunk processed in 2min 20s, throughput ~ 140kb/s
Text from Video: 6 frames per second, 24GB memory, throughput ~ 384 kb/s, accuracy 30%
Visual Object Type Recognition: 6GB GPU, NVIDIA Tesla K40c GPU, throughput ~ 2000 kb/s,

accuracy 99.5%

Named Entity from Text: tokenization (48% one core CPU, 3.7GB memory), KIT wikifier (10-40

cores used of 64), throughput ~ 112kb/s

Syntactic annotations for News Articles: <10% CPU, 5.5GB, throughput ~ 112kb/s
Cross-media Recommendations: social media (1.01 million posts), TV programs (350), news

articles (40k), throughput ~ 232kb/s

232 112 112 140 384 2000 2,1 1 10 100 1000 10000 Throughput (kb/s)* Named Entity Microposts Video Annotation Text from Video Speech to Text Named Entity Newsfeed Sentiment Annotations Cross-media Recommendations