Multimodal Interaction Multimodal Interaction and TV Wei Yun Yau - - PowerPoint PPT Presentation
Multimodal Interaction Multimodal Interaction and TV Wei Yun Yau Programme Manager, Personalised Interactive Tele Services Institute for Infocomm Research, A*STAR Institute for Infocomm Research, A STAR Sheau Ng Vi Vice President &
Programme Manager, Personalised Interactive Tele‐Services Institute for Infocomm Research, A*STAR Institute for Infocomm Research, A STAR
Vi P id t & H d f R&D Vice President & Head of R&D, NBC Universal
W3C Workshop on Rich Multimedia App Development
Institute for Infocomm Research (I2R)
2
A f S i T h l & Agency for Science, Technology & Research (A*STAR) is under the Ministry of Trade & Industry, Singapore.
Fostering world class
g p
Powering a vibrant and strong infocomm ecosystem for Fostering world-class infocomm and media research and developing a deep talent pool of i f f i l y Singapore. infocomm professionals to power a vibrant knowledge- based Singapore.
Singapore’s Largest ICT Research Institute g p
Staff today
Industry Collaborations 1220 Technology Disclosures 5430 Research Publications
Institute for Infocomm Research (I2R)
3
Institute for Infocomm Research (I2R)
3
INTERNET
T t i l Terrestrial Satellite Cable DVD Player / Blue-Ray Player
Institute for Infocomm Research (I2R)
4
Youtube News Local News station News
AlJazeera, AFP, Euronews
station (CNA)
Social Media
Institute for Infocomm Research (I2R)
5
– Where are the relevant multimedia content? – How relevance is determined?
What the user likes? – What the user likes? – How to track content consumed from different sources on different devices?
– What the user would be interested now? – What are the external events happening that the user could be interested in?
– What to show, how to show, and when? – Easy interaction & navigation to create better engagement?
Institute for Infocomm Research (I2R)
6
Institute for Infocomm Research (I2R)
6
External sources Story segmentation Named entities External sources Keyword Search / Timeline Speech‐to‐text segmentation Metadata enrichment Timeline News Video Tapestry Video‐OCR
Keywords & Summarization
Breaking News Alert Scene detection
Link analytics
Linked data
Low-level Analytics
detection
High-level Analytics
Link analytics Local news channels
Applications
Linked data
y
channels Story-K Story-K+1 Story-K+2
Institute for Infocomm Research (I2R)
7
Institute for Infocomm Research (I2R)
7
Viewing Device
Location Location
Structured Semi‐structured / Unstructured
Institute for Infocomm Research (I2R)
8
Institute for Infocomm Research (I2R)
8
Questionnaire External Events Remote Usage Logs Remote Usage Logs
Feedback to Personalized EPG
Institute for Infocomm Research (I2R)
9
Institute for Infocomm Research (I2R)
9
Institute for Infocomm Research (I2R)
10