Multimodal Interaction Multimodal Interaction and TV Wei Yun Yau - PowerPoint PPT Presentation

Multimodal Interaction Multimodal Interaction and TV Wei ‐ Yun Yau Programme Manager, Personalised Interactive Tele ‐ Services Institute for Infocomm Research, A*STAR Institute for Infocomm Research, A STAR Sheau Ng Vi Vice President & Head of R&D, P id t & H d f R&D NBC Universal W3C Workshop on Rich Multimedia App Development

Singapore 2 Institute for Infocomm Research (I 2 R)

Who We Are Agency for Science, Technology & A f S i T h l & Research (A*STAR) is under the Ministry of Trade & Industry, Singapore. g p Mission Vision Fostering world class Fostering world-class infocomm and media Powering a vibrant and research and developing a strong infocomm deep talent pool of ecosystem for y i f infocomm professionals to f i l Singapore. power a vibrant knowledge- based Singapore. Singapore’s Largest ICT Research Institute g g p Staff Industry Disclosures 5430 Research Technology 600 681 Collaborations 1220 today Publications 3 3 Institute for Infocomm Research (I 2 R) Institute for Infocomm Research (I 2 R)

INTERNET T Terrestrial t i l Satellite DVD Player / Cable Blue-Ray Player 4 Institute for Infocomm Research (I 2 R)

Enjoying Related Content Seamlessly Youtube Local News News News station station AlJazeera, (CNA) AFP, Euronews Social Media 5 Institute for Infocomm Research (I 2 R)

Challenges • Automated Content Discovery – Where are the relevant multimedia content? – How relevance is determined? • User Preference Modeling – What the user likes? What the user likes? – How to track content consumed from different sources on different devices? • Personalised Recommendation – What the user would be interested now? – What are the external events happening that the user could be interested in? • Attractive Presentation Attractive Presentation – What to show, how to show, and when? – Easy interaction & navigation to create better engagement? 6 6 Institute for Infocomm Research (I 2 R) Institute for Infocomm Research (I 2 R)

News Video Analytics External sources External sources Keyword Story Search / Named entities segmentation segmentation Timeline Timeline Metadata News Video Speech ‐ to ‐ text enrichment Tapestry Video ‐ OCR Keywords & Breaking Summarization News Alert … … … … Scene Link analytics Link analytics detection detection Linked data Linked data Low-level High-level Local news Applications Analytics Analytics y channels channels Story-K Story-K+1 Story-K+2 7 7 Institute for Infocomm Research (I 2 R) Institute for Infocomm Research (I 2 R)

Multiple Modality Social Interactions Demographics Demographics Viewing Device Location Location Mapping Contents to Viewers Semi ‐ structured / Unstructured Structured Web Content Web Resources TV Content EPG ‐ Metadata 8 8 Institute for Infocomm Research (I 2 R) Institute for Infocomm Research (I 2 R)

Personalised Recommendation Questionnaire External Events Remote Usage Logs Remote Usage Logs Social Groups Feedback to Personalized EPG Server: Contextual Input On Device: Daily Usage Input 9 9 Institute for Infocomm Research (I 2 R) Institute for Infocomm Research (I 2 R)

Thank you Contact: wyyau@i2r a star edu sg Contact: wyyau@i2r.a-star.edu.sg For additional information, please visit our website, htt http://www.i2r.a-star.edu.sg/ // i2 t d / 10 Institute for Infocomm Research (I 2 R)