Presenter Name Job title, company What is ReCAP? ReCAP = Re altime - PowerPoint PPT Presentation

Presenter Name Job title, company

What is ReCAP? ReCAP = Re altime C ontent A nalysis & P rocessing A project co-funded by a €1m grant from the European Union’s Horizon 2020 programme for research and innovation • 18 month Machine Learning & Computer Vision project • Started in December 2016 • To be complete by May 2018 To commercialise a scalable and flexible platform and range of real time content analysis services to automatically extract metadata from live and archive media content • Capable of processing broadcast quality video/audio content • Supports live streams from IP and SDI sources • Supports batch processing of existing media content • Flexible deployment options

Aims Adapt and integrate a set of existing visual descriptive & technical analysis algorithms developed in the EU by the consortium partners Develop APIs for integration with Media Asset Management (MAM), workflow automation, transcoding, Auto-QC software and 3rd party Machine Learning algorithms Identify use cases • Enable automatic archive content enrichment • Help with broadcast, legal and financial compliance challenges • Improve content rights management challenges Project outcome Make automatic metadata extraction affordable for SMEs

Consortium Partners • New Media Research (NMR) , a UK based media technology solutions provider, specialising in Media Asset Management (MAM), workflow automation and custom software development • ToolsOnAir (TOA) , an Austrian developer of applications that integrate broadcast quality ingest, scheduling and playout automation with real-time graphics, workflow automation and shared storage • Joanneum Research Institute (JRS) , an Austrian R&D company, focused on Artificial Intelligence (AI) and Machine Learning (ML) technologies • Nablet (NAB) , a German developer of streaming, codec, muxing, analysis and transcoding solutions

Descriptive Algorithms Face Recognition Automatic Speech Recognition (ASR) Logo Detection On-Screen Text Extraction (OCR) Duplication Similarity Fingerprinting Music Identification* Language Identification* *Future Release

Technical Algorithms Video Breakup Macroblocking Scene Changes Block Dropout Colour Bars Black Tone Silence Artifact Suppression* Interpolation* Denoiser* Deinterlacer* *Potential Automatic Quality Improvement

Technology • Software runs on Linux CentOS 7 • Optimised to utilise GStreamer open-source software - a multi-platform, pipeline-based multimedia framework • Real time processing engine using the GPU • Metadata output is stored as JSON • FIMS -compliant metadata structure • MPEG7 multimedia content description standard • RESTful APIs for 3rd party integrations

Prototype Web Player - NAB 2017

Potential Use Cases Archive Content Enrichment Automatically adding descriptive and technical metadata to legacy archive content, providing time stamped metadata search, to repurpose existing content. Avoid the need to reshoot content, for historical analysis/reporting or to provide opportunities to license and commercialise legacy content for news, documentaries, re-broadcast and VOD Compliance Improve compliance related video workflows, typically; • Broadcast - identify and remove inappropriate content prior to transmission • Financial - identify who said what/when for investigative purposes • Legal - qualify, preserve or expunge sensitive content from the legal record To assist production teams and compliance officers, to reduce time and effort to find and correct/remove inappropriate content

Other Use Cases Rights Management Significant challenges exist for owners and consumers of rights-managed materials. Although ReCAP does not aim to provide rights management functionality directly it can be used to identify and fingerprint content and thus help reduce the risk of using content with expired rights and thus reduce the risk of rights violations and potentially damaging legal action. Anti-Piracy Identify illegal streams of live events, in real time, to enable anti-piracy measures to be taken. Targeted Ad Insertion For live streaming/VOD/OTT based on content identification/similarity/competing brands And Many More...

Commercial Model TBC

Next Steps Alpha preview Exhibiting on Vidispine (3.A23) and ToolsOnAir (7.G45) stands demonstrating end-to-end Alpha preview - ingest file analyse content review result Beta preview Feature locked - under select private Beta testing with stakeholder board Project Conclusion May 2018 v1.0 available for early adopters - Q3 2018 Continued development