Semantic Audio Production Tools for Radio Chris Baume BBC Research and Development chris.baume@bbc.co.uk Supervisors: Prof. Mark Plumbley, Dr. Janko Ćalić 1/3/17 1
Structure M o t i v a t i o n ● Background ● Research questions ● Ethnographic study ● News – Drama – – Documentary Findings ● Future work ● 2
Motivation 3
Motivation 4
Motivation 5
Motivation Wolfgang Kӧhler, “Gestalt Psychology”, Liveright, 1929 6
Background L o w - l e v e l ( m a p p i n g ) ● Pseudocolour – False colour – High-level (labelling) ● Speech/music discrimination – – Speaker diarization/identification Speech-to-text – 7
Background Low-level: Pseudocolour Stephen V. Rice, “Frequency-Based Coloring of the Waveform Display to Facilitate Audio Editing and Retrieval”, Proc. Audio Engineering Society Convention, 2005 8
Background Low-level: False colour Michael Towsey, Liang Zhang, Mark Cottman-Fields, Jason Wimmer, Jinglan Zhang and Paul Roe, “Visualization of Long-duration Acoustic Recordings of the Environment”, Proc. International Conference on Computational Science, 2014 9
Background High-level: Speech/music discrimination Xia Cui, Matt Haynes, Jun Wang, “Evaluating Speech Music Discriminators”, Masters Thesis, University College London, 2014 10
Background High-level: Speaker diarization/identification http://worldservice.prototyping.bbc.co.uk 11
Background High-level: Speech-to-text Source: Sergio Grau Puerto, University of Oxford 12
Research questions What tasks are radio producers trying to achieve when interacting with audio recordings? What information can be extracted which would assist in this task? How can that information be presented in the most effective way? 13
Ethnographic study O b j e c t i v e s Discover how radio programmes are created ● Identify opportunities to improve the process using technology ● Scope End-to-end production process ● Varied, representative selection of programmes ● 14
Ethnographic study Data collection Passive observation ● Unstructured interviews ● Analysis Roles ● Environment ● Workflow ● Challenges and opportunities ● 15
Ethnographic study News 16
Ethnographic study News 17
Ethnographic study News 18
Ethnographic study News 19
Ethnographic study News 20
Ethnographic study News Challenges Finding clips in long recordings ● Adding useful metadata (title, in/out words) ● 21
Ethnographic study Drama 22
Ethnographic study Drama 23
Ethnographic study Drama 24
Ethnographic study Drama 25
Ethnographic study Drama Challenges Assembling rough edit from the script ● Comparing takes ● Undocumented re-takes ● Use of paper ● 26
Ethnographic study Documentary Image credit: James Beard 27
Ethnographic study Documentary Other enquiries – they were the priority? Certainly the Fusilier Rigby was a priority….and whole issue of privacy is important…you have to do things in sequence…but so much documentation that all that work had to be done anyway… As a committee? ….looking at budget…etc…. not possible to do two major enquiries in tandem. If we did try to do it I think we wouldn’t do justice to do either of them. Have you looked at any of the documents or only staff? Several of us have seen some of the key documents …and I’m quite familiar with the issues. But there will be new material….so I haven’t seen every key document. What do you hope to discover? (2’46) I genuinely don’t have a conclusion in mind… …has to be open-minded… but you harbour the hope that the government and agencies haven’t done anything illegal. When you saw the Senate committee report – anything you want to know? From UK point of view, nothing that people didn’t know anyway. Waterboarding etc. My concern is those issues… those areas that affect the UK that were redacted, we need to know what was in those…why requested that they redacted those areas …that’s our most important task and we’ve already asked the agencies what those redactions were (4’50). 28
Ethnographic study Documentary (1.45) When David Cameron in 2010 announces there’s going to be this judge led inquiry into what exactly our intelligence services were doing when other people were being tortured, when he went for that judge led inquiry what did you think of that? Well that was a good thing. I did say - I can’t remember how publicly - but I certainly said as it were on a more private level that we should be cognisant of who the judge should be because you know judges aren’t stamped out of a template, they’re not all the same – as anybody who deals with the courts knows – and they … and so we didn’t want to pick a judge who was already as it were a trustee to the system. In fact they ended up picking Gibson, who was a past commissioner, and therefore was seen slightly to be part of the system, and that … of course from that flowed the problems that came later. (2.33) Those problems … I mean at the time when David Cameron was asked in the Commons what about the Intelligence and Security Committee, shouldn’t they be looking at it and he said that they weren’t the right body - four years on, they are going to be the body who’s looking at this. I mean do you think they’re the right people to look at it? Well let’s start with where he was then. At that point when he said that, the committee had … broadly speaking had to rewrite its own report on the 7/7 thing, had completely missed rendition – the original part of this … this story, knew nothing of and had never brought into the public domain any of the sort of mass data collection stuff. There were … In other words there were a series of problems. And of course they didn’t spot the dodgy dossier either – the Iraq War problem. So there were a whole series of things they just simply had got wrong, you know? And there are many reasons why that is the case, but that was the case then. Since then, I guess if you were asking David Cameron this question what he would say is we have reformed the ISC. Now what that means is they’ve basically given them more powers. Whether that has … whether that’s done enough to make them the appropriate body, that’s a different question, and in my view no they haven’t. 29
Ethnographic study Documentary Challenges Transcription ● Use of paper ● Audio clean-up ● 30
Ethnographic study Findings Text-based working ● Use of paper ● Redundant speech ● Speaker diarization ● Comparing takes ● 31
Future work Next steps Use findings to develop a prototype audio production system ● Evaluate the system in a real production environment ● Refine and repeat ● 32
Future work Demo 33
Recommend
More recommend
