Semantic Audio Production Tools for Radio Chris Baume BBC Research - - PowerPoint PPT Presentation

1

Semantic Audio Production Tools for Radio

1/3/17

Chris Baume BBC Research and Development chris.baume@bbc.co.uk

Supervisors:

• Prof. Mark Plumbley, Dr. Janko Ćalić

2

Structure

• Motivation
• Background
• Research questions
• Ethnographic study

–

News

–

Drama

–

Documentary

• Findings
• Future work

3

Motivation

4

Motivation

5

Motivation

6

Motivation

Wolfgang Kӧhler, “Gestalt Psychology”, Liveright, 1929

7

Background

• Low-level (mapping)

–

Pseudocolour

–

False colour

• High-level (labelling)

–

Speech/music discrimination

–

Speaker diarization/identification

–

Speech-to-text

8

Background

Low-level: Pseudocolour

Stephen V. Rice, “Frequency-Based Coloring of the Waveform Display to Facilitate Audio Editing and Retrieval”, Proc. Audio Engineering Society Convention, 2005

9

Michael Towsey, Liang Zhang, Mark Cottman-Fields, Jason Wimmer, Jinglan Zhang and Paul Roe, “Visualization of Long-duration Acoustic Recordings of the Environment”, Proc. International Conference on Computational Science, 2014

Background

Low-level: False colour

10

Background

High-level: Speech/music discrimination

Xia Cui, Matt Haynes, Jun Wang, “Evaluating Speech Music Discriminators”, Masters Thesis, University College London, 2014

11

Background

High-level: Speaker diarization/identification

http://worldservice.prototyping.bbc.co.uk

12

Source: Sergio Grau Puerto, University of Oxford

Background

High-level: Speech-to-text

13

Research questions What tasks are radio producers trying to achieve when interacting with audio recordings? What information can be extracted which would assist in this task? How can that information be presented in the most effective way?

14

Ethnographic study

Objectives

• Discover how radio programmes are created
• Identify opportunities to improve the process using technology

Scope

• End-to-end production process
• Varied, representative selection of programmes

15

Data collection

• Passive observation
• Unstructured interviews

Analysis

• Roles
• Environment
• Workflow
• Challenges and opportunities

Ethnographic study

16

Ethnographic study

News

17

Ethnographic study

News

18

Ethnographic study

News

19

Ethnographic study

News

20

Ethnographic study

News

21

Ethnographic study

News

Challenges

• Finding clips in long recordings
• Adding useful metadata (title, in/out words)

22

Ethnographic study

Drama

23

Ethnographic study

Drama

24

Ethnographic study

Drama

25

Ethnographic study

Drama

26

Ethnographic study

Drama

Challenges

• Assembling rough edit from the script
• Comparing takes
• Undocumented re-takes
• Use of paper

27

Ethnographic study

Documentary

Image credit: James Beard

28

Ethnographic study

Documentary

Other enquiries – they were the priority? Certainly the Fusilier Rigby was a priority….and whole issue of privacy is important…you have to do things in sequence…but so much documentation that all that work had to be done anyway… As a committee? ….looking at budget…etc…. not possible to do two major enquiries in tandem. If we did try to do it I think we wouldn’t do justice to do either of them. Have you looked at any of the documents or only staff? Several of us have seen some of the key documents …and I’m quite familiar with the issues. But there will be new material….so I haven’t seen every key document. What do you hope to discover? (2’46) I genuinely don’t have a conclusion in mind… …has to be open-minded… but you harbour the hope that the government and agencies haven’t done anything illegal. When you saw the Senate committee report – anything you want to know? From UK point of view, nothing that people didn’t know anyway. Waterboarding etc. My concern is those issues… those areas that affect the UK that were redacted, we need to know what was in those…why requested that they redacted those areas …that’s our most important task and we’ve already asked the agencies what those redactions were (4’50).

29

Ethnographic study

Documentary

(1.45) When David Cameron in 2010 announces there’s going to be this judge led inquiry into what exactly our intelligence services were doing when other people were being tortured, when he went for that judge led inquiry what did you think of that? Well that was a good thing. I did say - I can’t remember how publicly - but I certainly said as it were on a more private level that we should be cognisant of who the judge should be because you know judges aren’t stamped out of a template, they’re not all the same – as anybody who deals with the courts knows – and they … and so we didn’t want to pick a judge who was already as it were a trustee to the system. In fact they ended up picking Gibson, who was a past commissioner, and therefore was seen slightly to be part of the system, and that … of course from that flowed the problems that came later. (2.33) Those problems … I mean at the time when David Cameron was asked in the Commons what about the Intelligence and Security Committee, shouldn’t they be looking at it and he said that they weren’t the right body - four years on, they are going to be the body who’s looking at this. I mean do you think they’re the right people to look at it? Well let’s start with where he was then. At that point when he said that, the committee had … broadly speaking had to rewrite its own report on the 7/7 thing, had completely missed rendition – the original part of this … this story, knew nothing of and had never brought into the public domain any of the sort of mass data collection stuff. There were … In other words there were a series of problems. And of course they didn’t spot the dodgy dossier either – the Iraq War problem. So there were a whole series of things they just simply had got wrong, you know? And there are many reasons why that is the case, but that was the case then. Since then, I guess if you were asking David Cameron this question what he would say is we have reformed the ISC. Now what that means is they’ve basically given them more powers. Whether that has … whether that’s done enough to make them the appropriate body, that’s a different question, and in my view no they haven’t.

30

Ethnographic study

Documentary

Challenges

• Transcription
• Use of paper
• Audio clean-up

31

Ethnographic study

Findings

• Text-based working
• Use of paper
• Redundant speech
• Speaker diarization
• Comparing takes

32

Future work

Next steps

• Use findings to develop a prototype audio production system
• Evaluate the system in a real production environment
• Refine and repeat

33

Future work

Demo