CTP431- Music and Audio Computing, Fall 2017 Introduction Graduate - - PowerPoint PPT Presentation

Graduate School of Culture Technology, KAIST Juhan Nam

1

CTP431- Music and Audio Computing, Fall 2017 Introduction

Who We Are

§ Instructor

– Juhan Nam (남주한) – Assistant Professor in GSCT, KAIST – Music and Audio Computing Lab: http://mac.kaist.ac.kr

§ TAs:

– Jongpil Lee (이종필), Ph.D. Student in GSCT, KAIST – Wonil Kim (김원일), M.S. Student in GSCT, KAIST

2

What Is This Course About?

§ Introduction to Music Technology

3

Music Technology?

Music Technology

§ A whole set of different technologies that have changed the way that people are engaged in music as composer, performer and listener

4

Music Composition Music Performance Music Listening

Distribution Musical Instruments Notation Production Education Entertainment

Piano

5

Cristofori’s FortePiano (1722) The Piano Action

§ Invented by Bartolomeo Cristofori (1720s)

Piano

6

Steinway Model D

Piano

§ Characteristics

– Rich harmonics – Sustained tone – Polyphonic and wide register

§ Influence on music

– Composers in the Romantic era: Chopin, Schumann, Liszt – Musical expressions – Tonal harmony: vertical relation of notes

7

Harmonics of Piano

Sound Recording

§ Phonautograph - Leon Scott (1857)

– The first invention of sound recording but recording only – Recent research on image to sound restoration: http://firstsounds.org/

http://www.newyorker.com/tech/elements/new-sounds-old-voices http://edcarter.net/home/phonautogram/

Sound Recording

§ Phonograph: Thomas Edison (1877)

Edison cylinder Grooves in Edison cylinder

Sound Recording

§ Gramophone: Emile Berliner (1887)

Effect of the Early Music Records

§ Globalization

– Worldwide distribution of music – Music becomes industry

Caruso <Vesti La Giubba> “The first million-seller record” 1930s superstar, 왕수복

Effect of the Early Music Records

§ Musical Form

– Changed by the limited duration (3-4 minute long) – “AABA” form (or 32-bar form) – Modern popular music

• Jazz standards

12

Souce: http://www.musicarrangerspage.com/251/what-is-a-bridge-interlude/

Musical Tone

§ A pitched tone has a periodic waveform

50 52 54 56 58 60 −0.4 −0.2 0.2 0.4 time−milliseconds amplitude

Fourier Series

§ Any periodic signal can be represented as a sum of harmonically related sine waves

Joseph Fourier

Additive Synthesis

§ Synthesize sounds by adding multiple sine oscillators

– Also called Fourier synthesis

15

OSC OSC OSC

. . .

Amp (Env) Amp (Env) Amp (Env)

. . .

+

Telharmonium

§ Additive synthesizer using electro- magnetic “tone wheels” (Cahill, 1897) § Transmitted through telephone lines

– Subscription only – The business failed

Tone wheel

Telharmonium

https://www.youtube.com/watch?v=TunkjJvbrHs

Evolved into Hammond Organ

18

Subtractive Synthesis

§ Synthesize sounds by sharping with filters

19

Filter Oscillators Amp

5 10 15 20 −60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB) 5 10 15 20 −60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB) 0.5 1 1.5 2 2.5 x 10

4

−60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB)

Oscillator Filter Filtered Sound

50 52 54 56 58 60 −0.4 −0.2 0.2 0.4 time−milliseconds amplitude

20

MiniMoog (1970)

21

Moog Synthesizers

https://www.youtube.com/watch?v=usl_TvIFtG0

23

“Switched-On-Bach” by Wendy Carlos (1968)

Magnetic Tape Recording

§ High Fidelity

– Nearly flat over audible frequency range

§ Malleable

– Record audio can be edited

§ Multi-track recording

– Record and playback simultaneously – Layer by layer recording

Music Concrete

§ Composition by tape editing

– Cut – Splice – Reverse – Speed up/down

https://www.youtube.com/watch?v=c4ea0sBrw6M

Pierre Schaeffer -- Études de bruits (1948)

Sample-based Synthesis

Mellotron (1963)

28

Les Paul’s Innovation

§ Overdubbing

– Ensemble Effects

§ Delay effects

– Tape delay or phasing effects

Les Paul

Digital Audio

… 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 …

Sample-based Sound Synthesis

… 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 …

Music Score Sound Samples + Digital Signal Processing

Digital Audio Effects

… 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 …

Digital Signal Processing Sound Transformed Sound

§ Save and play musical performance

MIDI and Sequencer

33

Step sequencer (Drum Machine) MIDI sequencer

Digital Audio Workstation

34

Launch Pad

35

DJ Machines

https://www.youtube.com/watch?v=44F0d2CbjM0

Max Mathews

§ Father of Computer Music § Developed the first programming language to synthesize sound, called Music-N (1957) § Invented Radio-baton, a baton-type controller for computer orchestra

http://opinionator.blogs.nytimes.com/2011/06/08/the-first-computer-musician/?_r=0

Music-N (Max Mathews)

"Daisy Bell”, programmed by John Kelly and Carol Lockbaum and the accompaniment was programmed by Max Mathews.

Audio Programming Language

MAX / MSP / Jitter SuperCollider

Web Audio

43

Web Audio (HTML5 Standard)

http://www.google.com/doodles/robert-moogs-78th-birthday

Web Audio

https://tonejs.github.io

Course Goals

§ Understanding theoretical backgrounds in music technology today

– Basic acoustics, digital audio – Sound synthesis, digital audio effects – Musical control and representation (e.g. MIDI) – Algorithmic composition – Music analysis and information retrieval

§ Hand-on experience with sound examples and code

– Programming: web audio API (HTML/CSS/Javascript) – Generate, modifying and controlling sounds – Visualization and graphics – Interaction

45

What is Web Audio API?

§ High-level JavaScript API for processing and synthesizing audio in web applications

– Contain a number of sound synthesis, processing, and analysis nodes

§ HTML5 standards supported in Chrome, Firefox and other web browsers

– Replacing “Flash” for multimedia capability

46

1995 2000 2005 2010 2015 1990 1985

Flash: ActionScript

VS

Why Web Audio API?

§ Based on existing language (i.e. JavaScript) § Easy to integrate with multimedia components

– MIDI (e.g. webMIDI) – Graphics (e.g. webGL) – Input sensor: camera, microphone, keyboard, mouse

§ Free and no installation § Platform-independent (but browser-dependent)

47

Why Web Audio API?

48 https://insights.stackoverflow.com/survey/2016#technology

Stackoverflow developer survey results (2016) Most popular languages used on GitHub (2016)

https://octoverse.github.com/

Course Information

§ Course webpage

– http://mac.kaist.ac.kr/~juhan/ctp431/ – Basic course info, schedule and resources

§ KLMS

– Announcement – Q&A: discussion board – Homework submission – Grading

49

Lecture Format

§ Monday

– Theory part – Read the slides before the class

§ Wednesday

– Practice part – Web audio programming – Bring your laptop

50

Grading

§ Attendance: 10%

– Attendance, participation in discussion, and so on

§ Assignments: 40%

– Javascript programming using web audio

§ Midterm: 20%

– Paper exam focusing on theories

§ Final Project: 30%

– Proposal / Presentation / Submission (by sharing on the web using Github)

51

Pre-requisites

§ Basic literacy

– Programming language: variable, control, loop, function, class – Signal processing: meaning of x, y, t and f, Fourier transform (hopefully…) – Music: basic music theory

§ HTML/CSS/Javascript: desired but not required

52