Sound analysis and synthesis with the package seewave J er ome - - PowerPoint PPT Presentation

▶
sound analysis and synthesis with the package seewave
SMART_READER_LITE
LIVE PREVIEW

Sound analysis and synthesis with the package seewave J er ome - - PowerPoint PPT Presentation

Feb 05, 2023 3.97k likes •4.39k views

seewave Sound analysis and synthesis with the package seewave J er ome Sueur , Thierry Aubin, Caroline Simonis Mus eum national dHistoire naturelle CNRS Universit e Paris XI sueur@mnhn.fr

slide-1
SLIDE 1

seewave

Sound analysis and synthesis with the package seewave

J´ erˆ

  • me Sueur∗, Thierry Aubin, Caroline Simonis

Mus´ eum national d’Histoire naturelle – CNRS – Universit´ e Paris XI

∗sueur@mnhn.fr – http://sueur.jerome.perso.neuf.fr/seewave.html

8 July 2008

Sueur & al. seewave

slide-2
SLIDE 2

seewave

Introduction I/O seewave Time/Amplitude Visualization Edition Analysis Frequency Visualization Analysis Modifications Synthesis Development Acknowledgments

Sueur & al. seewave

slide-3
SLIDE 3

seewave Introduction I/O

Sound packages

Maintainer First version tuneR Uwe Ligges September 2004 sound Matthias Heymann April 2005 seewave J´ erˆ

  • me Sueur

March 2006 audio Simon Urbanek September 2008

Sueur & al. seewave

slide-4
SLIDE 4

seewave Introduction I/O

Sound classes

Input Output Object tuneR readWave writeWave Wave sound loadSample saveSample Sample audio load.wave, record save.wave audioSample seewave – export, savewav vector, matrix, ts, mts, Wave, Sample, audioSample

Sueur & al. seewave

slide-5
SLIDE 5

seewave Introduction I/O

Principle

Sample Wave audioWave matrix vector ts play modification analysis synthesis .wav .flac .wav external softwares system() sound acquisition (microphone)

Sueur & al. seewave

slide-6
SLIDE 6

seewave Introduction seewave

What is seewave ?

  • signal analysis ⇒ bioacoustics, neurobiology, human voice,

telemetry, solid vibrations, radio signals, . . .

  • ≃ 70 user-end functions

Sueur & al. seewave

slide-7
SLIDE 7

seewave Time/Amplitude Visualization

Oscillogramm (single window)

0.0 0.5 1.0 1.5 Time (s) Amplitude Sueur & al. seewave

slide-8
SLIDE 8

seewave Time/Amplitude Visualization

Oscillogramm (multiple window)

0.0 0.1 0.2 0.3 0.4

Time (s) Amplitude

0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7

Sueur & al. seewave

slide-9
SLIDE 9

seewave Time/Amplitude Visualization

Envelope (absolute or Hilbert transform)

0.0 0.5 1.0 1.5 Time (s) Amplitude 0.0 0.5 1.0 1.5 Time (s) Amplitude Sueur & al. seewave

slide-10
SLIDE 10

seewave Time/Amplitude Visualization

Codes

Simple oscillogram

  • scillo(tico,f=22050)

Multi-frame oscillogram

  • scillo(tico,f=22050,k=2,j=2)

Oscillogram and enveloppe

  • scillo(tico,f=22050)

par(new=TRUE) env(tico,f=22050,msmooth=c(20,0),colwave=2)

Sueur & al. seewave

slide-11
SLIDE 11

seewave Time/Amplitude Edition

Copy, cut, . . .

  • copy
  • cut
  • paste
  • delete (trim)
  • repeat
  • everse
  • add
  • mute
  • remove silence

Sueur & al. seewave

slide-12
SLIDE 12

seewave Time/Amplitude Analysis

Time measurements

timer(orni, f=22050, threshold=5, smooth=40, bty="l", xaxs="i", colval="blue" )

Time (s) Amplitude 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

5 %

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0.02 0.1 0.08 0.09 0.08 0.06 Sueur & al. seewave

slide-13
SLIDE 13

seewave Time/Amplitude Analysis

Comparaison

  • cross-correlation
  • surface difference

Sueur & al. seewave

slide-14
SLIDE 14

seewave Time/Amplitude Analysis

Comparaison : surface difference (graph)

Sueur & al. seewave

slide-15
SLIDE 15

seewave Time/Amplitude Analysis

Comparaison : surface difference (code)

f<-16000 a<-synth(d=1,f=f,cf=2000,am=c(50,10),shape="sine") b<-synth(d=1,f=f,cf=2000,am=c(25,30),shape="tria") layout(matrix(c(1,2,3,3),byrow=TRUE,nc=2)) env(a,f=f,colwave="red",title="sound ’a’") env(b,f=f,colwave="blue",title="sound ’b’") diffenv(a,b,f=f,plot=TRUE, main="surface difference between ’a’ and ’b’")

Sueur & al. seewave

slide-16
SLIDE 16

seewave Frequency Visualization

Spectrogram (2D & 3D)

  • Short-term Fourier Transform (STFT)
  • several analysis windows (Hamming, Hanning, . . .)
  • parameters : window length, window overlap, zero-padding,

amplitude clipping level

  • with/without oscillogram
  • full color modifications

Sueur & al. seewave

slide-17
SLIDE 17

seewave Frequency Visualization

2D spectrogram (graph)

with oscillogram plot colour modifications

Sueur & al. seewave

slide-18
SLIDE 18

seewave Frequency Visualization

2D spectrogram (code)

with oscillogram plot

data(tico) spectro(tico,f=22050,

  • vlp=50,zp=16,

collevels=seq(-40,0,0.5),

  • sc=TRUE)

colour modifications

data(peewit) spectro(peewit,f=22050, palette=temp.colors, collevels=seq(-100,0,1))

Sueur & al. seewave

slide-19
SLIDE 19

seewave Frequency Visualization

3D spectrogram (graph)

false 3D (waterfall) true 3D using rgl

Sueur & al. seewave

slide-20
SLIDE 20

seewave Frequency Visualization

3D spectrogram (code)

false 3D (waterfall)

data(tico) wf(tico,f=22050,

  • vlp=50,hoff=0,voff=2,

border="#00000075")

true 3D using rgl

download.file( "http://sueur.jerome.perso.neuf.fr/ WebPage_Sounds/E_chopardi_whistle.wav", destfile="cock.wav") cock<-loadSample("cock.wav") spectro3D(cock, f=22050,wl=490,

  • vlp=85,zp=6,maga=4,

palette=spectro.colors)

Sueur & al. seewave

slide-21
SLIDE 21

seewave Frequency Analysis

Spectrum

  • static or dynamic using rpanel
  • discrete or mean with a sliding window
  • linear or dB
  • automatic or manual peak identification
  • symbolic analysis
  • 15 spectral properties (dominant peak, quantiles, flatness, skewness,

entropy, . . .)

Sueur & al. seewave

slide-22
SLIDE 22

seewave Frequency Analysis

Spectrum (graph)

discrete and dB automatic peak detection

Sueur & al. seewave

slide-23
SLIDE 23

seewave Frequency Analysis

Spectrum (code)

discrete and (weighted) dB

data(peewit) spec(peewit,f=22050, dB=TRUE,col="blue")

automatic peak detection

data(peewit) spec(peewit,f=22050,at=0.5,peak=21)

Sueur & al. seewave

slide-24
SLIDE 24

seewave Frequency Analysis

Comparaison

  • crosss-correlation
  • surface difference
  • coherence

Sueur & al. seewave

slide-25
SLIDE 25

seewave Frequency Analysis

Comparaison : cross-correlation (graph)

Sueur & al. seewave

slide-26
SLIDE 26

seewave Frequency Analysis

Comparaison : cross-correlation (code)

data(tico) spec1<-spec(tico, f=22050, at=0.2, plot=FALSE) spec2<-spec(tico, f=22050, at=1.1, plot=FALSE) corspec(spec1,spec2,main="cross-correlation between two spectra")

Sueur & al. seewave

slide-27
SLIDE 27

seewave Frequency Analysis

Frequency track

Value to track Principle Function Dominant Frequency STFT dfreq() Fundamental frequency Cepstrum fund() Autocorrelation autoc() Instantaneous frequency Zero-crossing zc() Hilbert transform ifreq()

Sueur & al. seewave

slide-28
SLIDE 28

seewave Frequency Analysis

Frequency track : example (graph)

Sueur & al. seewave

slide-29
SLIDE 29

seewave Frequency Analysis

Frequency track : example (code)

data(sheep) spectro(sheep, f=8000, ovlp=75, zp=16, scale=FALSE, collevels=seq(-45,0,1)) par(new=TRUE) dfreq(sheep, f=8000, wl=1024, ovlp=85, type="p", pch=24, bg="red", ann=FALSE) par(new=TRUE) fund(sheep, f=8000, wl=128, fmax=200, threshold=2, type="p", pch=21, bg="white", ann=FALSE) legend(1,3.9, c("Dominant frequency","Fundamental frequency"), pch=c(24,21), pt.bg=c("red","white"),bty=0)

Sueur & al. seewave

slide-30
SLIDE 30

seewave Modifications

  • frequency filters
  • positive and negative frequency shift
  • amplitude filters
  • amplitude fade in and fade out
  • remove amplitude (Hilbert transform)
  • echo generation
  • mix (+, −, ∗, /)

Sueur & al. seewave

slide-31
SLIDE 31

seewave Modifications

Frequency filter : example (graph)

Sueur & al. seewave

slide-32
SLIDE 32

seewave Modifications

Frequency filter : example (code)

par(mfrow=c(2,2)) f<-8000 a<-noise(f=f,d=1) ; spectro(a,f=f,scale=FALSE) title(main="input noise") b<-fir(a,f,to=1500) ; spectro(b,f=f,scale=FALSE) title(main="1.5 kHz bandpass") c<-fir(a,f=8000,from=1000,to=2000,bandpass=FALSE) ; spectro(c,f=f,scale=FALSE) title(main="1-2 kHz bandstop") myfilter1<-rep(c(rep(0,32),rep(1,32)),4) d<-fir(a,f=f,custom=myfilter1) ; spectro(d,f=f,scale=FALSE) title(main="custom filter")

Sueur & al. seewave

slide-33
SLIDE 33

seewave Synthesis

Crazy sound (graph)

Sueur & al. seewave

slide-34
SLIDE 34

seewave Synthesis

Crazy sound (code)

F1<-synth(f=22050,cf=2000,d=1,fm=c(500,5,1000)) F2<-synth(f=22050,a=0.8,cf=4000,d=1,fm=c(500,5,1000)) F3<-synth(f=22050,a=0.6,cf=6000,d=1,fm=c(500,5,1000)) F4<-synth(f=22050,a=0.4,cf=8000,d=1,fm=c(500,5,1000)) final<-F1+F2+F3+F4 spectro(final,f=22050,wl=512,ovlp=75,scale=FALSE)

Sueur & al. seewave

slide-35
SLIDE 35

seewave Synthesis

Imitation (graph)

Sueur & al. seewave

slide-36
SLIDE 36

seewave Synthesis

Imitation (code)

data(pellucens) f<-11025 natural<-cutw(pellucens,f=f,from=2.15, to=3.15) s1<-synth(d=0.03,f=f,cf=2300,fm=c(0,0,-315), shape="sine") s2<-synth(d=0.03,f=f,cf=2300*1.9,fm=c(0,0,-315), shape="sine") s3<-s1+(0.12*s2) s4<-s3/max(s3) s5<-addsilw(s4,f=f,d=0.015 s6<-repw(s5,f=f,times=20) s7<-fadew(s6,f=f,din=0.25,shape="cos") result1<-pastew(s7,pellu,f=f) result2<-addsilw(result1,f=f,at=1,d=0.25) spectro(result2, f=f, wl=256, ovlp=95, osc=TRUE, palette=rev.gray.colors.1) mtext(c("natural","synthetic"), side=3, at=c(0.2,0.7), line=1.5, font=3)

Sueur & al. seewave

slide-37
SLIDE 37

seewave Development

HELP !

  • Wigner-Ville distribution (WVD) for fast FM analysis
  • Gabor transform for time/frequency analysis
  • Maximum Entropy Spectral Analysis (MESA) for pulse frequency

analysis

  • Linear Predictive Coding (LPC) for speech analysis
  • Instantaneous frequency modification for signal synthesis and

modification

  • . . .

Sueur & al. seewave

slide-38
SLIDE 38

seewave Acknowledgments

THANKS !

  • Contributors : Jonathan Lees, Martin Maechler, Sandrine Pavoine,

Zev Ross, Luis J. Villanueva-Rivera, Carl G. Witthoft

  • Support : Michel Baylac, Emmanuel Paradis, and many users !
  • R Core Team : Kurt Hornik, Uwe Ligges

Sueur & al. seewave