Learning to Groove with Inverse Sequence Transformations Jon - - PowerPoint PPT Presentation

▶
learning to groove with inverse sequence transformations
SMART_READER_LITE
LIVE PREVIEW

Learning to Groove with Inverse Sequence Transformations Jon - - PowerPoint PPT Presentation

Mar 12, 2023 250 likes •390 views

Learning to Groove with Inverse Sequence Transformations Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, David Bamman contact: jongillick@berkeley.edu Google AI / Magenta Questions How well can we model drum performances with

slide-1
SLIDE 1

Learning to Groove with Inverse Sequence Transformations

Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, David Bamman contact: jongillick@berkeley.edu

Google AI / Magenta

slide-2
SLIDE 2

Questions

  • How well can we model

drum performances with machine learning?

  • Can we use these models to

make practical tools that give control to users?

slide-3
SLIDE 3
  • It is time consuming to edit

the precise timing and volume

  • f each note.
  • Our ears connect with human

performances.

  • Not everyone can play drums,

and recording drum kits is challenging and expensive.

Challenges in Editing Electronic Drums

slide-4
SLIDE 4

Some Components of a Performance

slide-5
SLIDE 5

Contributions

  • We build Machine Learning

models that condition on either a score or a groove, generating the other.

  • We collected and released

the Groove MIDI Dataset of professional drum performances for modeling.

slide-6
SLIDE 6

Models

Humanize

Model Architecture: Variational Autoencoder (VAE) or Variational Information Bottleneck (VIB) with recurrent encoders/decoders

slide-7
SLIDE 7

Models

Tap2Drum

Model Architecture: Variational Autoencoder (VAE) or Variational Information Bottleneck (VIB) with recurrent encoders/decoders

slide-8
SLIDE 8

Results: Listening Tests

Seq2Seq Real KNN

Percent of Wins

Humanized (vs KNN) Humanized (vs Real) Infilling (vs Real) Tap2Drum (vs Real)

slide-9
SLIDE 9

Groove Model Demonstration

slide-10
SLIDE 10

Drumify Model Demonstration

slide-11
SLIDE 11

Questions for the Future

  • How do professionals and/or amateur musicians

experience working with these tools?

  • How can/should we facilitate collaborations with

the expert creators (such as drummers) that enable this kind of research?

  • What specifically do these models learn? What

biases do they capture, and how does this inform future data collection?

slide-12
SLIDE 12

Thank you!

Stop by our poster: 6:30pm, Pacific Ballroom #242 for audio examples, interactive demos, and more!

g.co/magenta/groovae

Google AI / Magenta

Images Drummer by Luis Prado from the Noun Project
 Drum Machine by Clayton Meador from the Noun Project