High complexity GStreamer pipelines Buzztard / Audio / Gstreamer - - PowerPoint PPT Presentation

▶
high complexity gstreamer pipelines buzztard audio
SMART_READER_LITE
LIVE PREVIEW

High complexity GStreamer pipelines Buzztard / Audio / Gstreamer - - PowerPoint PPT Presentation

Jan 25, 2024 687 likes •1.08k views

High complexity GStreamer pipelines Buzztard / Audio / Gstreamer Stefan Sauer <ensonic@hora-obscura.de>, 24.10.2011 What is buzztard? What is buzztard? ? NO NO We're getting We're getting there there Short history Short history

slide-1
SLIDE 1

High complexity GStreamer pipelines Buzztard / Audio / Gstreamer

Stefan Sauer <ensonic@hora-obscura.de>, 24.10.2011

slide-2
SLIDE 2

What is buzztard? What is buzztard?

?

slide-3
SLIDE 3
slide-4
SLIDE 4

NO

slide-5
SLIDE 5
slide-6
SLIDE 6

NO

slide-7
SLIDE 7
slide-8
SLIDE 8

We're getting We're getting there there

slide-9
SLIDE 9

Short history Short history

  • Buzz: free, but closed source and windows only
  • Dead in 2002, continued in 2009
slide-10
SLIDE 10
  • Buzztard: open source, focuses on Linux
  • Since 2003
  • Uses GStreamer :)
slide-11
SLIDE 11
slide-12
SLIDE 12

Foundation: Registry Foundation: Registry

  • Book keeping of features
  • Fast startup (cache)
  • Safe plugin probing (out-of-process)
slide-13
SLIDE 13

Foundation: Scalability Foundation: Scalability

  • Huge pipelines
  • >500 elements
  • >50 threads
  • Example buzztard song of this size
  • ~ 50 CPU use on a 1.6 GHz Intel Atom
  • on a gstreamer debug enabled build
slide-14
SLIDE 14

Foundation: Sequencing Foundation: Sequencing

  • GstController
  • Sync control-changes with rendering
  • Outside of ui threads
slide-15
SLIDE 15

Foundation: Sparse streams Foundation: Sparse streams

  • Gap flag on buffers
  • elements take shortcuts on silence
slide-16
SLIDE 16

Foundation: DecodeBin(2) Foundation: DecodeBin(2)

  • Handling of different audio formats is a NO-OP

VORBIS

WAV AIFF MP3

AU

AAC

slide-17
SLIDE 17

Foundation : Audio IO Foundation : Audio IO

  • Everything is there!
  • alsa, jack, pulseaudio, ...
slide-18
SLIDE 18

Foundation: Elements Foundation: Elements

  • Lots of utility elements available
  • data conversion, level meter, spectrum

analyzer, ...

slide-19
SLIDE 19

Foundation: Introspection Foundation: Introspection

  • Lots of GObject goodness
  • Classify elements
  • Generate dialogs
slide-20
SLIDE 20

Tricky: trick modes Tricky: trick modes

  • Jog dial like scrubbing
  • All elements need to play to the rules
  • Adder fixed in git HEAD
  • Still occational lockups in reverse mode
slide-21
SLIDE 21

Tricky: seamless loops Tricky: seamless loops

  • Don't miss a beat
  • Adder fixed in git HEAD
slide-22
SLIDE 22

Tricky: dynamic linking Tricky: dynamic linking

  • Add/remove elements while playing
  • Fragile, complicated and under-documented
slide-23
SLIDE 23

Tricky: dynamic linking Tricky: dynamic linking

  • data flow on disconnected pads is fatal

=> pad-blocking

  • pad-blocking on inactive pads hangs
  • Pad-blocking is async
  • Happens on data-flow over the pad
  • Mind the direction: src for push, sink for pull
  • Block the the the last src pad for push, first sink pad

for pull

  • Need to emulate startup/shutdown
slide-24
SLIDE 24

Tricky: dynamic linking Tricky: dynamic linking

  • What to do with conditional elements
  • un-parent from the pipeline
  • Keep in the pipeline and lock the state
slide-25
SLIDE 25

Tricky: dynamic linking Tricky: dynamic linking

  • Add elements
  • Activate pads
  • Block pads
  • Change state to PAUSED
  • Link
  • Seek (the flush unblocks)
  • Unlock the state
  • Change state to PLAYING
slide-26
SLIDE 26

Tricky: dynamic linking : add 1 Tricky: dynamic linking : add 1

slide-27
SLIDE 27

Tricky: dynamic linking : add 2 Tricky: dynamic linking : add 2

slide-28
SLIDE 28

Tricky: dynamic linking : add 3 Tricky: dynamic linking : add 3

slide-29
SLIDE 29

Tricky: dynamic linking Tricky: dynamic linking

  • Remove elements
  • Block pads
  • change state to NULL
  • Lock the state
  • Unlink
  • Unblock pads
slide-30
SLIDE 30

Tricky: dynamic linking : remove Tricky: dynamic linking : remove

slide-31
SLIDE 31

Tricky: dynamic linking Tricky: dynamic linking

  • Buzztard svn repo: design/gst/dynlink{2,3}.c
slide-32
SLIDE 32

WIP: plugin wrappers WIP: plugin wrappers

  • Ladspa and lv2 still have issues (sources)
  • More wrappers needed (dssi, vst)
slide-33
SLIDE 33

WIP: low latency audio WIP: low latency audio

  • Determine effective latency in push mode
  • Make pull mode work for audio
  • It does not lockup anymore
  • Queues need changes to not buffer
slide-34
SLIDE 34

WIP: push mode latency WIP: push mode latency

  • Latency related to bpm/tpb and sampling rate
  • e.g.: chunk-size for sink 120000 µs = 120 ms
slide-35
SLIDE 35

WIP: push mode latency WIP: push mode latency

  • 140 ms delay ~ 46ms per queue
slide-36
SLIDE 36

Future Future

  • GStreamer 1.0 :)
  • EncodeBin for recording
  • discoverer in the wave-table browser
  • if the gtk+filechooser could be populated with disco-

info asynchronously …

  • tempo information as sticky events on the pads
  • beats-per-minute + ticks-per-beat
slide-37
SLIDE 37

Thanks! Questions?

slide-38
SLIDE 38

Thanks Thanks

  • sheetmusic: by 'starrise' Trey Jones
  • sequencer: shows propellerheads reason
  • icons are from gnome and tango icon themes