Paper Summaries � Any takers? Sound and Animation � This week is the last week for paper summaries. Projects Assignments and such � Presentations � Grading � Schedule is now up on Web site � Keyframing – most should have gotten � All projects have been scheduled! mail from me. � Please check date you are to present � Billiards – some should have gotten mail � Grad reports from me. � Next week � Final Presentation schedule – now final! � Please check date you are to present � Please include README files!!! Assignments and such Plan for Articulated Figure Motion � Articulated Figure Motion � Last week � Monday, Feb 6th � Today: Intro / Forward Kinematics / Spacetime Constraints � Group animation (flocking / particle) due � Wednesday: Inverse Kinematics � Hey the dropbox was left open! � This Week � Monday: Motion Capture / Assignment # 4 � Wednesday, Feb 15 th � Wednesday:Advanced Techniques in Articulated Figure � moCap assignment due Motion (using GAs /Procedural Methods) � Next week � Character animation / Guest Lecture on Falling � Sound & Animation 1
Projects Projects � Final Reports � Final Report � Textual description of your system � Note that final reports / code are � Sections SEPARATE grading components � Problem/Project Description � Approach � Final reports/code are due on the last day � Implementation of class (March 2nd) � Overall System Architecture. � Overall Program Architecture � Note this is a hard deadline � Description of major data structures / objects � Dropbox is set up for report and code � Results / User Documentation submissions. � Future Enhancements � Appendix -- All Code listings Plan for today Motivation Films � Sound and Animation � Animations by Wayne Lytle � Visualization Guru at Cornell Theory Center � Quit to start Animusic in 1995 Motivational Film Motivational Film � More Bells and Whistles (1990) � Pipe Dream (2001) � Lytle wrote the the code for each band � Animusic member � Can’t See too much Animusic � Motion is MIDI controlled � Sound drives motion � First of several Animusic pieces to be shown at SIGGRAPH 2
Motivational Film Motivational Film � Train Wreck (2003) � Geri’s Game (1997) � Martin Burolla � Technical challenge: Human & Clothing Modeling � From animation class (20022) � Geri makes a cameo appearance in Toy Story II. Sound and Animation Sound � Issues in Sound and Animation � What is sound? � Sound Generation � From webster.com � What do we play? � mechanical radiant energy that is transmitted by longitudinal pressure waves in a material � Sound Synchronization medium (as air) and is the objective cause of � When do we play? hearing � Spatial Sound � Where do we play Sound Remember this? � What is sound? � Spatial vs frequency domains � Sound can be described as a 1 dimensional � Most well behaved functions can be signal in time described as a sum of sin waves (possibly offset) at various frequencies � Describing a function by the contribution sound = f(t) (and offset) at each frequency is describing the function in the frequency domain 3
Sound Sound � A mathematical description of an audio signal: ∞ = ∑ π ω + φ f ( t ) A sin( 2 t ) i i i = i 0 Contribution/amplitude frequency phase Foley/VanDam Sound: Loudness Sound: Pitch � Looking at sound in the temporal � Looking at sound in the frequency domain domain. � Sound can be described as a 1 dimensional � Humans “hear” sounds because of signal in time periodicities in the audio signal. � Humans perceive frequency as the sensation of pitch. � Signal values represent amplitude. � Humans can perceive pitches due to periodicities ranging from 20 – 20000 � We perceive the effect of amplitude as vibrations / sec (Hz). loudness. Sound: Pitch Sound: Timbre � Tone quality of a sound � Remember our discussion of CD audio � Formally defined as � sampling rate of 44,100 samples/sec � ∆ = 1 sample every 2.26x10 -5 seconds � Characteristic of sound not due to amplitude and pitch. � CDs can accurately reproduce sounds with � Also defined frequencies as high as 22,050 Hz. � Quality of tone that distinguishes between musical instruments � Sound shape 4
Sound: Timbre Sound: Summary � Timbre is the perception of the Physical Perceptual “spectral makeup” of a signal. Characteristic Characteristic � Adding non-fundamental frequency to the Amplitude Loudness signal. Frequency Pitch � Another annoying audio applet – Timbre Spectral “shape” Timbre Sound Generation Sound Generation � So how does one generate sound for � When talking about digital (sampled animation? sound) � Easiest means � The process of digitizing is called pulse code modulation (PCM). � Recording / Sampling -- Still the primary means for sound generation in the film industry � PCM = = sampled sound � Using sampled sound – Still the primary means � WAV for sound use in games. � AIFF � MP3 (compressed PCM) Sound Generation Sound Generation � Subtractive Synthesis � Additive Synthesis ∞ ∞ = ∑ = ∑ π ω + φ π ω + φ f ( t ) A sin( 2 t ) f ( t ) A sin( 2 t ) i i i i i i = = i 0 i 0 � Define values for A i , ω i , and φ i � Start with noise (equal energies at all � Calculate sin and add frequencies) � Alternately, do in the addition in frequency � Subtract contribution of frequencies space. from noise. 5
Sound Generation Sound Generation � Granular Synthesis � Using physically based models � Like particle system � SIGGRAPH 2001 Videos � Combine a multitude of sound “grains” into a sound events � Questions Sound Synchronization Sound Synchronization � Sound driving motion � Sound must be synchronized to the � MIDI motion � Designed as a communication mode between � Methods: sythesizers, samplers, instruments, computers � Sound events � Motion driving sound � Pitch � Defining Sound events � Devices � Deriving timbre from motion � Used by Animusic in creating their videos � Sound driving motion � Example video from Bingo guys. Spatial Sound 3D Sound � Sounds (and listeners) have spatial positions � Making sounds appear as if they are � 3D sound emitted from a given position � Making sounds appear as if they are emitted from a accounting for listener position given position accounting for listener position � Head related transfer functions (HRTF) � Reverberation � Filtering of sound based on reflection off of environment � Audio cubes / surround sound � Doppler Effect � Strategic placing of speakers � Change in pitch due to moving objects 6
3D Sound: HRTF 3D Sound: HRTF � a description of all the physical cues of sound localisation. � Implemented as filters � function of four variables: ie three space coordinates and frequency. � Determined by measurement Anderson/Casey 3D Sound: HRTF 3D Sound: reverberation � Like light, sound can be seen as traveling in 3D environment in rays. � Unlike light, sound travels much slower � Speed of sound: � Speed of light Anderson/Casey 3D Sound: reverberation 3D Sound: reverberation � Reverberant sound is � Examples the collection of all the � From BKL Consultants Ltd. reflected sounds in an (http://www.bkla.com/reverb.htm) enclosed space � No reverb � Acoustics � 0.8 sec reverb time � Reverb Time = time � 1.5 sec reverb time required for sound to � 5.0 sec reverb time decay one millionth of the original power � One more SIGGRAPH video 7
3D Sound: Doppler effect 3D Sound: Doppler effect � Non-annoying applet break Sound: Putting It all Together Sound: Putting It all Together � Sound Rendering Video Examples Takala/Hahn Remember CGII: Procedural Approaches to sound generation Shading � Shade Trees [Cook84] � Hueristic � Shading calculated by combining basic � Whatever sounds good. functional operations. � Physically Based � Operations are organized in a tree. � Analysis of vibration � Nodes - Operations � Children - operands � Emperical data � Result of shade tree evaluation is a color � Sampled Sound � Equiv to parse tree (compiler design) � Basis of Renderman shading language. 8
Remember CGII: Procedural Remember CGII: Procedural Shading Shading � Shade Trees - example…copper � Basic ideas behind shade trees: � Describe textures / shading functionally � Using Parameters from 3D world � Can we use a similar model for sound? [Cook84] Timbre Trees Timbre trees � Functional sound synthesis � Sound related functions � Periodic functions � Convolution � Noise � Filtering � Nodes for animation, 3d parameters Hahn/Geigel. Et al Timbre trees Timbre trees � Nodes could also be used to simulate: � Reverberation � Delay � Spatial Sound Hahn/Geigel. Et al 9
Recommend
More recommend
