iOS Multimedia CS 4720 Mobile Application Development CS 4720 iOS - - PowerPoint PPT Presentation

CS 4720

iOS Multimedia

CS 4720 – Mobile Application Development

CS 4720

iOS Architecture

2

CS 4720

Two Levels of Access

• Some features live at a higher level in the main

media player framework - MediaCoreServices

• Fine-grained control is available in the

AVFoundation framework

• Most basic camera is in MediaCoreServices
• Most audio/video in AVFoundation

3

CS 4720

Aspects of Multimedia

• Camera
• Photo/Video Library
• Audio Recording and Playback
• Video Recording and Playback
• Music Library
• WebKit
• Games / Animation

4

CS 4720

Using the Camera

• Check for camera availablility

– Yes, most all iOS devices have a camera…

• Typical use case: you want a user to take a

picture from your app and then use the picture they took

• Similar to Android – we are going to instantiate

another part of the system to take care of this instead of writing our own

5

CS 4720

Using the Camera

• What circumstances would you want to use

the camera? How would it enhance your app?

6

CS 4720

Using the Camera

7 import MobileCoreServices

controller = UIImagePickerController() if let theController = controller{ theController.sourceType = .Camera theController.mediaTypes = [kUTTypeImage as String] theController.allowsEditing = true theController.delegate = self presentViewController(theController, animated: true, completion: nil) }

CS 4720

Playing Back Video

• Instead of MediaCoreServices, we now switch

to the AVFoundationFramework

• Lots of code for this… we will look at example
• n Github

8

CS 4720

Available Codecs

• Codec = Coder/Decoder
• Some codecs are hardware enabled and some

are software

• Hardware codecs are MUCH more efficient

– Power consumption – Speed – Processing power

9

CS 4720

Codecs

• Hardware Assisted

– AAC (only recording format supported) – ALAC – MP3

• Software

– All of the above – iLBC – PCM – Others

10

CS 4720

Playing Audio

• Try to use formats that have built-in decoding
• But you can only play one at a time using the

hardware!

• Others are done via software

11

CS 4720

Playing Audio

12

var audioPlayer = AVAudioPlayer() let sound = NSURL(fileURLWithPath: NSBundle.mainBundle().pathForResource("tac-episode_114", ofType: "mp3")!) do{ print("Loading sound") let audioPlayer = try AVAudioPlayer(contentsOfURL:sound) print("Preparing to play") audioPlayer.delegate = self audioPlayer.prepareToPlay() print("Play") audioPlayer.play() }catch { print("Error getting the audio file") }

CS 4720

Accessing the Music Library

13 mediaPicker = MPMediaPickerController(mediaTypes: .AnyAudio) if let picker = mediaPicker{ print("Successfully instantiated a media picker") picker.delegate = self picker.allowsPickingMultipleItems = true picker.showsCloudItems = true picker.prompt = "Pick a song please..." view.addSubview(picker.view) presentViewController(picker, animated: true, completion: nil) } else { print("Could not instantiate a media picker") }

CS 4720

Recording Audio

14

let audioRecordingURL = self.audioRecordingPath() do { audioRecorder = try AVAudioRecorder(URL: audioRecordingURL, settings: audioRecordingSettings()) guard let recorder = audioRecorder else{ return } recorder.delegate = self /* Prepare the recorder and then start the recording */ if recorder.prepareToRecord() && recorder.record()

CS 4720

WebKit

• WebKit is an HTML browser engine
• An engine takes HTML/CSS, processes it, and

returns what should be displayed to the screen

• Blink (Chrome)
• Trident (IE)
• Gecko (Firefox et al.)
• WebKit (Safari)

15

CS 4720

WebKit

• The WebKit component is built into iOS in two

ways:

– Opening a link in Safari – Embedding the engine in your own application

• When is the right time to do either option?
• When is the right time to do neither?

– (We’ll do more with this when we are done with iOS!)

16

CS 4720

Games

• Most all game control (if you are using default

Apple libraries) is found in SpriteKit

• A special Scene is loaded inside the

ViewController

• The ViewController still manages rotation, etc.

but all control is passed to the game Scene

17

CS 4720

GameScene

18

• verride func touchesBegan(touches: Set<UITouch>, withEvent event: UIEvent?)

{ /* Called when a touch begins */ for touch in touches { let location = touch.locationInNode(self) let sprite = SKSpriteNode(imageNamed:"Spaceship") sprite.xScale = 0.5 sprite.yScale = 0.5 sprite.position = location let action = SKAction.rotateByAngle(CGFloat(M_PI), duration:1) sprite.runAction(SKAction.repeatActionForever(action)) self.addChild(sprite) } }

CS 4720

Things to Remember

• import AVFoundation or MediaCoreServices
• Make sure your ViewController inherits from

the correct delegate

• Set the delegate correctly when media is being

returned (i.e delegate = self)

19

CS 4720

Using Media

• Targetted and sparse usage
• Have a real purpose for the usage – not just for

show

• Audio effects, in general, are a bad idea
• Accessing a user’s media library is usually not

the right idea (depends on feature/app)

• Users want to feel that an app lives unto itself
• Make sure to ask permission!

20