Embedded Software Streaming Embedded Software Streaming via Block - - PowerPoint PPT Presentation

Embedded Software Streaming Embedded Software Streaming via Block Stream via Block Stream

A Dissertation by

Pramote Kucharoen

Dissertation Advisor

Professor Vincent J. Mooney III

7 April 2004

2

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

3

Introduction Introduction

Package Package Download & Remote execution Download & Remote execution Software Streaming Software Streaming

4

Running Applications Running Applications

• On clients (download and install)

– Advantages

• Less servers’ computing requirements
• Faster user interaction after installation complete

– Disadvantages

• Long download time
• Clients’ limited resources
• Unused features downloaded
• On servers (remote execution)

– Advantages

• Applications not downloaded
• Less clients’ computing requirements

– Disadvantages

• Overloading of servers
• Slower user interactions
• Connection loss

5

Software Streaming Software Streaming 1 2 3 4 5 6 1 3 4

6

Definitions Definitions

• Software streaming

– A method for allowing the execution of stream- enabled software even while transmission/streaming

• f the software may still be in progress
• Application load time

– the amount of time from when the application is selected for download to when the application can be executed

• Application suspension time

– the amount of time from when the application is suspended due to missing code to when the application can be resumed

7

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

8

Related Work (1) Related Work (1)

• Java

– Allows execution without obtaining all classes – Sends class files when requested – Suspends the application for every class not in memory – Requires JVM – Assumes clients has enough memory to store the entire application

9

Related Work (2) Related Work (2)

• Software caching [CH02]

– Has high occurrence of application suspension

• Liquid software [JH96]

– Requires a fast (“gigabit”) compiler

10

Related Work (3) Related Work (3)

• Function/module streaming [CK98][UR01]

– Allows execution without obtaining all functions – Transfers functions speculatively to reduce the occurrence of application suspension – Assumes clients has enough memory to store the entire application – More difficult to manage memory

11

Related Work (4) Related Work (4)

• Block streaming

– Divides files into blocks – Streams at block level – No known prior work

12

Related Work (4) Related Work (4)

application load time application suspension resources Java Software Caching Function/Module Streaming Direct Download

Block Streaming

Remote Execution

13

Outline Outline

• Introduction
• Related Work
• Block streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

14

Block Streaming Block Streaming

• Does not require virtual machines, virtual file

system, compiler at client, special support from OS

• Uses a binary rewriting technique
• Supports embedded applications and small

memory footprint devices

• Implements at user level (not OS dependant)
• Provides stream-enabled file I/O support

15

Objectives Objectives

• To reduce application load time
• To reduce application suspension time and
• ccurrence of application suspension due to

missing code/data

• To efficiently utilize resources such as

bandwidth and memory

• To support small memory footprint

embedded devices

• To optimize the above four objectives

simultaneously (as opposed to tradeoffs)

16

Objective tradeoffs Objective tradeoffs

Send when requested Send the entire application Data transfer Low High

• ccurrence of

application suspension application load time, resources

17

Block Streaming: Stream Units Block Streaming: Stream Units

Program File/Data File Program File/Data File

Stream-Enabling Info Stream-Enabling Info

Block Block

Stream-Enabling Info Stream-Enabling Info

Block Block

Stream-Enabling Info Stream-Enabling Info

Block Block Stream Units

1 2 …

18

Softstream Softstream Client/Server Model Client/Server Model

19

Softstream Softstream Protocol Layers Protocol Layers

Stream-enabled Application Stream-enabled Application Softstream Assembly Softstream Assembly Softstream Protocol Softstream Protocol TCP TCP IP IP Subnetwork Subnetwork OSI Layers:

Application Presentation Session Transport Network Link Physical

20

Transmission Profile Transmission Profile

• Is created using a profiling method
• Is used to send stream units

1 1 2 2 3 3 4 4 5 5 6 6 1 1 2 2 3 3 4 4 5 5 6 6

6 3 7 10 4 10 10

i=0 i=1 j=0 j=1

21

Flow Control (1) Flow Control (1)

• Continuous stream

– Sends stream units according to the transmission profile – Restarts a new sequence when requesting a missing stream unit

22

Flow Control (2) Flow Control (2)

• On-demand stream

– Sends stream units according to the transmission profile based on resource constraints

23

Performance Metrics Performance Metrics

• Overhead

– Bandwidth – Memory – Processing time

• Application load time
• Application suspension time
• Occurrence of application suspensions due

to missing code/data

24

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

25

Stream Stream-

• Enabled Program Files

Enabled Program Files

• Code generation
• Code modification
• Code profiling

26

System Overview System Overview

Divide Binary Image into Blocks Divide Binary Image into Blocks Generate Stream Units Generate Stream Units Create a Transmission Profile Create a Transmission Profile Request Stream Unit Request Stream Unit Load Stream Block Load Stream Block Run the Application Run the Application Link Stream Block Link Stream Block Read Transmission Profile Read Transmission Profile Send Stream Unit Send Stream Unit Accept Request Accept Request Encounter Off- Block Branch

Server Client

Not in Memory In Memory Program Entry Point Receive Block Off-block

27

Generating Blocks Generating Blocks

Source Code

#include <stdio.h> int main() int i; #include <stdio.h> int main() int i; 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1111010011101001011110 1100010010100011101001 0010010001100010111101 0011101001011110110001 0010100011101001001001 0001100010111101001110 1001011110110001001010 0011101001001001000110 0010111101001110100101 1110110001001010001110 1001001001000110001011 1101001110100101111011 0001001010001110100100 1001000110001011110100 1110100101111011000100 1010001110100100100100 0110001011110100111010 0101111011000100100010 0110100100100100010010 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1111010011101001011110 1100010010100011101001 0010010001100010111101 0011101001011110110001 0010100011101001001001 0001100010111101001110 1001011110110001001010 0011101001001001000110 0010111101001110100101 1110110001001010001110 1001001001000110001011 1101001110100101111011 0001001010001110100100 1001000110001011110100 1110100101111011000100 1010001110100100100100 0110001011110100111010 0101111011000100100010 0110100100100100010010

Binary Image Stream-Enabled Application

Stream-Enabling Info Stream-Enabling Info 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011 Stream-Enabling Info Stream-Enabling Info 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011 Stream-Enabling Info Stream-Enabling Info 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011 1000111010010010010001 1000101111010011101001 0111101100010010100011 1010010010010001100010 1001011110110001001011

Stream Unit Stream Unit Stream Unit

1 2 GCC

28

Exiting and Entering a Block Exiting and Entering a Block

comp: stwu r1,-31(r1) lwz r0,8(r1) cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b .L4 .L3: sc li r0,1 stw r0,8(r31) comp: stwu r1,-31(r1) lwz r0,8(r1) cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b .L4 .L3: sc li r0,1 stw r0,8(r31) li r3,0 .L4: … blr … li r3,0 .L4: … blr … … bl comp … … bl comp …

branch exception after executing the last instruction return

Off-block branch: Branch instruction that may cause the CPU to execution an instruction in a different block

29

Code Generation Code Generation

• Preventing the execution of non-existing code

– Static branches – Dynamic branches (return, function pointer) – Exception instructions – Last instruction of the block

• Coping with non-interruptible sections

– Stream execution code prior to the current block

• Generating stream-enabling information

– Location of the off-block branches – Branch number assigned sequentially

30

Code Modification Code Modification

• Load time code modification

– Modifies off-block branches to jump to the branch table – Stores in off-block branch information in Branch Info Table

• Run time code modification

– Modifies the off-block branches to jump to the intended code

31

Code Modification Example (1) Code Modification Example (1)

if (i==1) i=0; else i=1;

cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b .L4 cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b .L4 .L3: li r0,1 stw r0,8(r31) .L4: … .L3: li r0,1 stw r0,8(r31) .L4: …

32

Code Modification Example (2) Code Modification Example (2)

cmpwi r0,1 bne load2_1 li r0,0 stw r0,8(r31) b load2_2 cmpwi r0,1 bne load2_1 li r0,0 stw r0,8(r31) b load2_2 load2_1: … load2_2: … load2_1: … load2_2: … bne .L3 .L3: li r0,1 stw r0,8(r31) .L4: … .L3: li r0,1 stw r0,8(r31) .L4: … load3_0: … load3_0: … b load3_0

33

Code Profiling Code Profiling

6 6 5 5 4 4 3 3 2 2 1 1 7 7

Program Entry Point Program Exit Point 7 6 10 1 9 3 5 3 2 3 8 2 9 1 10 1

34

Recap: Block Streaming for Program File Recap: Block Streaming for Program File

Divide Binary Image into Blocks Divide Binary Image into Blocks Generate Stream Units Generate Stream Units Create a Transmission Profile Create a Transmission Profile Request Stream Unit Request Stream Unit Load Stream Block Load Stream Block Run the Application Run the Application Link Stream Block Link Stream Block Read Transmission Profile Read Transmission Profile Send Stream Unit Send Stream Unit Accept Request Accept Request Encounter Off- Block Branch

Server Client

Not in Memory In Memory Program Entry Point Receive Block Off-block

35

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

36

Stream Stream-

• Enabled File I/O

Enabled File I/O

• To reduce file I/O operation latency
• Motivation

– A game application renders a 1MB scene

37

Stream Units for Data Block Stream Units for Data Block

• Generate stream units by dividing file into

fixed size blocks

• Create transmission profile by profiling data
• Provide SIO function calls

– sio_open() – sio_read() – sio_write() – sio_lseek() – sio_close()

38

Block Table Block Table

0x00010000 0x00010000 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0x00010800 0x00010800 … … 0xFFFFFFFF 0xFFFFFFFF … 0x00010000: … 0x00010004: … … 0x00010400: … 0x00010404: … … 0x00010800: … 0x00010804: … … … 0x00010000: … 0x00010004: … … 0x00010400: … 0x00010404: … … 0x00010800: … 0x00010804: … … Address ID N-1 3 1 2 0x00010400

39

Data Profiling Example (1) Data Profiling Example (1)

• 1 MB data file, data sorted in ascending
• rder
• Divide the file into 10 equal-size blocks
• A database application searches for a record

using a binary search algorithm

• Create a transmission profile according to

the binary search algorithm

• Assume that the record is in block 5

40

Data Profiling Example (2) Data Profiling Example (2)

1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9

0.5 0.5 1 0.5 0.5 0.5 1 1 1

1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9

1 MB

41

Recap: Block Streaming for Data File Recap: Block Streaming for Data File

Divide File into Blocks Divide File into Blocks Generate Stream Units Generate Stream Units Create a Transmission Profile Create a Transmission Profile sio_write() sio_write()

Server Client

sio_open() sio_open() sio_close() sio_close() sio_read() sio_read() Read Transmission Profile Read Transmission Profile Send Stream Unit Send Stream Unit Accept Request Accept Request Request Stream Unit Request Stream Unit Load Stream Block Load Stream Block Not in Memory Receive Block sio_lseek() sio_lseek() Not in Memory

42

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

43

Performance Enhancement Performance Enhancement

• Objectives

– To reduce occurrence of application suspensions – To support small memory footprint embedded devices

• Code transformation

– Enforcing block boundaries – Remapping functions

• Steam unit removal

– Unlinking mechanism – Stream unit replacement

• Requirement

– Fixed sized stream blocks

44

Enforcing Block Boundaries Example Enforcing Block Boundaries Example

fn1: stwu 1,-31(1) stw 3,8(1) lwz 0,8(1) … blr fn1: stwu 1,-31(1) stw 3,8(1) lwz 0,8(1) … blr fn2: stwu 1,-31(1) stw 3,8(1) li 0,1 … blr fn3: … fn1: stwu 1,-31(1) stw 3,8(1) lwz 0,8(1) … blr fn1: stwu 1,-31(1) stw 3,8(1) lwz 0,8(1) … blr fn2: stwu 1,-31(1) stw 3,8(1) li 0,1 … blr fn3: …

45

Remapping Functions Remapping Functions

• Observations

– Programmer places functions in the file randomly – Compiler places the functions the same order as written – Program jumps from block to block – Higher occurrence of software suspensions

46

Remapping Function Example Remapping Function Example

int fn1(…) { … x = fn5(a,b); … } int fn2(…) { … } int fn3(…) { … } int fn1(…) { … x = fn5(a,b); … } int fn2(…) { … } int fn3(…) { … } int fn4(…) { … } int fn5(…) { … y = fn7(z); … } int fn6(…) { … } int fn4(…) { … } int fn5(…) { … y = fn7(z); … } int fn6(…) { … } int fn7(…) { … } int fn8(…) { … } int fn9(…) { … } int fn7(…) { … } int fn8(…) { … } int fn9(…) { … }

47

Unlinking Mechanism Unlinking Mechanism

• Linking

– Run efficiently, no code checking

• Unlinking

– Remove blocks – Need to know location of incoming branches to the block to be removed

48

Unlinking Mechanism Example Unlinking Mechanism Example

cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b load2_2 cmpwi r0,1 bne .L3 li r0,0 stw r0,8(r31) b load2_2 load2_1: … load2_2: … load2_1: … load2_2: … bne load2_1 .L3: li r0,1 stw r0,8(r31) .L4: … .L3: li r0,1 stw r0,8(r31) .L4: …

49

Stream Unit Replacement Stream Unit Replacement

• Objective

– Reduce number of retransmissions

• Example

– Game application (e.g., Doom)

• 6 MB
• 6 blocks, 1MB each

– PDA with 3 MB memory available – 3 Blocks, 1 MB each

50

Stream Unit Replacement Example Stream Unit Replacement Example

Execution profile: 6 1 2 3 1 4 1 5 3 4 1 4 3 2 3 1 2 6 1 2 Transmission profile: 6 1 2 3 4 5 1 2 6

6 6 6 6 1 1 6 6 1 1 2 2 3 3 1 1 2 2 3 3 1 1 4 4 3 3 5 5 4 4 3 3 1 1 4 4 3 3 1 1 2 2 6 6 1 1 2 2

Client memory:

9 occurrences of application suspension for demand loading potentially 6 occurrences with block streaming

51

Outline Outline

• Introduction
• Related Work
• Block Streaming
• Stream-Enabled Program Files
• Stream-Enabled File I/O
• Performance Enhancement
• Experiments and Results
• Conclusion

52

Experiments and Results Experiments and Results

• Hardware setup
• Stream-enabled program file (SPF)
• Stream-enabled file I/O (SIO)
• Stream-enable program with stream-enabled

file I/O

53

Simulation Environment Simulation Environment

VCS VCS Seamless CVE Seamless CVE XRAY XRAY

Memory Memory MPC750 MPC750 MPC750 MPC750

Address/Data Bus Main processor I/O processor MPC750: 400 MHz Bus: 83 MHZ Memory: 16 MB

54

MBX860 Board Environment MBX860 Board Environment

Network Cloud

MBX860:

• PowerPC 860, 40MHz
• 10BaseT Ethernet
• 4 MB DRAM, 2 MB Flash
• Linux 2.4.21

PC: Linux Traffic Shaper 10Mbps

55

Code Size Code Size

≈ ≈ ≈ ≈ 1400 softstream client ≈ ≈ ≈ ≈ 2200 softstream generator ≈ ≈ ≈ ≈ 1500 stream-enabled file I/O ≈ ≈ ≈ ≈ 1300 softstream loader/linker ≈ ≈ ≈ ≈ 3400 softstream server C lines Implementation

Server: softstream server softstream generator Client: softstream client softstream loader/linker stream-enabled file I/O

56

SPF Overhead (1) SPF Overhead (1)

• Overhead per off-block branch

– Bandwidth 4 bytes (location of the branch) – Memory 20 bytes (12 bytes for block table, 4 bytes for instruction, 4 bytes for location)

20 bytes Memory 4 bytes Bandwidth Overhead per off-block branch Type of overhead

57

SPF Overhead (2) SPF Overhead (2)

• Overhead per block

– Bandwidth: 12 bytes for headers + 4*n – Memory: 20*n – n = number of off-block branches

58

SPF Simulation Scenario SPF Simulation Scenario

• Adaptive autonomous robot exploration
• Impossible to write and load software for all

possible environments

• The mission control needs to update the

robot software over a 128Kbps link

• The new code is 10MB
• The robot does not need all 10MB initially
• The robot must run the software to react to

the new environment within 120 s

59

SPF Simulation Results SPF Simulation Results

0.03 7.0313% 20480 512 0.06 3.5156% 10240 1K 0.64 0.3516% 1024 10K 6.40 0.0352% 103 100K 32.77 0.0069% 20 0.5M 65.54 0.0034% 10 1M 131.07 0.0017% 5 2M 327.68 0.0007% 2 5M 655.36 0.0003% 1 10M Load time (s) Added code/block Total # of blocks Block size (bytes)

60

SPF MBX860 Board Results SPF MBX860 Board Results

0.03 0.06 0.69 7.25 34.06 66.28 132.49 331.27 662.52

100 200 300 400 500 600 700 512 1K 10K 100K 0.5M 1M 2M 5M 10M

Block Size (bytes)

Application Load Time (s)

61

Stream Stream-

• Enabled Program File Results

Enabled Program File Results

• Sending the whole software takes over 10

minutes: the deadline is missed

• Using software streaming with the first

blocks of size of 1MB, the new software can be executed within 66 seconds: the deadline is met

• The application load time improves by a

factor of ≈ ≈ ≈ ≈ 10X

• Function streaming can potentially achieve

the same result

– But function streaming lacks file I/O support

62

Stream Stream-

• Enabled File I/O (SIO)

Enabled File I/O (SIO)

• Overhead per block

– Bandwidth 8 bytes (Stream Unit ID, Data Size) – Memory 4 bytes (Entry in Block Table)

63

SIO Board Experiment (1) SIO Board Experiment (1)

• File size 1 MB
• Benchmarks

– Seq: read data sequentially – Rand 1K: read 1KB randomly – Stat: calculate various statistical values of distinct pieces the data – BSearch: find a specific value in the file using a binary search algorithm

• Implementations

– DD using Linux TCP 1.0 for NET4.0, NFS version 3, SIO

64

SIO Board Results (1) SIO Board Results (1)

61 .33 28.46 73.76 1 .1 2 80.40 52.1 8 1 1 0.48 5.54 65.01 61 .36 80.31 62.47

20 40 60 80 100 120 Seq Rand 1K Stat BSearch Time (s) SIO NFS DD

Up to 55X faster

65

SIO Board Experiment (2) SIO Board Experiment (2)

• Data acquisition

– Read a certain amount of data from a 1 MB file – Link speed 128 Kbps

66

SIO Board Results (2) SIO Board Results (2)

Time to acquire a certain amount of data

10 20 30 40 50 60 70 80 90 200 400 600 800 1000 Data (Kbytes) Time (s) SIO NFS DD

67

SIO Board Experiment (3) SIO Board Experiment (3)

• Data utilization (Kbytes per second)

– How fast data being consumed – Process a 1 MB file using various data utilization rates – Link speed 128 Kbps

68

SIO Board Results (3) SIO Board Results (3)

The amount of time it takes to process a 1 MB file

40 80 120 160 200 20 40 60 80 100 120 Data Utilization Rate (KB/s) Time (s) SIO NFS DD

69

Stream Stream-

• Enabled Application Experiment

Enabled Application Experiment

• Game application
• Program size 512 KB
• Data size 1 MB
• 128 KB code and 256 KB data needed for the

first scene

• Stream-enabled program file (SPF) embeds

data inside program

• Implementation: SIO+SPF, SPF, NFS, DD

70

Stream Stream-

• Enabled Application Results

Enabled Application Results

22.85 72.60 32.38 97.78

20 40 60 80 100 SIO+SPF SPF NFS DD User Perceived Application Load Time (s)

71

Conclusion Conclusion

• Reduce application load time by enabling

execution while transferring (10X)

• Lower application suspension time by

profiling

• Reduce the occurrence of application

suspensions

• Support small memory footprint embedded

devices

• Provide stream-enabled file I/O (55X)

72

Publications Publications

• Kuacharoen, P. and Mooney, V., “Memory management for embedded devices

using software streaming,” to be published in Proceedings of the Mobility Conference & Exhibition, Aug. 2004.

• Akgul, B., Mooney, V., Thane, H., and Kuacharoen, P., “Hardware Support for

Priority Inheritance,” in Proceedings of the IEEE Real-Time Systems Symposium, pp.246-254, Dec. 2003.

• Kuacharoen, P., Mooney, V., and Madisetti, V., “Software streaming via block

streaming,” in the book Embedded Software for SoC, edited by Jerraya, A., Yoo, S., Verkest, D. and Wehn, N., Boston, MA: Kluwer Academic Publishers,

• pp. 435-448, Sep. 2003.
• Kuacharoen, P., Mooney, V., and Madisetti, V., “Software streaming via block

streaming,” in Proceedings of the Design Automation and Test in Europe, pp. 912-917, Mar. 2003.

• Kuacharoen, P., Shalan, M., and Mooney, V., “A congurable hardware scheduler

for real-time systems,” in Proceedings of the International Conference on Engineering of Recongurable Systems and Algorithms, pp. 96-101, June 2003.

• Kuacharoen, P., Akgul, T., Mooney, V., and Madisetti, V., “Adaptability,

extensibility, and exibility in real-time operating systems,” in Proceedings of the EUROMICRO Symposium on Digital Systems Design, pp. 400-405, Sep. 2001.

• Akgul, T., Kuacharoen, P., Mooney, V., and Madisetti, V., “A debugger RTOS for

embedded systems,” in Proceedings of the 27th EUROMICRO Conference, pp. 264-269, Sep. 2001.

73

Patents Patents

• Kuacharoen, P., Mooney, V., and Madisetti, V., “Methods and

systems for transmitting application software,” U.S. Patent Application 20040006637, Jan. 2004.

• Kuacharoen, P., Akgul, T., Mooney, V., and Madisetti, V.,

“Dynamic

• perating

system,” U.S. Patent Application 20030074487, Apr. 2003.

• Akgul, T., Kuacharoen, P., Mooney, V., and Madisetti, V.,

“Debugger operating system for embedded systems,” U.S. Patent Application 20030074650, Apr. 2003.

74

Security: Issues not Addressed Security: Issues not Addressed

• Network security

– Stream applications from trusted site – SSL – Certificate

• Memory protection

75

Questions? Questions?