Operating System Support for Application-Specific Speculation - - PowerPoint PPT Presentation

▶

Dec 27, 2022 324 likes •589 views

Operating System Support for Application-Specific Speculation Benjamin Wester Peter Chen and Jason Flinn University of Michigan Speculative Execution TIME Sequential dependent tasks Predict A Predict results of Task A to break

SLIDE 1

Operating System Support for Application-Specific Speculation

Benjamin Wester Peter Chen and Jason Flinn University of Michigan

SLIDE 2

Speculative Execution

Sequential dependent tasks
Predict results of Task A

to break dependence

Execute Task B in parallel
Isolate all effects
Correct prediction: commit
Wrong prediction: abort

A B

TIME

2 EuroSys 2011

Predict A

SLIDE 3

Speculation Everywhere!

Discrete event simulation
I/O prefetching
Distributed shared memory
Distributed file systems
Deadlock detection
Remote displays
Web page pre-rendering

EuroSys 2011 3

SLIDE 4

Speculation as a Service to Apps

How is this system designed? In what ways can it be customized for an app? How can those customizations be specified?

4 EuroSys 2011

SLIDE 5

Outline

Introduction
Designing Speculation as a Service
Implementation
Evaluation
Conclusion

EuroSys 2011 5

SLIDE 6

Design 1: In-App Speculation

+ Complete semantic info + Predict arbitrary app

perations

+ Safe operations allowed ‒ No reuse: significant development needed ‒ Scope is limited: unsafe

perations block

EuroSys 2011 6

App 1

Data

App 2

Data

SLIDE 7

OS

Design 2: Generic OS Speculation

+ Apps need no modifications + Wide scope: unsafe

perations taint

‒ Lacks semantic understanding of app ‒ Predict system calls only ‒ Handle application conservatively

EuroSys 2011 7

App 2 App 1

SLIDE 8

Separate Mechanism and Policy

Mechanism implements isolation Policy describes customizations

Best of both extremes

Mechanism built in OS
Common implementation
Wide scope
Policy specified in Applications
Expose semantic information

EuroSys 2011 8

SLIDE 9

OS

Design 3: Expose Predictions

+ Predict arbitrary app

perations

+ Reuse OS mechanism (with app assistance) + Wide scope for taint propagation ‒ Limited semantic info

‒ Speculative external output never allowed ‒ Commit on identical results

EuroSys 2011 9

App 2 App 1

SLIDE 10

OS

Design 4: Expose Safety

+ Predict arbitrary app

perations

+ Reuse OS mechanism (with app assistance) + Wide scope for taint propagation + More semantic info

+ Allow safe output + Commit on equivalent results

EuroSys 2011 10

App 2 App 1

SLIDE 11

Customizable Policy

Creation
What tasks are predictable
How to predict them
Output
What output is safe to allow
Commit
Which results are acceptable to commit

EuroSys 2011 11

SLIDE 12

Outline

Introduction
Designing Speculation as an OS Service
Implementation
Evaluation
Conclusion

EuroSys 2011 12

SLIDE 13

Implementation

Mechanism built in OS
Based on Speculator kernel
Checkpoints & logs processes, files, IPC, etc.
Policies expressed using system call API

EuroSys 2011 13

SLIDE 14

Speculative Process Control Process

spec_fork()

EuroSys 2011 14

TIME spec_fork predict A

B

commit

A

commit/ abort commit abort

SLIDE 15

API Example

int main() { int x; int prediction = get_prediction(); if (spec_fork() == SPECULATIVE) { x = prediction; } else { x = slow_function(); if (equiv(x, prediction)) commit(); else abort(); } set_output_policy(stdout, ALLOW); printf(“%d”, x); }

EuroSys 2011 15

Output Policy Creation Policy Commit Policy

SLIDE 16

Outline

Introduction
Designing Speculation as an OS Service
Implementation
Evaluation
Conclusion

EuroSys 2011 16

SLIDE 17

Evaluation

Can apps effectively use API to increase parallelism? Case studies

1. Predictive application launching in Bash
2. SSL certificate checks in Firefox
3. Replicated service in PBFT-CS

EuroSys 2011 17

SLIDE 18

Check command

App 1: Predictive Launching in Bash

EuroSys 2011 18

TIME Run program …finished bwester $ ▌ …finished bwester $ grep foo –r . ▌ grep foo –r . Creation Policy: Use machine learning to predict user input Commit Policy: Normalize user input before comparison Output Policy: Safe X11 Messages: Allow

SLIDE 19

How Much Work Can Be Hidden?

1 2 3 4 5 6 Run Time (seconds)

EuroSys 2011 19

TIME Speculative Non-Speculative 45 Up to 86% hidden Prompt Get cmd Spec. Execute cmd Execute Prompt Execute cmd Get cmd

SLIDE 20

Done Request page

App 2: Firefox SSL Connections

EuroSys 2011 20

TIME Web Server Validation Server Open https:// Check cert. Validate Get session key GET /?id=0123 Creation Policy: Predict certificate is valid Output Policy: Alow SSL connection Output Policy: Block private data

SLIDE 21

Connection Latency Hidden?

EuroSys 2011 21

TIME Non-Speculative Speculative 100 200 300 400 500 600 Connect Time (milliseconds)

Avg 60ms hidden

https:// Check cert. Session key Done Request https:// Check cert. Session key Done Request Validate

SLIDE 22

App 3: PBFT-CS Protocol

EuroSys 2011 22

TIME Send Request Distributed Replicated Service (Reply, sig1) Check reply Check reply Request (Reply, sig2) Work… Creation Policy: Agree on 1st reply Output Policy: PBFT-CS Messages: Allow

SLIDE 23

Improved Client Throughput?

EuroSys 2011 23

TIME Non-Speculative Speculative 100 200 300 400 500 600 700 800 900 0.5 1 1.5 2 Client Ops per Second Latency between replicas (milliseconds)

1.9x Throughput

Request1 1st Reply Verify Request2 1st Reply Request1 1st Reply Verify Request2 1st Reply Null requests

SLIDE 24

Cost of Generic Mechanism

EuroSys 2011 24

100 200 300 400 500 600 700 800 900 0.5 1 1.5 2 Client Ops per Second Latency between replicas (milliseconds)

App-specific: 8% faster

App-specific Mech Shared OS Mech Non-speculative

Null requests

SLIDE 25

Conclusion

Mechanism
Common: checkpoints, output buffering, taint propagation
Implemented in OS
Policy
App-specific: Controls creation, output, and commit
Implemented in applications
Demonstrated with 3 case studies
Improved parallelism
Small overhead relative to app-specific mechanism

Questions?

EuroSys 2011 25