Operating System Support for Application-Specific Speculation - PowerPoint PPT Presentation

Operating System Support for Application-Specific Speculation Benjamin Wester Peter Chen and Jason Flinn University of Michigan

Speculative Execution TIME  Sequential dependent tasks Predict A  Predict results of Task A to break dependence A  Execute Task B in parallel B  Isolate all effects  Correct prediction: commit  Wrong prediction: abort EuroSys 2011 2

Speculation Everywhere!  Discrete event simulation  I/O prefetching  Distributed shared memory  Distributed file systems  Deadlock detection  Remote displays  Web page pre-rendering EuroSys 2011 3

Speculation as a Service to Apps How is this system designed? In what ways can it be customized for an app? How can those customizations be specified? EuroSys 2011 4

Outline  Introduction  Designing Speculation as a Service  Implementation  Evaluation  Conclusion EuroSys 2011 5

Design 1: In-App Speculation + Complete semantic info + Predict arbitrary app operations + Safe operations allowed Data ‒ No reuse : significant development needed Data ‒ Scope is limited : unsafe App 2 App 1 operations block EuroSys 2011 6

Design 2: Generic OS Speculation + Apps need no modifications + Wide scope : unsafe operations taint ‒ Lacks semantic App 2 App 1 understanding of app ‒ Predict system calls only ‒ Handle application OS conservatively EuroSys 2011 7

Separate Mechanism and Policy Mechanism implements isolation Policy describes customizations Best of both extremes  Mechanism built in OS o Common implementation o Wide scope  Policy specified in Applications o Expose semantic information EuroSys 2011 8

Design 3: Expose Predictions + Predict arbitrary app operations + Reuse OS mechanism (with app assistance) + Wide scope for taint App 2 App 1 propagation ‒ Limited semantic info OS ‒ Speculative external output never allowed ‒ Commit on identical results EuroSys 2011 9

Design 4: Expose Safety + Predict arbitrary app operations + Reuse OS mechanism (with app assistance) + Wide scope for taint App 2 App 1 propagation + More semantic info OS + Allow safe output + Commit on equivalent results EuroSys 2011 10

Customizable Policy  Creation o What tasks are predictable o How to predict them  Output o What output is safe to allow  Commit o Which results are acceptable to commit EuroSys 2011 11

Outline  Introduction  Designing Speculation as an OS Service  Implementation  Evaluation  Conclusion EuroSys 2011 12

Implementation  Mechanism built in OS o Based on Speculator kernel o Checkpoints & logs processes, files, IPC, etc.  Policies expressed using system call API EuroSys 2011 13

spec_fork() TIME spec_fork predict A Speculative Control A Process Process B commit/ commit abort commit abort EuroSys 2011 14

API Example int main() { int x; int prediction = get_prediction(); Creation Policy if (spec_fork() == SPECULATIVE) { x = prediction; } else { x = slow_function(); if (equiv(x, prediction)) Commit Policy commit(); else abort(); } Output Policy set_output_policy(stdout, ALLOW); printf (“%d”, x); } EuroSys 2011 15

Outline  Introduction  Designing Speculation as an OS Service  Implementation  Evaluation  Conclusion EuroSys 2011 16

Evaluation Can apps effectively use API to increase parallelism? Case studies 1. Predictive application launching in Bash 2. SSL certificate checks in Firefox 3. Replicated service in PBFT-CS EuroSys 2011 17

App 1: Predictive Launching in Bash Run program …finished TIME bwester $ ▌ Creation Policy : Use machine learning to predict user input grep foo – r . …finished bwester $ grep foo – r . ▌ Check command Commit Policy : Normalize user input Output Policy : before comparison Safe X11 Messages: Allow EuroSys 2011 18

How Much Work Can Be Hidden? Non-Speculative Speculative 45 Up to 86% hidden TIME Prompt 6 Prompt 5 Run Time (seconds) 4 3 Spec. 2 Execute cmd 1 0 Get cmd Get cmd Execute Execute cmd EuroSys 2011 19

App 2: Firefox SSL Connections TIME Open https:// Web Server Creation Policy : Output Policy : Check cert. Predict certificate is valid Alow SSL connection Get session key Validation Server Request page Validate GET /?id=0123 Done Output Policy : Block private data EuroSys 2011 20

Connection Latency Hidden? Non-Speculative Speculative Avg 60ms hidden TIME 600 https:// https:// Connect Time (milliseconds) 500 Check cert. Check cert. 400 Session key 300 200 100 Session key Validate Request 0 Request Done Done EuroSys 2011 21

App 3: PBFT-CS Protocol TIME Creation Policy : Request Send Request Agree on 1 st reply Distributed Replicated Service (Reply, sig1) Check reply Work… (Reply, sig2) Check reply Output Policy : PBFT-CS Messages: Allow EuroSys 2011 22

Improved Client Throughput? Non-Speculative Speculative 1.9x Throughput 900 TIME Request1 Request1 800 Client Ops per Second 700 600 1 st Reply 1 st Reply 500 Request2 400 Null requests 300 200 1 st Reply Verify Verify 100 Request2 0 0 0.5 1 1.5 2 Latency between replicas 1 st Reply (milliseconds) EuroSys 2011 23

Cost of Generic Mechanism App-specific: 900 8% faster 800 Client Ops per Second 700 600 500 App-specific Mech 400 Shared OS Mech Null requests 300 Non-speculative 200 100 0 0 0.5 1 1.5 2 Latency between replicas (milliseconds) EuroSys 2011 24

Conclusion  Mechanism o Common: checkpoints, output buffering, taint propagation o Implemented in OS  Policy o App-specific: Controls creation, output, and commit o Implemented in applications  Demonstrated with 3 case studies o Improved parallelism o Small overhead relative to app-specific mechanism Questions? EuroSys 2011 25