Interruptible Tasks: Treating Memory Pressure as Interrupts for - - PowerPoint PPT Presentation

▶
interruptible tasks treating memory pressure as
SMART_READER_LITE
LIVE PREVIEW

Interruptible Tasks: Treating Memory Pressure as Interrupts for - - PowerPoint PPT Presentation

Jun 04, 2023 633 likes •1.31k views

Interruptible Tasks: Treating Memory Pressure as Interrupts for Highly Scalable Data-Parallel Programs Lu Fang 1 , Khanh Nguyen 1 , Guoqing(Harry) Xu 1 , Brian Demsky 1 , Shan Lu 2 1 University of California, Irvine 2 University of Chicago

slide-1
SLIDE 1

Interruptible Tasks: Treating Memory Pressure as Interrupts for Highly Scalable Data-Parallel Programs

Lu Fang 1, Khanh Nguyen 1, Guoqing(Harry) Xu 1, Brian Demsky 1, Shan Lu 2

1University of California, Irvine 2University of Chicago

SOSP’15, October 7, 2015, Monterey, California, USA

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 1 / 25

slide-2
SLIDE 2

Motivation

Data-parallel system

◮ Input data are divided into independent partitions ◮ Many popular big data systems

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 2 / 25

slide-3
SLIDE 3

Motivation

Data-parallel system

◮ Input data are divided into independent partitions ◮ Many popular big data systems

  • Memory pressure on single nodes

Our study

◮ Search “out of memory” and “data parallel” in StackOverflow ◮ We have collected 126 related problems

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 2 / 25

slide-4
SLIDE 4

Memory Pressure in the Real World

Memory pressure on individual nodes

◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory

Memory consumption Execution time Heap size OutOfMemoryError point Long and useless GC

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 3 / 25

slide-5
SLIDE 5

Memory Pressure in the Real World

Memory pressure on individual nodes

◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory

Memory consumption Execution time Heap size OutOfMemoryError point Long and useless GC

CRASH OutOfMemory Error

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 3 / 25

slide-6
SLIDE 6

Memory Pressure in the Real World

Memory pressure on individual nodes

◮ Executions push heap limit (using managed language) ◮ Data-parallel systems struggle for memory

Memory consumption Execution time Heap size OutOfMemoryError point Long and useless GC

CRASH OutOfMemory Error SLOW Huge GC effort

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 3 / 25

slide-7
SLIDE 7

Root Cause 1: Hot Keys

Key-value pairs

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 4 / 25

slide-8
SLIDE 8

Root Cause 1: Hot Keys

Key-value pairs Popular keys have many associated values

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 4 / 25

slide-9
SLIDE 9

Root Cause 1: Hot Keys

Key-value pairs Popular keys have many associated values Case study (from StackOverflow)

◮ Process StackOverflow posts ◮ Long and popular posts ◮ Many tasks process long and popular posts

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 4 / 25

slide-10
SLIDE 10

Root Cause 2: Large Intermediate Results

Temporary data structures

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 5 / 25

slide-11
SLIDE 11

Root Cause 2: Large Intermediate Results

Temporary data structures Case study (from StackOverflow)

◮ Use NLP library to process customers’ review ◮ Some reviews are quite long ◮ NLP library creates giant temporary data structures for long

reviews

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 5 / 25

slide-12
SLIDE 12

Existing Solutions

More memory? Not really!

◮ Data double in size every two years, [http://goo.gl/tM92i0] ◮ Memory double in size every three years, [http://goo.gl/50Rrgk]

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 6 / 25

slide-13
SLIDE 13

Existing Solutions

More memory? Not really!

◮ Data double in size every two years, [http://goo.gl/tM92i0] ◮ Memory double in size every three years, [http://goo.gl/50Rrgk]

Application-level solutions

◮ Configuration tuning ◮ Skew fixing

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 6 / 25

slide-14
SLIDE 14

Existing Solutions

More memory? Not really!

◮ Data double in size every two years, [http://goo.gl/tM92i0] ◮ Memory double in size every three years, [http://goo.gl/50Rrgk]

Application-level solutions

◮ Configuration tuning ◮ Skew fixing

System-level solutions

◮ Cluster-wide resource manager, such as YARN

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 6 / 25

slide-15
SLIDE 15

Existing Solutions

More memory? Not really!

◮ Data double in size every two years, [http://goo.gl/tM92i0] ◮ Memory double in size every three years, [http://goo.gl/50Rrgk]

Application-level solutions

◮ Configuration tuning ◮ Skew fixing

System-level solutions

◮ Cluster-wide resource manager, such as YARN

We need a systematic and effective solution!

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 6 / 25

slide-16
SLIDE 16

Our Solution Interruptible Task: treat memory pressure as interrupt Dynamically change parallelism degree

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 7 / 25

slide-17
SLIDE 17

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Program starts with multiple tasks Task Consumed Memory Task Consumed Memory Task Consumed Memory

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-18
SLIDE 18

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Program pushes heap limit Task Consumed Memory Task Consumed Memory Task Consumed Memory

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-19
SLIDE 19

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Long and useless GC Task Consumed Memory Task Consumed Memory Task Consumed Memory

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-20
SLIDE 20

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size OutOfMemory Error Task Consumed Memory Task Consumed Memory Task Consumed Memory

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-21
SLIDE 21

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Task Consumed Memory Task Consumed Memory Task Consumed Memory Long and useless GCs are detected

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-22
SLIDE 22

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Task Consumed Memory Task Consumed Memory Task Consumed Memory Killed Killed Long and useless GCs are detected, start interrupting tasks

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-23
SLIDE 23

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Killed

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-24
SLIDE 24

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Released Killed

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-25
SLIDE 25

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Released Killed Released

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-26
SLIDE 26

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Released Kept in memory, can be serialized Killed Released

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-27
SLIDE 27

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Released Kept in memory, can be serialized Final result: push

  • ut and released

Killed Released

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-28
SLIDE 28

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Release the memory, memory pressure is gone Task Consumed Memory Task Consumed Memory Task Consumed Memory Local Data Structures Processed Input Unprocessed Input Output Killed Consumed Memory Released Kept in memory, can be serialized Intermediate result: kept in memory, can be serialized Final result: push

  • ut and released

Killed Released

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-29
SLIDE 29

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size Program executes without memory pressure Task Consumed Memory Task Consumed Memory Task Consumed Memory Killed Killed

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-30
SLIDE 30

Why Does Our Technique Help

Task Consumed Memory Memory consumption Execution time Heap size If there is enough memory, increase parallelism degree Task Consumed Memory Task Consumed Memory Task Consumed Memory Killed Killed Task Consumed Memory Newly created

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 8 / 25

slide-31
SLIDE 31

Challenges How to expose semantics How to interrupt/reactivate tasks

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 9 / 25

slide-32
SLIDE 32

Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 9 / 25

slide-33
SLIDE 33

Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks → a runtime system

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 9 / 25

slide-34
SLIDE 34

Challenges How to expose semantics → a programming model How to interrupt/reactivate tasks → a runtime system

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 9 / 25

slide-35
SLIDE 35

The Programming Model

A unified representation of input/output

◮ Separate processed and unprocessed input ◮ Specify how to serialize and deserialize

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 10 / 25

slide-36
SLIDE 36

The Programming Model

A unified representation of input/output

◮ Separate processed and unprocessed input ◮ Specify how to serialize and deserialize

A definition of an interruptible task

◮ Safely interrupt tasks ◮ Specify the actions when interrupt happens ◮ Merge the intermediate results

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 10 / 25

slide-37
SLIDE 37

Representing Input/Output as DataPartitions

◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data

DataPartition Abstract Class

// The DataPartition abstract class abstract class DataPartition { // Some fields and methods ... // A cursor points to the first // unprocessed tuple int cursor; // Serialize the DataPartition abstract void serialize(); // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 11 / 25

slide-38
SLIDE 38

Representing Input/Output as DataPartitions

◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data

1 A cursor points to the first

unprocessed tuple

DataPartition Abstract Class

// The DataPartition abstract class abstract class DataPartition { // Some fields and methods ... // A cursor points to the first // unprocessed tuple int cursor; // Serialize the DataPartition abstract void serialize(); // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 11 / 25

slide-39
SLIDE 39

Representing Input/Output as DataPartitions

◮ How to separate processed and unprocessed input ◮ How to serialize and deserialize the data

1 A cursor points to the first

unprocessed tuple

2 Users implement serialize and

deserialize methods

DataPartition Abstract Class

// The DataPartition abstract class abstract class DataPartition { // Some fields and methods ... // A cursor points to the first // unprocessed tuple int cursor; // Serialize the DataPartition abstract void serialize(); // Deserialize the DataPartition abstract DataPartition deserialize(); } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 11 / 25

slide-40
SLIDE 40

Defining an ITask

◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task

ITask Abstract Class

// The ITask interface in the library abstract class ITask { // Some methods ... abstract void interrupt(); boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt if (HasMemoryPressure()) { interrupt(); return false; } process(); } } } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 12 / 25

slide-41
SLIDE 41

Defining an ITask

◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task

1 In interrupt, we define how to deal

with partial results

ITask Abstract Class

// The ITask interface in the library abstract class ITask { // Some methods ... abstract void interrupt(); boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt if (HasMemoryPressure()) { interrupt(); return false; } process(); } } } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 12 / 25

slide-42
SLIDE 42

Defining an ITask

◮ What actions should be taken when interrupt happens ◮ How to safely interrupt a task

1 In interrupt, we define how to deal

with partial results

2 Tasks are always interrupted at the

beginning in the scaleLoop

ITask Abstract Class

// The ITask interface in the library abstract class ITask { // Some methods ... abstract void interrupt(); boolean scaleLoop(DataPartition dp) { // Iterate dp, and process each tuple while (dp.hasNext()) { // If pressure occurs, interrupt if (HasMemoryPressure()) { interrupt(); return false; } process(); } } } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 12 / 25

slide-43
SLIDE 43

Multiple Input for an ITask

◮ How to merge intermediate results

MITask Abstract Class

// The MITask interface in the library abstract class MITask extends ITask{ // Most parts are the same as ITask ... // Only difference boolean scaleLoop( PartitionIterator<DataPartition> i) { // Iterate partitions through iterator while (i.hasNext()) { DataPartition dp = (DataPartition) i.next(); // Iterate all the data tuples in this partition ... } return true; } } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 13 / 25

slide-44
SLIDE 44

Multiple Input for an ITask

◮ How to merge intermediate results

1 scaleLoop takes a

PartitionIterator as input

MITask Abstract Class

// The MITask interface in the library abstract class MITask extends ITask{ // Most parts are the same as ITask ... // Only difference boolean scaleLoop( PartitionIterator<DataPartition> i) { // Iterate partitions through iterator while (i.hasNext()) { DataPartition dp = (DataPartition) i.next(); // Iterate all the data tuples in this partition ... } return true; } } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 13 / 25

slide-45
SLIDE 45

ITask WordCount on Hyracks

Map Operator Map Operator Map Operator Merge Operator Reduce Operator Final

...

Final Final Shuffling Reduce Operator

...

Reduce Operator 1 HDFS Merge Operator 1 1 n n HDFS

MapOperator

class MapOperator extends ITask implements HyracksOperator { void interrupt() {

// Push out final // results to shuffling

... } // Some other fields and methods ... } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 14 / 25

slide-46
SLIDE 46

ITask WordCount on Hyracks

Map Operator Map Operator Map Operator Merge Operator Reduce Operator Final

...

Final Final Shuffling Reduce Operator

...

Reduce Operator 1 HDFS Merge Operator 1 1 n n HDFS

ReduceOperator

class ReduceOperator extends ITask implements HyracksOperator { void interrupt() {

// Tag the results; // Output as intermediate // results

... } // Some other fields and methods ... } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 14 / 25

slide-47
SLIDE 47

ITask WordCount on Hyracks

Map Operator Map Operator Map Operator Merge Operator Reduce Operator Final

...

Final Final Shuffling Reduce Operator

...

Reduce Operator 1 HDFS Merge Operator 1 1 n n HDFS

MergeOperator

class MergeTask extends MITask { void interrupt() {

// Tag the results; // Output as intermediate // results

} // Some other fields and methods ... } Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 14 / 25

slide-48
SLIDE 48

Challenges How to expose semantics → a programming model How to interrupt/activate tasks → a runtime system

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 15 / 25

slide-49
SLIDE 49

ITask Runtime System

Monitor Scheduler Partition Manager

Data Partition Data Partition Data Partition Data Partition

ITask Runtime System

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 16 / 25

slide-50
SLIDE 50

ITask Runtime System

Monitor Scheduler Partition Manager

Data Partition Data Partition Data Partition Data Partition

Memory ITask Runtime System

Grow/Reduce Check Reduce

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 16 / 25

slide-51
SLIDE 51

ITask Runtime System

Monitor Scheduler Partition Manager

Data Partition Data Partition Data Partition Data Partition

Memory ITasks Disk ITask Runtime System

Grow/Reduce Check Serialize/Deserialize Input/Output Reduce

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 16 / 25

slide-52
SLIDE 52

ITask Runtime System

Monitor Scheduler Partition Manager

Data Partition Data Partition Data Partition Data Partition

Memory ITasks Disk ITask Runtime System

Grow/Reduce Check Interrupt/Create Serialize/Deserialize Input/Output Reduce

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 16 / 25

slide-53
SLIDE 53

Evaluation Environments

We have implemented ITask on

◮ Hadoop 2.6.0 ◮ Hyracks 0.2.14

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 17 / 25

slide-54
SLIDE 54

Evaluation Environments

We have implemented ITask on

◮ Hadoop 2.6.0 ◮ Hyracks 0.2.14

An 11-node Amazon EC2 cluster

◮ Each machine: 8 cores, 15GB, 80GB*2 SSD

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 17 / 25

slide-55
SLIDE 55

Experiments on Hadoop

Goal

◮ Show the effectiveness on real-world problems

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 18 / 25

slide-56
SLIDE 56

Experiments on Hadoop

Goal

◮ Show the effectiveness on real-world problems

Benchmarks

◮ Original: five real-world programs collected from Stack Overflow ◮ RFix: apply the fixes recommended on websites ◮ ITask: apply ITask on original programs

Name Dataset Map-Side Aggregation (MSA) Stack Overflow Full Dump In-Map Combiner (IMC) Wikipedia Full Dump Inverted-Index Building (IIB) Wikipedia Full Dump Word Cooccurrence Matrix (WCM) Wikipedia Full Dump Customer Review Processing (CRP) Wikipedia Sample Dump

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 18 / 25

slide-57
SLIDE 57

Improvements

Benchmark Original Time RFix Time ITask Time Speed Up MSA 1047 (crashed) 48 72

  • 33.3%

IMC 5200 (crashed) 337 238 41.6% IIB 1322 (crashed) 2568 1210 112.2% WCM 2643 (crashed) 2151 1287 67.1% CRP 567 (crashed) 6761 2001 237.9%

◮ With ITask, all programs survive memory pressure ◮ On average, ITask versions are 62.5% faster than RFix

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 19 / 25

slide-58
SLIDE 58

Experiments on Hyracks

Goal

◮ Show the improvements on performance ◮ Show the improvements on scalability

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 20 / 25

slide-59
SLIDE 59

Experiments on Hyracks

Goal

◮ Show the improvements on performance ◮ Show the improvements on scalability

Benchmarks

◮ Original: five hand-optimized applications from repository ◮ ITask: apply ITask on original programs

Name Dataset WordCount (WC) Yahoo Web Map and Its Subgraphs Heap Sort (HS) Yahoo Web Map and Its Subgraphs Inverted Index (II) Yahoo Web Map and Its Subgraphs Hash Join (HJ) TPC-H Data Group By (GR) TPC-H Data

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 20 / 25

slide-60
SLIDE 60

Tuning Configurations for Original Programs

Configurations for best performance

Name Thread Number Task Granularity WordCount (WC) 2 32KB Heap Sort (HS) 6 32KB Inverted Index (II) 8 16KB Hash Join (HJ) 8 32KB Group By (GR) 6 16KB

Configurations for best scalability

Name Thread Number Task Granularity WordCount (WC) 1 4KB Heap Sort (HS) 1 4KB Inverted Index (II) 1 4KB Hash Join (HJ) 1 4KB Group By (GR) 1 4KB

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 21 / 25

slide-61
SLIDE 61

Improvements on Performance

WC HS II HJ GR 1 2

1 1 1 1 1 1.4 1.11 1.28 1.67 1.61 Normalized Speed Up Original Best ITask

On average, ITask is 34.4% faster

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 22 / 25

slide-62
SLIDE 62

Improvements on Scalability

WC HS II HJ GR 10 20

1 1 1 1 1 5.1 2.7 24 6 5 Normalized Dataset Size Original Best ITask

On average, ITask scales to 6.3×+ larger datasets

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 23 / 25

slide-63
SLIDE 63

Conclusions

A programming model + a runtime system

◮ Non-intrusive ◮ Easy to use

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 24 / 25

slide-64
SLIDE 64

Conclusions

A programming model + a runtime system

◮ Non-intrusive ◮ Easy to use

First systematic approach

◮ Help data-parallel tasks survive memory pressure

ITask improves performance and scalability

◮ On Hadoop, ITask is 62.5% faster ◮ On Hyracks, ITask is 34.4% faster ◮ ITask helps programs scale to 6.3× larger datasets

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 24 / 25

slide-65
SLIDE 65

Thank You

Q & A

Lu Fang et al. Interruptible Tasks SOSP’15, October 7, 2015 25 / 25