StreamFlex High-throughput Stream Programming in Java Jesper Spring - - PowerPoint PPT Presentation

StreamFlex

High-throughput Stream Programming in Java

Jesper Spring Jean Privat, Rachid Guerraoui, Jan Vitek

High Throughput Stream Programming in Java

Realtime and Java

The Real-time Specification for Java is being used today in real world programs...

...but it is large, complex and error-prone

A simpler programming model for a particular class of applications will help increase adoption

Focus on streaming-style concurrent and real-time applications

High Throughput Stream Programming in Java

Challenges

Minimal stream processing requirements:

keep the data moving ⇒ no buffering respond instantaneously ⇒ no messages dropped

High Throughput Stream Programming in Java

StreamFlex

Programming abstraction for high-throughput stream processing:

sub-millisecond response times without interference from non-time critical activities familiar programming model that reuses standard libraries safe, non-blocking interaction with plain Java type safe while remaining expressive minimal changes to the virtual machine

High Throughput Stream Programming in Java

Plan of this talk

Challenges for high-throughput stream processing Programming model Static checking Implementation Empirical validation Conclusions

High Throughput Stream Programming in Java

StreamFlex Programming Model

A program is a graph of concurrently executing filters communicating through non-blocking channels

Filter Filter

Clock

Plain Java Plain Java

Filter Filter

Clock

High Throughput Stream Programming in Java

Programmer’s View of Memory

Transient Area Stable Area Heap Area Channels

StreamFlexGraph Filter Filter Channel Capsule

• ut

in

High Throughput Stream Programming in Java

Programming Model

A Filter defines an activity, which:

specified in abstract work() method made up of user-defined and library data types Clocks are priority scheduled filters that can preempt GC

Filters operate on disjoint partitions of the heap

stable holds persistent data, not GC’ed transient is a scratch pad reclaimed after each execution

• bject lifetimes are per-class, using Stable markers

High Throughput Stream Programming in Java

Programming Model

Filters communicate with capsules, which are:

restricted to primitives and primitive array types allocated and managed by the runtime

Capsules are put on channels, which:

are bounded non-blocking queues

Trigger expressions release filters

High Throughput Stream Programming in Java

Interaction with Java threads

Java can invoke methods on Filters causing:

a change in allocation context pinning of @borrow arguments start of a lightweight transaction

class PacketReader extends Filter { ... @atomic void write(@borrow byte[] b) { ... } }

Plain Java

High Throughput Stream Programming in Java

Intrusion Detection with StreamFlex

Analyzes stream of incoming raw network packets Detects intrusion attempts through pattern matching ~1,500 LOC, 27 classes, 10 stable classes, 1 atomic method, 7 filters, 1 clock, 1 plain Java thread etc. Processing rate is 750Mb/s !

Read

Clock

Trust VSIP Tear Join Dump

Ok Fail

High Throughput Stream Programming in Java

Intrusion Detection with StreamFlex

class Intrusion extends StreamFlexGraph { Intrusion(int periodInMicros) { clock = makeClock(periodInMicros); read = makeFilter(PacketReader.class); ... connect(clock, read, “in”); connect(read, “out”, trust, “in”, 10, 1); ... validate(); } } ... new Intrusion(80).start();

High Throughput Stream Programming in Java

Intrusion Detection with StreamFlex

class PacketReader extends Filter { private TimeChannel in; private Channel<Ether_Hdr> out; private Buffer buffer = new Buffer(16384); public void work() { TCP_Hdr p = (TCP_Hdr)makeCapsule(TCP_Hdr.class); readPacket(p);

• ut.put(p);

} } class Buffer implements Stable { .. }

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Programmer’s View of Memory

public void work() { TCP_Hdr p = ...

• ut.put(p);

}

Buffer Intrusion Clock, 80us PacketReader in

• ut

TimeChannel Channel

public void work() {

• ut.put();

}

Transient Area Stable Area Heap Area Channels

High Throughput Stream Programming in Java

Plan of this talk

Challenges for high-throughput stream processing Programming model Static checking Implementation Empirical validation Conclusions

High Throughput Stream Programming in Java

Implicit Ownership Type System

Checking is modular at the granularity of a Filter Ensure no dangling pointers by:

Objects owned by a Filter never accessed from

• utside the Filter

Filters never refer to

• bjects subject to GC

Stable objects don’t refer to transient Stable objects don’t refer to Capsules

Transient Area Stable Area Heap Area Capsules

Filter

illegal references

High Throughput Stream Programming in Java

Plan of this talk

Challenges for high-throughput stream processing Programming model Static checking Implementation Empirical validation Conclusions

High Throughput Stream Programming in Java

Virtual Machine Support

Priority-Preemptive Scheduling Memory management

memory regions pinning of borrowed objects

• perations for changing allocation context

Software Transactional Memory

simple log-based implementation

Type Checker Implementation

A pluggable type system integrated with javac

High Throughput Stream Programming in Java

Plan of this talk

Challenges for high-throughput stream processing Programming model Static checking Implementation Empirical validation Conclusions

High Throughput Stream Programming in Java

Event Correlation Benchmark

Processing time: StreamFlex vs Java

Ovm, AMD Athlon 64 X2 Dual Core 4400+ with 2GB. Linux 2.6.17 with high resolution timer patches and a tick period of 1 μs.

50 100 150 200 250 300 350 400 1 101 201 301 401 501 601 701 801 901 1001

Sfx/Ovm Java/Ovm

High Throughput Stream Programming in Java

Predictability

Inter-arrival times for a task with 80μs period Corresponds to predictability level of 98%

Ovm, AMD Athlon 64 X2 Dual Core 4400+ with 2GB. Linux 2.6.17 with high resolution timer patches and a tick period of 1 μs.

microseconds counts 40 60 80 100 120 140 160 180 500 1000 1500

High Throughput Stream Programming in Java

Plan of this talk

Challenges for high-throughput stream processing Programming model Static checking Implementation Empirical validation Conclusions

High Throughput Stream Programming in Java

Related Work

Stream Processing Thies, Karczmarek, Amarasinghe. Streamit: A language for streaming applications, CC 02. Real-time Bollella et al. The Real-time Specification for Java. Addison-Wesley, 2000. Spoonhower, Auerbach, Bacon, Cheng, Grove Eventrons: A Safe Programming Construct for High-Frequency Hard Real-Time Applications. PLDI, 2006 Spring, Pizlo, Guerraoui, Vitek. Reflexes: Programming Abstractions for Highly Responsive Systems, VEE 07. Auerbach, Bacon, Iercan, Christoph Kirsch, V.T. Rajan, Harald Röck, and Rainer

• Trummer. Java Takes Flight: Time-portable Real-time Programming with Exotasks,

LCTES 07. Ownership Types Noble, Potter, Vitek. Flexible Alias Protection. ECOOP 98. Boyapati, Lee, Rinard. Ownership type for safe memory management in real-time

• Java. PLDI 03.

High Throughput Stream Programming in Java

Conclusions

StreamFlex is a practical abstraction for integrating

high throughput stream processing with non-time critical applications The StreamFlex static type safety rules are sufficiently expressive The benchmarks demonstrate high throughput and high level of predictability

High Throughput Stream Programming in Java

High Throughput Stream Programming in Java

Predictability

Measured missed deadlines (BeamFormer).

AMD Athlon 64 X2 Dual Core 4400+ with 2GB. Linux 2.6.17 with high resolution timer patches and a tick period of 1 μs.

1000 2000 3000 4000 5000 10 20 30 40 50 60 microseconds

High Throughput Stream Programming in Java

Throughput benchmarks

BeamForm calculation on a set of inputs FilterBank for multirate signal processing

Programs adapted from StreamIt benchmarks, executed on Ovm. AMD Athlon 64 X2 Dual Core 4400+ with 2GB. Linux 2.6.17 with high resolution timer patches and a tick period of 1 μs.

StreamFlex Java BeamFormer 314ms 1285ms FilterBank 1260ms 4350ms

BeamFormer