CPL 2016, week 4 Inter-thread communication Oleg Batrashev - - PowerPoint PPT Presentation

CPL 2016, week 4

Inter-thread communication Oleg Batrashev

Institute of Computer Science, Tartu, Estonia

March 2, 2016

Overview

Studies so far:

• 1. Inter-thread visibility: JMM
• 2. Inter-thread synchronization: locks and monitors
• 3. Thread management: executors, tasks, cancelation

Today:

◮ Inter-thread communication

Next week:

◮ Actor model

Inter-thread communication 75/101 Confinements -

Outline

Inter-thread communication Confinements Queues and messages

Producer/consumer pattern Pros and cons

Advanced patterns

Back pressure Non-FIFO queues

Channels vs streams

Non-determinism Declarative concurrency

Inter-thread communication 76/101 Confinements -

Confinement idea

◮ need to syncrhonize data accesses from multiple threads ◮ ensure some other way, that the data is accessed from single

thread at a time

◮ use language semantics, programming pattern/protocol, design

rules...

◮ if we can do that, no locking is needed!

◮ confine (bound) data to

• 1. single method/function, sequence of methods
• 2. single thread
• 3. single object
• 4. single object/thread from a group of objects dynamically

◮ i.e. data is changing “ownership” over time

Inter-thread communication 77/101 Confinements -

Method confinement

Within single method only executing thread can access:

◮ local variables of primitive type, ◮ new objects, which reference do not escape.

An object reference escapes if:

• 1. it is passed as an argument to another method/constructor
• 2. it is returned from method invocation
• 3. stored to global variable (static or object field, accessible from

another thread)

• 4. if another object escapes that can be traversed to this object

Inter-thread communication 78/101 Confinements -

Hand-off and other protocols

Alternative to using exclusively inside a method:

◮ pass “ownership” (read/write) of the object (hand-off)

◮ tail call hand-off – ensure the object is never used locally after

it escapes

Location loc = new Location (10 ,20); // use loc here

• therMethod (loc ); // do not use loc

here after } ◮ caller copies, receiver copies the object ◮ use scalar (primitive) arguments

• therMethod (loc.lon , loc.lat );

◮ trust – beleive a callee does not publish the object, so it can

be used without locking after the callee returns

◮ similar problem exists for a single thread – when passing

Location to a method, can we be sure it is not changed?

Inter-thread communication 79/101 Confinements -

Thread confinement

Make sure data is only visible from single thread:

◮ private field in Thread subclass

◮ make sure it is accessed only from the run() method or its

private helper subroutines

◮ similarly, can be private field in Runnable ◮ dedicated objects are also ok if references are not escaped

◮ ThreadLocal in Java

◮ every thread has its own version of the reference stored in the

ThreadLocal object

◮ often used for sessions and other contexts

Thread confinement is essential part of Actor model (next lecture) and CSP (Concurrent Sequential Processes [read Google Go language])

◮ how it can be useful if accessed from single thread? see queues

and messages

Inter-thread communication 80/101 Confinements -

Object confinement

An object is accessed from single thread if

◮ is only visible within another (encapsulation in OOP), ◮ is only accessed from synchronized methods

◮ does not escape

In other words, synchronization is delegated to the containing

• bject

◮ remember, proper usage may require client synchronization

list data is escaped through its iterators

Inter-thread communication 81/101 Confinements -

Confinement within groups

Consider example:

• 1. data is confined to an object

◮ object responsibility is to synchronize access to it

• 2. take() returns the data but deletes reference in the object

◮ remember collection example with 2 locks (Refining locks)

• 3. now data is confined to the thread

◮ only the thread can access it through its local variable

• 4. put data to some other object and do not use anymore in the

thread

• 5. now data is confined to another object

Variants and names:

◮ known as tokens, capabilities, resources ◮ ring communication in hardware or software to pass the token

Requires careful code design.

Inter-thread communication 82/101 Queues and messages -

Outline

Inter-thread communication Confinements Queues and messages

Producer/consumer pattern Pros and cons

Advanced patterns

Back pressure Non-FIFO queues

Channels vs streams

Non-determinism Declarative concurrency

Inter-thread communication 83/101 Queues and messages -

Total thread confinement

In ideal situation, data is completely confined to (two) threads if:

◮ they run on different machines, ◮ they communicate by sending messages over the network.

In JVM threads may almost acheive this if:

◮ almost all data is local to threads,

◮ except queues that contain messages and reflect network

communication;

◮ data is cloned (duplicated) if sent using message to another

thread. No synchronization is needed, except queue access!

◮ cloning data is inefficient – consider immutable objects.

Inter-thread communication 84/101 Queues and messages -

Immutable messages

• 1. mutable Location objects passed inside Traveler to

Server.push()

public class Location { public double lon , lat;

◮ end up being encoded to the network socket, ◮ modified in place by the simulation, ◮ if done from two different threads lon and lat may have values

from different simulation runs;

◮ should copy the location once Server gets its own thread.

• 2. immutable Point objects has no such problem and may be

passed to different threads without worry

public class Point { public final int x, y;

Messages are passed from one thread to another – it may be good idea to make them immutable.

Inter-thread communication 85/101 Queues and messages -

Java Queue interfaces

From Java docs of the Queue and BlockingQueue interfaces

Throws exception Returns special value Blocks Times out

Insert add(e)

• ffer(e)

put(e)

• ffer(e,time,unit)

Remove remove() poll() take() poll(time,unit) Examine element() peek()

• ◮ first line 4 methods to insert elements into queue

◮ second line 4 methods to remove elements from queue ◮ third line 2 methods to examine the queue without removing

elements

◮ first 2 columns differ only if queue is empty or full ◮ last 2 columns are only available for BlockingQueue

Inter-thread communication 86/101 Queues and messages - Producer/consumer pattern

Producer and consumer

The simplest scenario:

◮ one or several threads put elements to the queue – producers ◮ one or several threads take elements from the queue and

process them – consumers Queue implementation in Java are

◮ ConcurrentLinkedQueue -- unbounded; non-blocking

◮ non-blocking write – obvious because the queue is unlimited, ◮ non-blocking read – you cannot wait for an element to appear

in the queue;

◮ LinkedBlockingQueue – unbounded; blocking read ◮ ArrayBlockingQueue – bounded; blocking

Inter-thread communication 87/101 Queues and messages - Producer/consumer pattern

Transducers

One thread may be both consumer and producer – transducer (transformer)

◮ take element from one queue ◮ transform it, e.g.

◮ throw away not needed elements (filter) ◮ extract useful information from each element (map) ◮ aggregate messages and produce compound value (reduce)

◮ put result into another queue while (isRunning) { e = queue1.take (); if (! isAcceptable (e)) continue; aggrValue += e.value; if (aggrValue > 100) queue2.put(new Message(aggrValue )); }

Inter-thread communication 88/101 Queues and messages - Pros and cons

Deadlock free execution

Consider application with producers/consumers/transducers only:

Producer

queue1

TransducerA TransducerB

queue2

◮ messages and their referenced objects are either immutable,

copied, or ensured not to be used when sent out

◮ i.e. no synchronization on message contents are needed

◮ threads need to synchronize only on queue access, ◮ no thread ever takes two locks at a time.

Hence this system is deadlock free! It may stall only because of lack of generated messages.

◮ This is essentially confinement within groups pattern. ◮ Actual idea – limit thread execution boundaries.

◮ no thread walks from network layer to GUI layer!

Inter-thread communication 89/101 Queues and messages - Pros and cons

Problem with queues

Not everything is so easy though...

◮ Unbounded queues may lead to the exhaustion of (memory)

• resources. Solutions:

◮ stop producers through back-pressure: its simplest form is

bounded queue;

◮ do not let the queue grow: throw away or aggregate messages.

◮ Race conditions are still there and even more, for example:

• 1. Producer generates messages data and stop;
• 2. TransducerA reads from queue1, puts result to queue2;
• 3. TransducerB reads and puts transformed message to queue1;
• 4. TransducerA processes this transformed message.

◮ It may happen that stop message is put by the producer

between 1 and 3, in which case the transformed message is put after the stop is processed by TransducerA.

◮ It may not happen with single thread “penetrating all layers”.

Inter-thread communication 90/101 Advanced patterns -

Outline

Inter-thread communication Confinements Queues and messages

Producer/consumer pattern Pros and cons

Advanced patterns

Back pressure Non-FIFO queues

Channels vs streams

Non-determinism Declarative concurrency

Inter-thread communication 91/101 Advanced patterns - Back pressure

Bounded blocking queues

Back pressure – stop or slower the producer if the consumers cannot keep in pace. Simplest back pressure solution, make the queue that is:

◮ bounded – limit size of the queue, ◮ blocking – block the thread if the queue is full.

Producer/transducer may be completely stopped by the full queue. It may end up in the deadlock:

Producer

queue1

TransducerA TransducerB

queue2

• 1. TransducerA generates too many messages and is blocked on

queue2,

• 2. TransducerB is blocked on queue1 by the same reason.

Inter-thread communication 92/101 Advanced patterns - Back pressure

Advanced back pressure techniques

Some more sophisticated techniques:

◮ Do not limit the queues but implement smarter transducers:

◮ communicate back to producer if ready to accept new

messages,

◮ producer suspends generating new messages until receives

notification from the consumer,

◮ for example, TCP protocol.

◮ Deadlock recognition by the thread scheduler that may enlarge

certain queues.

◮ Scheduler looks at the queues and threads and

suspends/resumes threads that threaten the execution (flood the queues).

Inter-thread communication 93/101 Advanced patterns - Back pressure

Synchronous queue

There is synchronous queue, where producer and consumer block until message is transfered:

◮ blocking queues with limit 0 ◮ Java SynchronousQueue ◮ Go language has built-in synchronous queues called channels

Block producer until certain message is transfered:

◮ Java TransferQueue

◮ “A BlockingQueue in which producers may wait for consumers

to receive elements.”

Inter-thread communication 94/101 Advanced patterns - Non-FIFO queues

Discarding/aggregating queue

Possibilities for a bounded queue that is full:

◮ discard new messages – bounded non-blocking queue

◮ use add(e) or offer(e) in Java Queue interface

◮ discard old messages – overwrite old data in the queue

◮ Java existing implementation? ◮ for example traveler old coordinates are not relevant.

◮ discard unimportant messages – (see also priority queue) ◮ aggregating – combine messages based on some criteria.

Remember filter/map/reduce in transducers – boundary between transducers and queues is blured:

◮ queue borrows producer thread for short period to

filter/aggregate elements.

Inter-thread communication 95/101 Advanced patterns - Non-FIFO queues

Priority queues and delayed queues

Deliver message in the different order, in Java:

◮ PriorityQueue – unbounded queue

◮ priority specified by the natural ordering of elements or

Comparator object

◮ PriorityBlockingQueue – also unbounded but supports

blocking read Deliver message in the different order by applying delay:

◮ in Java DelayQueue ◮ ordering is specified by the delay during message add(e) ◮ take() returns only when time for some message has expired

Inter-thread communication 96/101 Advanced patterns - Non-FIFO queues

Paced execution

Consider cases:

◮ do not want to redraw more often than 24 times a second ◮ may take many messages at once and update our state

Paced execution – aggregate updates (messages) and delay before running the code:

◮ may hugely increase efficiency of the application ◮ need careful combination of delays and aggregation

Example - update GUI not more often than once in 50ms:

◮ DelayQueue, put

◮ data update messages with delay 0 ◮ GUI update message with delay 50

◮ Upon receive

◮ data update is aggregated into existing collection ◮ GUI message checks if new data has been received, puts

aggregated data to the next queue and re-schedules itself in another 50ms

Inter-thread communication 97/101 Channels vs streams -

Outline

Inter-thread communication Confinements Queues and messages

Producer/consumer pattern Pros and cons

Advanced patterns

Back pressure Non-FIFO queues

Channels vs streams

Non-determinism Declarative concurrency

Inter-thread communication 98/101 Channels vs streams - Non-determinism

Message preserving read

Are multi-threaded programs doomed to have race conditions?

◮ one thread writes numbers 1 to 10 into the queue ◮ another thread reads them, sums up and prints out ◮ are different results possible?

◮ two threads read from the queue... ◮ two threads iterate the queue... ◮ two threads write to the queue + reading thread multiplies by

the value from every second message...

If every thread iterates a queue

◮ it does not consume the messages and leaves them for other

threads to read

◮ in some languages it is stream, as opposed to channel

◮ no other sources of non-determinism (next slide) ◮ then there are no race conditions and the program is

completely deterministic.

Inter-thread communication 99/101 Channels vs streams - Non-determinism

Sources of non-determinism

◮ Math.random() ◮ polling – do another thing if message has not yet arrived ◮ several threads consume the queue with no order

◮ iterating the queue is still ok

◮ several threads write to the same queue with no order ◮ a thread does not try to read from 2 queues with no order

◮ e.g. reading one here after one there is still ok

Every such source of non-determinism may cause race condition,

• therwise there are none.

Inter-thread communication 100/101 Channels vs streams - Declarative concurrency

Race condition free execution

Consider map/filter/reduce execution:

◮ every part (function) runs in its own thread ◮ threads wait for data to be available - reading streams of data

Final result is deterministic (number crunching usually must be). Other names:

◮ lockstep execution, declarative concurrency

Problems with such execution:

◮ bad efficiency in high latency networks – cannot re-order

messages

◮ completely broken on exception in single part ◮ completely broken on network single link failure

We need more independent components - e.g. Actor model (next lecture)

Inter-thread communication 101/101 Channels vs streams - Declarative concurrency

Summary

◮ confinement hides data in an owner: method, thread or object ◮ confinement within groups allows to change ownership for the

• bject

◮ immutable objects help in concurrency ◮ structure your aplication using consumers/producers and

queues that they use for communication

◮ avoids thread knots – no deadlocks with unbounded queues ◮ usually need to bound queues to avoid out-of-memory

◮ back presure technques as well as non-FIFO queues may help ◮ race condition free concurrency is possible but has limited

application