Design Patterns & Concurrency Sebastian Graf, Oliver Haase 1 - - PowerPoint PPT Presentation

Sebastian Graf, Oliver Haase

Design Patterns & Concurrency

1

Montag, 25. April 2011

Fundamentals

2

Montag, 25. April 2011

Recap

• Threadsafety
• What is threadsafety?
• How can threadsafety be ensured?
• Amdahls Law

3

Montag, 25. April 2011

Recap Threadsafety

4

public final class HitCounter { /** internal state counter variable */ private int counter = 0; /** * Getting the counter right now * @return the counts */ public final int getCounter() { return this.counter; } /** * Counting multiple values * @param countsToAdd numbers to add to the counter */ public final void increment(final int countsToAdd) { counter = counter + countsToAdd; } /** Counting by one */ public final void incrementByOne() { increment(1); } }

Montag, 25. April 2011

Recap Threadsafety

5

public final class SynchronizedCounter { /** internal state counter variable */ private int counter = 0; /** * Getting the counter right now * @return the counts */ public synchronized final int getCounter() { return this.counter; } /** * Counting multiple values * @param countsToAdd numbers to add to the counter */ public synchronized final void increment(final int countsToAdd) { counter = counter + countsToAdd; } /** Counting by one */ public final void incrementByOne() { increment(1); } }

Montag, 25. April 2011

Recap Threadsafety

6

import java.util.List; public final class ListSum { /** * Summing multiple variables * * @param multiple * variables * @return the sum of the variables */ public final int sum(final List<Integer> factors) { int sum = 0; for (final int number : factors) { sum = sum + number; } return sum; } }

Montag, 25. April 2011

Recap Threadsafety

7

public final class IDHandler { /**Local set of ids*/ private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" }; /**Getting the ids. * * @return the ids */ public String[] getIDs() { return ids; } }

Montag, 25. April 2011

Recap Threadsafety

8

public final class IDHandler { /**Local set of ids*/ private final List<String >ids = new ArrayList()<String>; public IDHandler() { ids.add("e634g"); ids.add("6fhg4"); ids.add("hd4fg"); ids.add("dgf53"); } /**Getting the ids. * * @return the ids */ public String[] getIDs() { return ids.toArray(new String[4]); } }

Montag, 25. April 2011

Threadsafe?

9

public class ReadyStater { private static boolean ready; private static int number; private static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield(); System.out.println(number); } } public static void main(String[] args) { new ReaderThread().start(); number = 42; ready = true; } }

Montag, 25. April 2011

Locking & Visibility

10

Montag, 25. April 2011

Difference regarding visibility

11

@NotThreadSafe public class MutableInteger { private int value; public int get() { return value; } public void set(int value) { this.value = value; } } @NotThreadSafe public class MutableInteger { private long value; public long get() { return value; } public void set(long value) { this.value = value; } }

Montag, 25. April 2011

Always Synchronize?

12

public class MutableInteger { private volatile long value; public long get() { return value; } public void set(long value) { this.value = value; } } public class MutableInteger { private long value; public synchronized long get() { return value; } public synchronized void set(long value) { this.value = value; } }

Montag, 25. April 2011

Volatile VS Synchronize

Volatile

• No Locking
• no synchronization
• visibility
• Usage:
• using if only one thread

updates value OR

• value is not participating

to invariants OR

• any other reason

13

Synchonize

• Locking
• synchronization
• visibility
• Usage:
• guarding multiple variables

(invariants)

• accessing within multiple

threads

Montag, 25. April 2011

Publishing and Escaping

• Publishing: Making a variable visible out of its current scope
• Escaping: Doing the same by accident

14

public final class IDHandler { /**Local set of ids*/ private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" }; /**Getting the ids. * * @return the ids */ public String[] getIDs() { return ids; } }

Montag, 25. April 2011

Escaping in Construtor

15

public class ThisEscape { public ThisEscape(EventSource source) { source.registerListener( new EventListener() { public void onEvent(Event e) { doSomething(e); } }); } }

Montag, 25. April 2011

Safe Construction Process

Do not allow to escape this during construction.

16

public class SafeListener { private final EventListener listener; private SafeListener() { listener = new EventListener() { public void onEvent(Event e) { doSomething(e); } }; } public static SafeListener newInstance(EventSource source) { SafeListener safe = new SafeListener(); source.registerListener(safe.listener); return safe; } }

Montag, 25. April 2011

Further techniques to ensure thread safety

• Confinement
• Immutability
• Publication
• Design
• Documentation

17

Montag, 25. April 2011

Confinement

• Thread Confinement

Binding an object to exactly one thread (which cannot be enforced by the JVM)

• Ad-hoc Confinement

Responsibility of maintaining thread safety is on implementation side (which is mostly fragile)

• Stack Confinement

Object bound to local variables only (happening in methods)

• ThreadLocal

18

private static ThreadLocal<Connection> connectionHolder = new ThreadLocal<Connection>() { public Connection initialValue() { return DriverManager.getConnection(DB_URL); } }; public static Connection getConnection() { return connectionHolder.get(); }

Montag, 25. April 2011

Immutability

Criterias for Immutability

• State cannot be modified after construction
• All fields are final
• properly constructed

Immutable Objects are always thread-safe!

19

public final class ThreeStooges { private final Set<String> stooges = new HashSet<String>(); public ThreeStooges() { stooges.add("Moe"); stooges.add("Larry"); stooges.add("Curly"); } public boolean isStooge(String name) { return stooges.contains(name); } }

Montag, 25. April 2011

Immutability and Publishing

20

@Immutable class OneValueCache { private final BigInteger lastNumber; private final BigInteger[] lastFactors; public OneValueCache(BigInteger i, BigInteger[] factors) { lastNumber = i; lastFactors = Arrays.copyOf(factors, factors.length); } public BigInteger[] getFactors(BigInteger i) { if (lastNumber == null || !lastNumber.equals(i)) return null; else return Arrays.copyOf(lastFactors, lastFactors.length); } } @ThreadSafe public class VolatileCachedFactorizer implements Servlet { private volatile OneValueCache cache = new OneValueCache(null, null); public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = cache.getFactors(i); if (factors == null) { factors = factor(i); cache = new OneValueCache(i, factors); } encodeIntoResponse(resp, factors); } }

Montag, 25. April 2011

Publishing of data

21

public Holder holder; public void initialize() { holder = new Holder(42); }

public class Holder { private int n; public Holder(int n) { this.n = n; } public void assertSanity() { if (n != n) throw new AssertionError("This statement is false."); } }

Montag, 25. April 2011

Safe publication idioms

• Initialize objects references from a static initializer
• Storing a reference into a volatile or AtomicReference
• Storing a reference into a final field of a proper constructed
• bject
• Guard the field that stores the object with a lock

22

Publication requirements

• Immutable objects: can be published with any mechanism
• Effectively immutable objects: must be published safely
• Mutable objects: must be published safely and be guared by

a lock

Montag, 25. April 2011

Useful policies for sharing objects

• Thread-confined: Object is owned exclusively by one thread
• Shared read-only: Object can be accessed concurrently by

multiple threads without additional synchronization

• Shared thread-safe: Synchronization on object is handled

internally.

• Guarded: Access to the object only possible when given

lock held including encluding other thread-safe objects

23

Montag, 25. April 2011

Designing Objects

Identify the variables that form the object's state;

• Identify the invariants that constrain the state variables;
• Establish a policy for managing concurrent access to the
• bject's state

24

Montag, 25. April 2011

States?

State of an object includes state of all fields

• primitive: simple
• n-primitives: n-tuple of the fields
• LinkedList: all linked nodes

Stay aware of pre-/postconditions and invariants: Synchronization-Policy

25

Montag, 25. April 2011

Thread-Safety redefined

Making a class thread-safe means ensuring that its invariants hold under concurrent access; this requires reasoning about its state

26

@ThreadSafe

public final class Counter { @GuardedBy("this") private long value = 0; public synchronized long getValue() { return value; } public synchronized long increment() { if (value == Long.MAX_VALUE) throw new IllegalStateException("counter overflow"); return ++value; } }

Montag, 25. April 2011

Instance Confinement

Guarding the access with wrapper

• Useful for debugging issues
• Class itself is thread-safe

(as are the states only accessible within this class)

• Example: Collections.synchronizedList representing

Decorator-Pattern for guarding collections Caution: wrapped object must not escape!

27

Montag, 25. April 2011

Monitor-Based Locking

Each object holds its own lock

28 @ThreadSafe public final class Counter { @GuardedBy("this") private long value = 0; public synchronized long getValue() { return value; } public synchronized long increment() { if (value == Long.MAX_VALUE) throw new IllegalStateException("counter overflow"); return ++value; } }

Montag, 25. April 2011

Public VS Private Locking

29

public class PrivateLock { private final Object myLock = new Object(); @GuardedBy("myLock") Widget widget; void someMethod() { synchronized(myLock) { // Access or modify the state of widget } } } public class PublicLock { @GuardedBy("this") Widget widget; void synchronized someMethod() { // Access or modify the state of widget } }

Montag, 25. April 2011

Delegation of Thread safety

30

public class MutablePoint { public int x, y; public MutablePoint() { x = 0; y = 0; } public MutablePoint(MutablePoint p) { this.x = p.x; this.y = p.y; } }

Montag, 25. April 2011

Delegation of Thread safety

31

@ThreadSafe public class MonitorVehicleTracker { @GuardedBy("this") private final Map<String, MutablePoint> locations; public MonitorVehicleTracker( Map<String, MutablePoint> locations) { this.locations = deepCopy(locations); } public synchronized Map<String, MutablePoint> getLocations() { return deepCopy(locations); } public synchronized MutablePoint getLocation(String id) { MutablePoint loc = locations.get(id); return loc == null ? null : new MutablePoint(loc); } public synchronized void setLocation(String id, int x, int y) { MutablePoint loc = locations.get(id); if (loc == null) throw new IllegalArgumentException("No such ID: " + id); loc.x = x; loc.y = y; } private static Map<String, MutablePoint> deepCopy( Map<String, MutablePoint> m) { Map<String, MutablePoint> result = new HashMap<String, MutablePoint>(); for (String id : m.keySet()) result.put(id, new MutablePoint(m.get(id))); return Collections.unmodifiableMap(result); } }

Montag, 25. April 2011

Better…

32

@Immutable public class Point { public final int x, y; public Point(int x, int y) { this.x = x; this.y = y; } }

Montag, 25. April 2011

Better

33

@ThreadSafe public class DelegatingVehicleTracker { private final ConcurrentMap<String, Point> locations; private final Map<String, Point> unmodifiableMap; public DelegatingVehicleTracker(Map<String, Point> points) { locations = new ConcurrentHashMap<String, Point>(points); unmodifiableMap = Collections.unmodifiableMap(locations); } public Map<String, Point> getLocations() { return unmodifiableMap; } public Point getLocation(String id) { return locations.get(id); } public void setLocation(String id, int x, int y) { if (locations.replace(id, new Point(x, y)) == null) throw new IllegalArgumentException( "invalid vehicle name: " + id); } }

Montag, 25. April 2011

Independence of state variables

34

public class VisualComponent { private final List<KeyListener> keyListeners = new CopyOnWriteArrayList<KeyListener>(); private final List<MouseListener> mouseListeners = new CopyOnWriteArrayList<MouseListener>(); public void addKeyListener(KeyListener listener) { keyListeners.add(listener); } public void addMouseListener(MouseListener listener) { mouseListeners.add(listener); } public void removeKeyListener(KeyListener listener) { keyListeners.remove(listener); } public void removeMouseListener(MouseListener listener) { mouseListeners.remove(listener); } }

public class NumberRange { // INVARIANT: lower <= upper private final AtomicInteger lower = new AtomicInteger (0); private final AtomicInteger upper = new AtomicInteger (0); public void setLower(int i) { if (i > upper.get()) throw new IllegalArgumentException( "can't set lower to " + i + " > upper"); lower.set(i); } public void setUpper(int i) { if (i < lower.get()) throw new IllegalArgumentException( "can't set upper to " + i + " < lower"); upper.set(i); } public boolean isInRange(int i) { return (i >= lower.get() && i <= upper.get()); } }

Montag, 25. April 2011

Extending existing sources

• Extending? Inheritence? Composition?
• Locking is based on functionality to be offered

Alien-Calls in foreign source code complicate things further.

35

Montag, 25. April 2011

What to lock?

36

public class ListHelper<E> { public List<E> list = Collections.synchronizedList(new ArrayList<E>()); ... public synchronized boolean putIfAbsent(E x) { boolean absent = !list.contains(x); if (absent) list.add(x); return absent; } } public class ListHelper<E> { public List<E> list = Collections.synchronizedList(new ArrayList<E>()); ... public boolean putIfAbsent(E x) { synchronized (list) { boolean absent = !list.contains(x); if (absent) list.add(x); return absent; } } }

Montag, 25. April 2011

• r…Composition…

37

@ThreadSafe public class ImprovedList<T> implements List<T> { private final List<T> list; public ImprovedList(List<T> list) { this.list = list; } public synchronized boolean putIfAbsent(T x) { boolean contains = list.contains(x); if (contains) list.add(x); return !contains; } public synchronized void clear() { list.clear(); } // ... similarly delegate other List methods }

Montag, 25. April 2011

Further Example: BlockingQueues

38

Void call()

Producer

BlockingQueue Void call()

Consumer

Item Item Item Item Item Item Item Item Pill

Montag, 25. April 2011

Documentation

• Each use of volatile / synchronize reflects a synchronization

strategy for mainting invariants/pre-/post-conditions

• JavaDoc (often not really usable regarding thread safety)
• Assumptions
• Annotations

39

Montag, 25. April 2011

Annotations

• Class-Intended:
• @Immutable
• @ThreadSafe
• @NotThreadSafe
• Field- and Method-Intended:
• @GuardedBy(lock)
• @GuardedBy(“this“)
• @GuardedBy(“(Class.)fieldName“)
• @GuardedBy(“methodName()“)

40

Montag, 25. April 2011

Outcome of first part

It's the mutable state, stupid.

• Make fields final unless they need to be mutable.
• Immutable objects are automatically thread-safe.
• Encapsulation makes it practical to manage the complexity.
• Guard each mutable variable with a lock.
• Guard all variables in an invariant with the same lock.
• Hold locks for the duration of compound actions.
• A program that accesses a mutable variable from multiple threads without synchronization is a

broken program.

• Don't rely on clever reasoning about why you don't need to synchronize.
• Include thread safety in the design processor explicitly document that your class is not thread-

safe.

• Document your synchronization policy.

41

Montag, 25. April 2011

Structuring Applications

42

Montag, 25. April 2011

Task Execution

43

class ThreadPerTaskWebServer { public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); } } }

Montag, 25. April 2011

How to start a thread

• Starting of threads should be

based on defined pool of threads

• Support of lifecycle as well

44

run() void sum(final int[][] numbers) run()

Runnable 1 Runnable 2 Thread 0

run()

Runnable 3

run()

Runnable 4

run()

Runnable 5

run()

Runnable 6

Montag, 25. April 2011

A summing example

45

Integer call() int[] sum(final int[][] numbers) Integer call()

Callable 2 Thread 0 Callable 1

ExecutorService.submit… FutureList.add… FutureList.get… Future.get… public void sum(final int[][] numbers) { final List<Future<Integer>> futures = new ArrayList<Future<Integer>>(); final ExecutorService exec = Executors.newCachedThreadPool(); for (int i = 0; i < numbers.length; i++) { final int index = i; Callable<Integer> summer = new Callable<Integer>() { public Integer call() throws Exception { return sum; } }; futures.add(exec.submit(summer)); // getting all futures for (int i = 0; i < futures.size(); i++) { try { returnVal[i] = futures.get(i).get(); } catch (final InterruptedException e) { e.printStackTrace(); returnVal[i] = Integer.MIN_VALUE; } catch (final ExecutionException e) { e.printStackTrace(); returnVal[i] = Integer.MIN_VALUE; } } }

Montag, 25. April 2011

Advantages of bound execution

• Defined Thread lifecycle
• Adaptive and independent resource management
• Stability

46

Montag, 25. April 2011

Hazards while not using Executors

47

run() void sum(final int[][] numbers) run()

Runnable 1 Runnable 2 Thread 0

run()

Runnable 3

run()

Runnable 4

run()

Runnable 5

run()

Runnable 6

Montag, 25. April 2011

Further Hazards

• Poor Exception-Handling
• Overwhelming the resources
• Hard support for individual multithreaded tasks
• Unequal tasks
• Scheduling
• Cancelation
• No lifecycle for execution itself

48

Montag, 25. April 2011

Executors

49

ExecutorService

Thread Thread Thread Thread Thread ThreadFactory Queue

Fixed Pool Size

ExecutorService

Thread ThreadFactory Queue

Pool Size = 1

ExecutorService

Thread Thread ThreadFactory Queue Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread …

No Pool Size Additional Executors:

• ScheduledExecutor,
• CompletionServiceExecutor

…

Montag, 25. April 2011

Stopping Threads

• ...

50

Montag, 25. April 2011

There is no safe way to stop a thread

51

public class PrimeGenerator implements Runnable { @GuardedBy("this") private final List<BigInteger> primes = new ArrayList<BigInteger>(); private volatile boolean cancelled; public void run() { BigInteger p = BigInteger.ONE; while (!cancelled ) { p = p.nextProbablePrime(); synchronized (this) { primes.add(p); } } } public void cancel() { Thread.stop(); } public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes); } }

Montag, 25. April 2011

Flag?

52

@ThreadSafe public class PrimeGenerator implements Runnable { @GuardedBy("this") private final List<BigInteger> primes = new ArrayList<BigInteger>(); private volatile boolean cancelled; public void run() { BigInteger p = BigInteger.ONE; while (!cancelled ) { p = p.nextProbablePrime(); synchronized (this) { primes.add(p); } } } public void cancel() { cancelled = true; } public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes); } }

Montag, 25. April 2011

Producer / Consumer ?

53

class BrokenPrimeProducer extends Thread { private final BlockingQueue<BigInteger> queue; private volatile boolean cancelled = false; BrokenPrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue; } public void run() { try { BigInteger p = BigInteger.ONE; while (!cancelled) queue.put(p = p.nextProbablePrime()); } catch (InterruptedException consumed) { } } public void cancel() { cancelled = true; } } void consumePrimes() throws InterruptedException { BlockingQueue<BigInteger> primes = ...; BrokenPrimeProducer producer = new BrokenPrimeProducer(primes); producer.start(); try { while (needMorePrimes()) consume(primes.take()); } finally { producer.cancel(); } }

Montag, 25. April 2011

Interruption?

54

class PrimeProducer extends Thread { private final BlockingQueue<BigInteger> queue; PrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue; } public void run() { try { BigInteger p = BigInteger.ONE; while (!Thread.currentThread().isInterrupted()) queue.put(p = p.nextProbablePrime()); } catch (InterruptedException consumed) { /* Allow thread to exit */ } } public void cancel() { interrupt(); } }

Montag, 25. April 2011

Interruption

• Necessary Interruption Policy (what should happen if thread

gets interrupted)

• freeing connection, resources
• unblocking
• keeping invariants
• cleaning up
• Tasks own not the thread (with the usage of thread pools),

policy mostly based on „get out of the way“

55

Montag, 25. April 2011

Interruption

What to do if InterruptedException is catched?

• Propagate the Interruption
• Restore the internal state

56

public Task getNextTask(BlockingQueue<Taskgt; queue) { boolean interrupted = false; try { while (true) { try { return queue.take(); } catch (InterruptedException e) { interrupted = true; // fall through and retry } } } finally { if (interrupted) Thread.currentThread().interrupt(); } }

Montag, 25. April 2011

Interruption of other threads

57

private static final ScheduledExecutorService cancelExec = ...; public static void timedRun(Runnable r, long timeout, TimeUnit unit) { final Thread taskThread = Thread.currentThread(); cancelExec.schedule(new Runnable() { public void run() { taskThread.interrupt(); } }, timeout, unit); r.run(); }

Montag, 25. April 2011

Future.cancel

58

public static void timedRun(Runnable r, long timeout, TimeUnit unit) throws InterruptedException { Future<?> task = taskExec.submit(r); try { task.get(timeout, unit); } catch (TimeoutException e) { // task will be cancelled below } catch (ExecutionException e) { // exception thrown in task; rethrow throw launderThrowable(e.getCause()); } finally { // Harmless if task already completed task.cancel(true); // interrupt if running } }

Montag, 25. April 2011

Producer / Consumer, Poison Pill

59

Void call()

Producer

BlockingQueue Void call()

Consumer

Item Item Item Item Item Item Item Item Pill

Montag, 25. April 2011

Executor Shutdown?

60 public class LogService { private final ExecutorService exec = newSingleThreadExecutor(); ... public void start() { } public void stop() throws InterruptedException { try { exec.shutdown(); exec.awaitTermination(TIMEOUT, UNIT); } finally { writer.close(); } } public void log(String msg) { try { exec.execute(new WriteTask(msg)); } catch (RejectedExecutionException ignored) { } } }

Montag, 25. April 2011

Exceptions ?

61

public void log(String msg) throws InterruptedException { if (!shutdownRequested) queue.put(msg); else throw new IllegalStateException("logger is shut down"); }

Montag, 25. April 2011

Always be aware of the exceptions

• Proactive reaction regarding exceptions
• Propagate the Interruption
• Restore the internal state
• UncaughtExceptionHandler for detection of uncaught

exceptions

62

Montag, 25. April 2011

Other „artifacts“

• Finalizers
• Daemon Threads
• JVM Shutdowns

63

Montag, 25. April 2011

The last one…

64

public interface Runnable2 { public void run() throws Exception; }

Montag, 25. April 2011