COMPLEXITY OF MULTI-VALUED REGISTER SIMULATIONS: A RETROSPECTIVE - - PowerPoint PPT Presentation

COMPLEXITY OF MULTI-VALUED REGISTER SIMULATIONS: A RETROSPECTIVE

Jennifer L. Welch Texas A&M University OPODIS December 2018

The Problem

• Use a collection of shared read-write

registers with certain properties to simulate a shared read-write register with stronger properties in a wait-free manner

1

Complexity of Multi-Valued Register Simulations OPODIS 2018

Register Properties

• consistency condition in presence of

concurrent operations:

• safe
• regular
• atomic
• number of storable values
• number of readers supported
• number of writers supported

2

Complexity of Multi-Valued Register Simulations OPODIS 2018

Register Simulation

3

Complexity of Multi-Valued Register Simulations OPODIS 2018

• 1. ...
• 2. ...
• 3. ...
• 4. ...
• 5. ...
• 1. ...
• 2. ...
• 3. ...
• 4. ...
• 5. ...

READ WRITE read read read write write write

Wait-Free

• Each simulated operation terminates in a finite

number of steps by invoking process regardless

• f behavior of other processes.
• Other processes can be arbitrarily fast, or

arbitrarily slow, or crash…

4

Complexity of Multi-Valued Register Simulations OPODIS 2018

Complexity Measures (Worst-Case)

• Number of base

registers (M)

• Number of steps by

READ algorithm (R)

• Number of steps by

WRITE algorithm (W)

OPODIS 2018 Complexity of Multi-Valued Register Simulations

5

• 1. public class HelloWorld {

2. public static void main(String[] args) { 3. System.out.println("Hello World!"); 4. }

• 5. }

Background and Motivation: Multiprogramming Computers

• Multi-programming environments developed in early

1970’s

• Need to control/synchronize concurrent processes when

accessing shared resources

6

Complexity of Multi-Valued Register Simulations OPODIS 2018

HP 3000. Photo credit: http://www.hpmuseum.net/exhibit.php?class=3&cat=32

Background and Motivation: Readers/Writers Problem

• [Courtois, Heymans, Parnas 1971]: motivated by 1960’s

work on mutual exclusion but distinguish between readers and writers (“readers/writers problem”):

• writers must have exclusive access to the resource
• readers can share access to the resource

Algorithms use semaphores to provide exclusive access

• [Brinch Hansen 1972]: simplified one of the algorithms to

use “conditional critical regions” instead of semaphores

7

Complexity of Multi-Valued Register Simulations OPODIS 2018

Background and Motivation: Avoid Mutual Exclusion

• Drawbacks of using mutual exclusion for the

readers/writers problem:

• writer must wait until all current reads are done but what if writer

has higher priority?

• concurrent reading and writing may be needed to implement mutual

exclusion

• creates a serialization, decreasing parallelism
• not fault-tolerant
• [Lamport 1977]: Assumes hardware solves

readers/writers problem for basic units of data and proposes algorithm for solving the problem for a composite data item

• [Peterson 1983]: Improves on Lamport’s solution by not

locking out readers and using bounded-size base units

8

Complexity of Multi-Valued Register Simulations OPODIS 2018

Background and Motivation: The Golden Age

• [Lamport 1986]: Landmark 2-part paper on register

simulations

• formalized notions of safe, regular and atomic registers
• proposed formal model
• provided numerous algorithms
• Initiated a spate of work to understand what kind of

registers can be simulated out of what other kinds of registers

OPODIS 2018 Complexity of Multi-Valued Register Simulations

9

Background and Motivation: Resurgence

• Then a pause as community’s interest shifted to shared

memory algorithms using stronger objects, such as T&S, F&A, CAS, etc.and to message-passing implementations

• f registers (“distributed storage”)
• Recent papers have shown connections to register

simulations, e.g.:

• how to provide concurrent write ability to objects of any type,

including those stronger than registers [Aghazadeh, Golab, Woelfel 2014]

• lower bounds on space needed to share data in various distributed

storage applications [Berger, Keidar, Spiegelman 2018]

• Rekindled interest in register simulations

OPODIS 2018 Complexity of Multi-Valued Register Simulations

10

Focus of This Talk: Binary Base Registers

• Base registers are binary: can store one of two

values (i.e., single bit)

• Simulated register should store one of k > 2

values (i.e., multi-bit)

• Simplification yet still gets at the heart of

difficulties in simulating “large” registers using “small” registers

• Literature includes generalizations for case when

base registers can store b > 2 values and k > b

• Assume k is a power of 2 and logs are base 2

11

Complexity of Multi-Valued Register Simulations OPODIS 2018

Focus of This Talk: Single-Writer Registers

• Only one process can write each base register
• Only one process can write the simulated register
• Historically, safe and regular were only defined

for single-writer registers

• Even for atomic, most research on multi-valued

simulations has been for single-writer registers

12

Complexity of Multi-Valued Register Simulations OPODIS 2018

Focus of This Talk: Only Change Number of Values

• Consistency condition is the same for base registers as

for simulated register

• safe multi-valued out of safe binary
• regular multi-valued out of regular binary
• atomic multi-valued out of atomic binary
• Number of readers is the same for base registers as for

simulated register

• Won’t cover numerous results for building regular out of

safe, atomic out of regular, many-reader out of single- reader, etc.

OPODIS 2018 Complexity of Multi-Valued Register Simulations

13

OPODIS 2018 Complexity of Multi-Valued Register Simulations

14

Image credit: http://coull.com/our-blog/coull-putting-brand-safety-first/

Safe Register

[Lamport 1986]

• A read that does not overlap a write returns the

value of the latest preceding write

• A read that overlaps a write can return any value
• x and y can be anything (e.g., 3, 4, …)

15

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(1) WRITE(2) READ(1) READ(x) READ(y)

Binary Representation Algorithm

[Lamport 1986]

16

Complexity of Multi-Valued Register Simulations OPODIS 2018

READ: read 0, read 1, read 1, read 0, read 1 and RETURN(13) WRITE(19): write 1, write 0, write 0, write 1, write 1

log k base registers MSB LSB MSB LSB

Efficiency of Binary Representation Algorithm

• M: log k base registers
• optimal since k different values must be stored
• R: log k steps by Reader
• optimal since k different values must be read

[Chaudhuri, Welch 1994]

• W: log k steps by Writer
• not optimal — see next algorithm…

17

Complexity of Multi-Valued Register Simulations OPODIS 2018

Hypercube Algorithm

[Chaudhuri, Welch 1994]

• based on coloring vertices of a (k−1)-dimensional

hypercube with k colors

• Example for k = 4:

18

Complexity of Multi-Valued Register Simulations OPODIS 2018

000 001 010 011 100 101 110 111

Hypercube Algorithm

[Chaudhuri, Welch 1994]

• each bit in the vertex labels is a

base register

• M = k−1
• each vertex color is a value of the

simulated register: {R, G, B, Y}

• READ: read all base registers, determine vertex in (3-

dimensional) hypercube, and return color of vertex

• R = k − 1
• WRITE(v): determine which base register to flip in order

to go from current vertex to vertex with color v

• W = 1

19

Complexity of Multi-Valued Register Simulations OPODIS 2018 00 00 1 01 011 10 10 1 110 111

1

Independent Bounds for Safe Registers

Complexity Measure Tight Value R (number of steps in READ alg) log k W (number of steps in WRITE alg) 1 M (number of base register) log k

20

Complexity of Multi-Valued Register Simulations OPODIS 2018

Safe Algorithms with Optimal Writes

[Chaudhuri, Welch 1994]

• Can we improve on hypercube algorithm

while keeping W = 1?

• No!
• If W = 1, then R ≥ k−1.
• If W = 1, then M ≥ k−1.

21

Complexity of Multi-Valued Register Simulations OPODIS 2018

Safe Algorithms with Optimal Registers

[Chaudhuri, Welch 1994]

• Can we improve on binary representation

algorithm while keeping M = log k?

• No!
• If M = log k, then R ≥ log k.
• If M = log k, then W ≥ log k.

22

Complexity of Multi-Valued Register Simulations OPODIS 2018

Safe Algorithms with Optimal Reads

[Chaudhuri, Frank, Welch in prep]

• Can we improve on binary representation

algorithm while keeping R = log k?

• No!
• If R = log k, then W ≥ log k.
• And M already has optimal value, log k

23

Complexity of Multi-Valued Register Simulations OPODIS 2018

Trade-Off Bounds for Safe Registers

Optimal Measure Other Measure Tight Value R = log k W log k “ M log k W = 1 R k−1 “ M k−1 M = log k R log k “ W log k

24

Complexity of Multi-Valued Register Simulations OPODIS 2018

OPODIS 2018 Complexity of Multi-Valued Register Simulations

25

Image credit: https://www.thetruthaboutcars.com/2008/06/in-defense-of-regular-gas/

Regular Register

[Lamport 1986]

• A read that does not overlap a write returns the value of the

latest preceding write

• A read that overlaps a write can return any value written by an
• verlapping write or by latest preceding write
• x can be 1 or 2
• y can be 1 or 2

26

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(1) WRITE(2) READ(1) READ(x) READ(y)

Difficulties

• Even if base registers are regular, neither binary

representation algorithm nor hypercube algorithm simulates a regular register

• A slow read that overlaps fast writes (or vice versa) can
• bserve a combination of bits that represents neither old

nor new value

27

Complexity of Multi-Valued Register Simulations OPODIS 2018

Unary Representation Algorithm

[Lamport 1986]

28

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(2): write 1 into base register 2, write 0 into base register 1, write 0 into base register 0

k = 5 base registers 4 3 2 1 0 4 3 2 1 0

Unary Representation Algorithm

[Lamport 1986]

29

Complexity of Multi-Valued Register Simulations OPODIS 2018

READ: read 0 from base register 0, read 0 from base register 1, read 1 from base register 2, RETURN(2)

k = 5 base registers 4 3 2 1 0

Note the “going in opposite directions” idea.

k–1 = 4

Efficiency of (Optimized) Unary Representation Algorithm

• R: k−1 steps by Reader
• this can be improved – see next algorithm…
• W: k−1 steps by Writer
• this can be improved – see next algorithm…
• M: k−1 base registers
• optimal? Stay tuned

OPODIS 2018 Complexity of Multi-Valued Register Simulations

30

Tree Algorithm

[Chaudhuri, Welch 1994]

• k−1 binary (regular) registers are conceptually arranged

as the internal nodes in a binary tree, with the values being the leaves

31

Complexity of Multi-Valued Register Simulations OPODIS 2018

1 2 3

Tree Algorithm

[Chaudhuri, Welch 1994]

• READ: start at root, follow

indicated path down to a leaf, return that value

• WRITE(v): start at leaf for v,

go up path to root, changing base registers as needed.

• Note idea of “going in opposite

directions”

32

Complexity of Multi-Valued Register Simulations OPODIS 2018

1 2 3 1 2 3

Efficiency of Tree Algorithm

• R: log k steps by Reader
• optimal since k different values must be read
• W: log k steps by Writer
• can be improved – see next algorithm…
• M: k – 1 base registers
• optimal? Stay tuned

33

Complexity of Multi-Valued Register Simulations OPODIS 2018

Clique Algorithm

[Chaudhuri, Kosa, Welch 2000]

• Each edge in a (conceptual) k-vertex clique is

labeled with a base regular register

• M = k(k−1)/2

34

Complexity of Multi-Valued Register Simulations OPODIS 2018

1 2 3

Clique Algorithm

[Chaudhuri, Kosa, Welch 2000]

• WRITE(v): Flip base register
• n edge between current

vertex and vertex labeled v

• W = 1
• Invariant: Either (1) no vertex

has odd number of incident edges labeled 1, or (2) exactly two vertices do, initial value 0 and some other value v.

• READ: read all base

registers; if case (1) then return 0 else return v

• R = k(k − 1)/2

35

Complexity of Multi-Valued Register Simulations OPODIS 2018

1 2 3 1 2 3

Efficiency of Clique Algorithm

• R: k(k–1)/2 steps by Reader
• not optimal
• W: 1 step by Writer
• optimal
• M: k(k–1)/2 base registers
• not optimal

OPODIS 2018 Complexity of Multi-Valued Register Simulations

36

Reducing Number of Base Registers

• The unary representation, tree, and clique algorithms all

use Ω(k) base registers to simulate a k-valued register

• Can we do better?
• Yes! [Lamport 1986]
• Complex composition of 5 constructions, resulting in an

algorithm using n(3(log k) + 68) = Θ(n log k) base registers, where n is number of Readers

• Uses a new idea: have Readers write to base registers

OPODIS 2018 Complexity of Multi-Valued Register Simulations

37

Invisible vs. Visible Reads

• Invisible: READ algorithm does not write to any base

registers

• Visible: READ algorithm writes to some base register(s)

OPODIS 2018 Complexity of Multi-Valued Register Simulations

38

Number of Base Registers: Lower Bounds

• log k + 1 [Chaudhuri, Welch 1994]
• 2(log k) – log log k – 2 [Chaudhuri, Welch 1994]
• For a restricted class of algorithms:

[Berger, Keidar, Spiegelman 2018]

• invisible reads: expression proportional to k–1
• visible reads: expression proportional to n, number of

Readers

• General case: [Chen, Wei in prep]
• invisible reads: k–1
• visible reads: n + log k, where n is number of Readers

OPODIS 2018 Complexity of Multi-Valued Register Simulations

39

Independent Bounds for Regular Registers

Complexity Measure Tight Value R (number of steps in READ alg) log k W (number of steps in WRITE alg) 1 M (number of base registers) with invisible reads k – 1 M (number of base registers) with visible reads (n = number of Readers) between n + log k and n(3(log k) + 68)

40

Complexity of Multi-Valued Register Simulations OPODIS 2018

Regular Algorithms with Optimal Reads

• Can we improve on tree algorithm while keeping

R = log k? No!

• If R = O(log k), then W = Ω(log k). [Chen, Wei 2016]
• And in fact, if R = log k, then W ≥ log k.

[Chaudhuri, Frank, Welch in prep]

• If R = log k, then reads must be invisible,

so M ≥ k − 1. [Chen, Wei in prep]

41

Complexity of Multi-Valued Register Simulations OPODIS 2018

Regular Algorithms with Optimal Writes

[Chaudhuri, Welch 1994]

• Can we improve on clique algorithm while

keeping W = 1?

• Maybe…
• If W = 1, then R ≥ k−1.
• If W = 1, then M ≥ k.
• There are larger lower bounds on M for restricted

classes of algorithms [Chaudhuri, Kosa, Welch 2000]

42

Complexity of Multi-Valued Register Simulations OPODIS 2018

Trade-Off Bounds for Regular Registers

Optimal Measure Other Measure Tight Value R = log k W log k “ M k − 1 W = 1 R between k – 1 and k(k − 1)/2 “ M between k and k(k − 1)/2 M = k – 1 (invisible reads) R log k “ W between 1 and log k

43

Complexity of Multi-Valued Register Simulations OPODIS 2018

OPODIS 2018 Complexity of Multi-Valued Register Simulations

44

Atomic Register

[Lamport 1986]

• There exists a total order of operations that respects the
• rder of non-overlapping operations in which every read

returns the value of the latest preceding write

• x can be 1 or 2, y can be 1 or 2, but if x = 2 then y = 2

45

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(1) WRITE(2) READ(1) READ(x) READ(y)

Difficulties

• Even if base registers are atomic, neither unary

representation algorithm nor tree algorithm simulates an atomic register

• Both are subject to new-old inversion, e.g.,

46

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(1) WRITE(2) READ(2) READ(1)

Modified Unary Represen- tation Algorithm [Vidyasankar 1988]

47

Complexity of Multi-Valued Register Simulations OPODIS 2018

WRITE(2): write 1 into base register 2, write 0 into base register 1, write 0 into base register 0

• M = k

4 3 2 1 0 4 3 2 1 0

• W = k

Modified Unary Represen- tation Algorithm [Vidyasankar 1988]

48

Complexity of Multi-Valued Register Simulations OPODIS 2018

READ:

• read base registers 0, 1, 2, etc. (right-to-left) until
• btaining a 1, say in base register j.
• Then read base registers j−1, j−2, …, 0 (left-to-right).
• Return smallest index from which a 1 was obtained.
• M = k

4 3 2 1 0

• M = k–1

Efficiency of (Optimized) Modified Unary Representation Algorithm

• R: 2k–3 steps by Reader
• not optimal –– see modified tree algorithm coming up
• W: k–1 steps by Writer
• not optimal –– see clique algorithm
• M: k–1 base registers
• optimal? Stay tuned

OPODIS 2018 Complexity of Multi-Valued Register Simulations

49

Clique Algorithm Revisited

[Chaudhuri, Kosa, Welch 2000]

• If base registers are atomic, then clique

algorithm simulates an atomic register. Recall:

• W = 1
• R = k(k−1)/2
• M = k(k−1)/2

50

Complexity of Multi-Valued Register Simulations OPODIS 2018

1 2 3

Efficiency of Clique Algorithm

• R: k(k–1)/2 steps by Reader
• not optimal
• W: 1 step by Writer
• optimal
• M: k(k–1)/2 base registers
• not optimal

OPODIS 2018 Complexity of Multi-Valued Register Simulations

51

Modified Tree Algorithm

[Chen, Wei 2016]

• Modification of tree algorithm to simulate an atomic register
• 2k2 − 1 binary (atomic) base registers are conceptually

arranged as the internal nodes in a binary tree with k2 leaves

• M = 2k2 – 1
• Leaves are labeled, left to right, as 0, 0, 0, 1, 0, 2, 0, 3, …,1, 0,

1, 1, 1, 2, 1, 3, …, i.e., all pairs of values

• READ: same as in original tree algorithm
• R = 1 + 2(log k)
• WRITE: find old-new pair, write old value as in original tree

algorithm, then flip base register just above old-new pair

• W = 2 + 2(log k)

52

Complexity of Multi-Valued Register Simulations OPODIS 2018

Modified Tree Algorithm: READ(3)

[Chen, Wei 2016]

53

Complexity of Multi-Valued Register Simulations OPODIS 2018

…

1 2 3 3 3 1 3 2 3 3

… …

…

Modified Tree Algorithm: WRITE(2)

[Chen, Wei 2016]

54

Complexity of Multi-Valued Register Simulations OPODIS 2018

…

1 2 3 3 1 3 2 3 3

… …

3 2 …

Reducing Number of Base Registers

• The modified unary representation, modified tree, and

clique algorithms all use Ω(k) base registers to simulate a k-valued register

• Can we do better?
• Yes, using visible reads, as in the regular case!
• [Peterson 1983]

OPODIS 2018 Complexity of Multi-Valued Register Simulations

55

Idea of Peterson’s Algorithm

OPODIS 2018 Complexity of Multi-Valued Register Simulations

56

written by Writer, read by all Readers Buff1 Buff2 Wflag Switch Reading[i] Writing[i] Copybuff[i] written by Writer, read by Reader i, 1 ≤ i ≤ n X n k=16 written by Reader i, read by Writer

Improvements on Peterson’s Algorithm

• Θ(n log k) registers [Peterson 1983]
• R = Θ(log k) and W = Θ(n log k)
• Θ(n log k) registers and supports multiple Writers

[Aghazadeh, Golab, Woelfel 2014]

• R = Θ(log k) and W = Θ(log k) as long as n = O(1)
• Θ(n log k) registers and fast Writes for any number of

readers [Chen, Wei 2016]

• R = Θ(log k) and W = Θ(log k)
• 5n + 2(log k) + 1 registers [Chen, Wei in prep]
• R = Θ((log k)2) and W = Θ(n + log k)

OPODIS 2018 Complexity of Multi-Valued Register Simulations

57

Number of Base Registers: Lower Bounds

• [Peterson 1983] applies to “buffer-based” algorithms,

where a buffer holds a copy of the simulated value: n+2 buffers are necessary (and sufficient)

• and same lower bounds as in regular

OPODIS 2018 Complexity of Multi-Valued Register Simulations

58

Independent Bounds for Atomic Registers

59

Complexity of Multi-Valued Register Simulations OPODIS 2018

Complexity Measure Tight Value R between log k and 1 + 2(log k) W 1 M (invisible reads) k–1 M (visible reads) between n + log k and 5n + 2(log k) + 1

https://www.vectorsecurity.com/blog/life-according-to-the-jetsons-technology-that-exists-today

60

Complexity of Multi-Valued Register Simulations OPODIS 2018

Future Work

• close gaps, especially for
• safety tradeoff results: when W = 1 and k is not a power of 2
• regular tradeoff results: when W/M is optimal, what are best values

for the other measures?

• atomic independent bounds on R and M
• atomic tradeoff results: when R/W/M is optimal, what are best

values for the other measures?

• asymptotic versus non-asymptotic bounds
• multiple Writers
• faults (in addition to crashes)
• allocate-on-use register complexity (cf. [Aspnes, Haeupler, Tong,

Woelfel 2018])

• randomized algorithms

61

Complexity of Multi-Valued Register Simulations OPODIS 2018

Acknowledgements

• Thanks to Soma Chaudhuri, Faith Ellen, Reggie Frank,

Sasha Spiegelman, and Hao Wei for helpful conversations

OPODIS 2018 Complexity of Multi-Valued Register Simulations

62

Thank you! Questions?

63

Complexity of Multi-Valued Register Simulations OPODIS 2018

OPODIS 2018 Complexity of Multi-Valued Register Simulations

64

R/2/1 R/16/1 (Constr. 4) X 16 R/2/1 A/2/1 (Constr. 5) X 4 S/2/1 X log k S/k/1 (Constr. 2) X 3 A/k/1 [Peterson 1983] X n R/k/n (Constr. 1) Simulating regular k-valued register using regular binary registers [Lamport 1986] Notation: consistency condition/ number of values/ number of Readers Constructions 1, 2, 4, and 5 from [Lamport 1986] Construction 5 and Peterson’s algorithm both require the Reader to write to a base register!