Distributed Shared Memory (DSM) Robert Gasparyan, Angela Gong, - - PowerPoint PPT Presentation

▶
distributed shared memory dsm
SMART_READER_LITE
LIVE PREVIEW

Distributed Shared Memory (DSM) Robert Gasparyan, Angela Gong, - - PowerPoint PPT Presentation

Jul 07, 2023 44 likes •297 views

Distributed Shared Memory (DSM) Robert Gasparyan, Angela Gong, Judson Wilson CS 240, Spring 2015 What is DSM? Physically separate memory addressed as one shared address space Memory shared on page-by-page basis CPU CPU CPU CPU

slide-1
SLIDE 1

Distributed Shared Memory (DSM)

Robert Gasparyan, Angela Gong, Judson Wilson

CS 240, Spring 2015

slide-2
SLIDE 2

What is DSM?

  • Physically separate memory addressed as one

shared address space

  • Memory shared on page-by-page basis

Virtual Memory

Network

Memory Memory Memory Memory CPU CPU CPU CPU

slide-3
SLIDE 3

Problem

Thread

CPU

Thread

CPU

Thread

CPU

Thread

CPU

Result

? ? ? Thread

  • Make use of multiple machines
  • Manage dependencies
slide-4
SLIDE 4

DSM: Simple Interface

Shared Memory Thread A

CPU

Thread B

CPU

slide-5
SLIDE 5

Consistency Model

Local Memory

R/W

Thread A

CPU CPU

Thread B

R/W

Local Memory Shared Region Shared Region

slide-6
SLIDE 6

Release Consistency

  • Critical sections protected by same lock

execute sequentially

  • All changes from previously protected regions

guaranteed to be visible

  • Saves network traffic because don’t need to

synchronize until lock is released

slide-7
SLIDE 7

Design and Implementation

slide-8
SLIDE 8

Deployment of Binaries

Server

Daemon Daemon Daemon Daemon

A A A A Future capability:

slide-9
SLIDE 9

Master: Locks and Page Info

Master

Worker 0 Worker 1 Worker 2 Worker 3

v: 28

  • : 3
slide-10
SLIDE 10

Lock Requests

  • Spawn thread
  • Wait on
  • Reply after acquired

Similar for unlock

Locks

Master

Worker

slide-11
SLIDE 11
  • Background thread that distributes page data

upon request

  • All workers have a page server

Page Servers: Page Contents

Worker Page Server Worker Page Server Worker Page Server Worker Page Server

slide-12
SLIDE 12

Implicitly locked memory can be smaller than a

  • page. Used diff-patches to merge modifications

Patches

+

  • Release

Acquire Modify

slide-13
SLIDE 13

Transparent Interface

Fault handlers, lock/unlock:

  • starts (lazy)
  • to protect pages
  • fault to catch first read/write
  • 1st: Upgrade to

access → get latest version

  • 2nd: Upgrade to

→ mark modified

  • does
  • Pull versions, merge modified pages, create new version
slide-14
SLIDE 14

Transparent Shared Memory

Across Processes Across Network (DSM)

slide-15
SLIDE 15

Benchmarks

slide-16
SLIDE 16

Single Machine Using DSM

Benchmark Setup

  • Compare performance of single machine

versus DSM.

slide-17
SLIDE 17
  • 9

7

  • 1

6

  • 5

2 6

  • 4

1

Matrix Multiplication

3 3 3 1

  • 1

3 2

  • 1

4 2 2 1

=

slide-18
SLIDE 18

Matrix Multiplication

1/ms

0.001 0.005 0.000 0.0005 0.0003 0.0000

Single Machine Using DSM

Number of Cores 0 5 10 15 20 Number of Cores 0 5 10

1/ms

slide-19
SLIDE 19

Word Count

Donald Ervin Knuth: The Art of Computer Programming: Generating all Combinations and Partitions

slide-20
SLIDE 20

Word Count

0 5 10 15 20 Number of Cores

1/ms

0.0012 0.0062 0.0000

0 5 10

0.0007 0.0004 0.0000

Number of Cores

Single Machine Using DSM

1/ms

slide-21
SLIDE 21

Demo!

slide-22
SLIDE 22

Conclusion

  • We made transparent DSM!
  • Focus: Correctness first, then scalability
  • Tedious:

○ C data structures ○ Message passing / handling

slide-23
SLIDE 23

Questions?

slide-24
SLIDE 24

Bonus: Correctness Test

Private Buckets Worker 0 Worker 1 Worker 2

+1 +2

Shared Buckets Invariant: Sum Shared = Sum Private

slide-25
SLIDE 25

Bonus: Correctness Test

Nested Increment Serial Transfer 1 Count Nested Transfer All Counts