Storage April 2, 2018 1 IO + Buffering def Select(predicate, - - PowerPoint PPT Presentation

▶
storage
SMART_READER_LITE
LIVE PREVIEW

Storage April 2, 2018 1 IO + Buffering def Select(predicate, - - PowerPoint PPT Presentation

Apr 01, 2023 229 likes •414 views

Storage April 2, 2018 1 IO + Buffering def Select(predicate, source) while(source.hasMoreTuples) in_buffer = source.fetch() while(in_buffer.hasMoreTuples) tuple = in_buffer.readTuple() if(predicate(tuple)) out_buffer.output(tuple)

slide-1
SLIDE 1

Storage

April 2, 2018

1

slide-2
SLIDE 2

IO + Buffering

def Select(predicate, source) while(source.hasMoreTuples) in_buffer = source.fetch() while(in_buffer.hasMoreTuples) tuple = in_buffer.readTuple() if(predicate(tuple))

  • ut_buffer.output(tuple)

if(out_buffer.isFull)

  • ut_buffer.flush()

2

slide-3
SLIDE 3

IO + Buffering

Generalize & Standardize! Have a component that handles buffering!

3

slide-4
SLIDE 4

The Buffer Manager

Query Interpretation and Optimization Relational Operators Files/Data Accessors Buffer Management Disk Management Database

API Allocate a page Deallocate a page Read from a page Write to a page

4

slide-5
SLIDE 5

The Buffer Manager

Higher levels of the DB Disk Page Free Frame

Pages allocated to frames as per page replacement policy

5

slide-6
SLIDE 6

Pinned Pages

  • Pinning a page indicates that it is being used.
  • The requestor must unpin the page when done.
  • The requestor must also indicate whether the

page has been modified (with a ‘dirty’ bit)

  • Dirty pages must be written to disk
  • Pages may be requested multiple times
  • Use a pin count (reference count) to keep track.
  • Concurrency Control/Recovery may require other
  • perations when replacing a frame.

6

slide-7
SLIDE 7

Buffer Replacement

  • Frames are chosen for replacement by a

buffer replacement policy.

  • (e.g., LRU, MRU, Clock)
  • Policy can have a big impact!
  • Depends on the access pattern.
  • What is a worst-case scenario for LRU?

7

slide-8
SLIDE 8

Buffer Replacement

  • Frames are chosen for replacement by a

buffer replacement policy.

  • (e.g., LRU, MRU, Clock)
  • Policy can have a big impact!
  • Depends on the access pattern.
  • What is a worst-case scenario for LRU?

7

Hmmm… this sounds awfully familiar…

slide-9
SLIDE 9

8

Hey… Oliver! This sounds a lot like virtual memory!

slide-10
SLIDE 10

Buffer Managers vs Virtual Memory

  • Not a huge difference
  • Many lightweight DBs use

VMem as a buffer manager!

  • Reasons to implement an explicit buffer manager:
  • Control when and how paging happens.
  • e.g., better/more efficient prefetching.
  • Control what gets paged in/out.
  • e.g., better knowledge of data access patterns.
  • Integrate additional memory layers (e.g., Network)

9

slide-11
SLIDE 11

Example-OS Paging

image credit: openclipart.org

10

slide-12
SLIDE 12

Example-OS Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’

10

slide-13
SLIDE 13

Example-OS Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’ Read

10

slide-14
SLIDE 14

Example-OS Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’ Read

10

slide-15
SLIDE 15

Example-OS Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’ Read Aha, a Sequential Read!

10

slide-16
SLIDE 16

Example-DB Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’ Read

11

slide-17
SLIDE 17

Example-DB Paging

image credit: openclipart.org

Index: ‘Trees with ids in this range’ Read Read in precisely what you need.

11