Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy - - PowerPoint PPT Presentation

Cache Memory

Raul Queiroz Feitosa

Content

• Memory Hierarchy
• Principle of Locality
• Some Definitions
• Cache Architectures
• Fully Associative
• Direct Mapping
• Set Associative
• Replacement Policy
• Main Memory Update Policy

Memory Hierarchy

• Tradeoff cost  speed
• Memory split in hierarchical levels
• Request sent to the next level below until it is carried
• ut.

Cache operation – overview

• CPU requests contents of memory location
• Check cache for this data
• If present, get from cache (fast)
• If not present, read required block from main

memory to cache

• Then deliver from cache to CPU
• Cache includes tags to identify which block of

main memory is in each cache slot

Cache Read Operation

Cache and Main Memory

from now on

Cache Addressing

• Where does cache sit?
• Between processor and virtual memory management unit
• Between MMU and main memory
• Logical cache (virtual cache) stores data using virtual

addresses

• Processor accesses cache directly, not through physical cache
• Cache access faster, before MMU address translation
• Virtual addresses use same address space for different applications
• Must flush cache on each context switch
• Physical cache stores data using main memory physical

addresses

Principle of locality

• Spatial

The processor tends to access few restricted areas of the address space.

• Temporal

The processor tends to access in the near future addresses accessed in the recent past..

Definitions

• Hit : access served by the cache
• Miss: access not served by the cache
• Hitratio: proportion of accesses served by the cache
• Missratio: proportion of accesses not served by the cache
• Clearly m+h =1

accesses

• f

number total cach by the served accesses

• f

number  h accesses

• f

number total cach by the served not accesses

• f

number  m

Definitions

Example: Let

 h be the hitratio  thit the access time on a hit  tmiss the access time on a miss

The average memory access time t will be: t = h thit + (1-h) tmiss

1 h

thit tmiss

Definitions

Block

are zero.

the b least significant bytes .

Does it make sense?

block 0 block 1 … … … …

Fully Associative Cache

Architeture

• VALUE

TAG

valid bit

• f the block copied in

1 2L-1

• lines

2L lines

Fully Associative Cache

Operation

 The cache controller compares the block number and the TAG field of

all lines simultaneously (associative search).

 If a TAG matches the block number and the valid bit is “on”, it is a hit,

• therwise it is a miss.

 The b least significant bits are used as points to the byte/word within the

block.

ADDRESS GENERATED BY THE CPU a-1 b b-1 0 Block number of the addressed byte/word points to the byte/word in the block b

Fully Associative Cache

Problem

To compare the block number with the TAG fields of all cache lines simultaneously (associative search)

• ne needs lots of comparators.

Consequence

Fully associative design is only used for small

capacity caches..

Direct Mapped Caches

Basic Idea:

 Assign each main memory block to a single cache line.

• main memory

blocks cache lines f

Direct Mapped Caches

Basic Idea:

Each main memory block can only be loaded into

the cache line it is mapped to.

Thus, it will be no more necessary to check all lines

but just one.

Direct Mapped Caches

Operation:

 The cache controller compares the address field to the left

with the TAG field of the (single) cache line defined by the L bits.

ADDRESS GENERATED BY THE CPU a-1 b+L b+L-1 b b-1 0

points to a byte/work within the block

b L

points toa cache line

Direct Mapped Caches

Problem:

Some lines may be often requested by different blocks, while other lines are rarely requested, which implies in a non-optimal use of the cache capacity.

Set Associative Caches

Basic Idea:

 Instead of assigning each main memory block to a single

cache line, assign each block to a set (associative) of cache lines.

• main memory

blocks cache lines associative sets f

Set Associative Caches

Basic Idea:

 A block may be loaded into any cache line of the associative

set it is assigned to.

• main memory

blocks cache lines associative sets f

Set Associative Caches

Architecture

set 2S sets

2S-1

1 … … … … … … …

Set Associative Caches

Operation: assume that there are 2S sets

 The cache controller compares the address field to the left

with the TAG field of all rows of the associative set defined by the S bits (associative search).

ADDRESS GENERATED BY THE CPU a-1 b+S b+S-1 b b-1 0 b S

to be compared with the TAG field points to a byte/work within the block points toa set

Set Associative Caches

 Fully Associative Caches:

Are set associative caches with a single associative set.

 Direct Mapped Caches:

Are set associative caches whose “associative” sets

contain each a single line.

Set Associative Caches

Set size:

Keeping the overal cache capacity constant and

changing the number of lines/set.

Lines/set missratio (h) 20 21 22 23 Direct mappedtwo-way tour-wayeigth-way Fully associative Above 4 lines/set the missratio does not change significativelly

Set Associative Caches

Replacement Policy

 Least Recently Used - LRU

 The least recently used line will be merged from cache to

make room for a new main memory block.  Pseudo LRU

Example: a four-way set associative cache

 The least recently used line of the least recently used half is

elected to leave the cache.

bit I0

=0 =1

=0 =1 =0 =1

bit I1 bit I2

Replacement Policy

• Example:

• LRU
• Pseudo LRU

a b a c b a d c b a a d c b e a d c b e a d e b a d a b a c b a d c b a d c a b e d a b e d b a e d b a a b c d a e b e f a b c d a e b e f

• nly accesses to the set →
• nly accesses to the set →

Main Memory Update Policy

• Write Through

 All writes are carried out in the cache and in the main

memory.

 The CPU does not halt until the main memory is updated.

• Problem

 Lots of traffic → specially harmful in multiprocessors  15% of memory references are writes.

Main Memory Update Policy

• Write Back

 Each cache line has a bit (dirty) that indicates when set (=1),

that te block copy in the cache differ from the main memory.

 When the block is brought from main memory into the

cache, dirty =0

 All writes are performed in the cache only and, in this case,

dirty=1.

 The main memory is updated when the block selected for

replacement has dirty=1.

• I/O must access main memory through cache

Multilevel Caches

• High logic density enables caches on chip

Faster than bus access Frees bus for other transfers

• Common to use both on and off chip cache

L1 on chip, L2 off chip in static RAM L2 access much faster than DRAM or ROM L2 often uses separate data path L2 may now be on chip Resulting in L3 cache

Multilevel Caches(L1 & L2)

• nly advantageous if L2 > L1

Unified v Split Caches

• One cache for data and instructions or two, one for

data and one for instructions

• Advantages of unified cache
• Higher hit rate
• Balances load of instruction and data fetch
• Only one cache to design & implement
• Advantages of split cache
• Eliminates cache contention between instruction

fetch/decode unit and execution unit

• Important in pipelining

Exercises

Exercise 1 A cache with 64 Kbyte capacity operates with 8 byte blocks and is organized in associative sets having 4 lines each. What is the number of the associative set that may contain a copy of the byte in the main memory address 3B EF 56 H ?

Exercises

Exercise 2

Determine the address of the byte stored in the VALUE field in the byte indicated by the shadowed box. It is a 8-way set associative cache with 256 Kbytes capacity and 8 byte blocks. It is further known that the byte is stored in the associative set number 24H and the TAG field has the value 18H.

VALUE TAG

Exercises

Exercise 3

How many bits are required to represent all relevant configurations of the eight lines of a single associative set in a cache, which uses the LRU policy? And for the pseudo-LRU?

Exercises

Exercise 4

The physical address space of processor is 4 Gbyte (230 byte) memory. Its cache stores up to 1 Mbyte (220 byte), operates with 256 byte blocks and is

• rganized in associative sets containing four lines each.

How many main memory blocks map into a single associative set?

Exercises

Exercise 5:

A processor accesses 6 blocks of main memory in the folowing sequence: a b c d d c e b c a d f All of them map into the same set of a four-way set associative cache. Assume that at the beginning all lines of this set are empty and are being filled as misses occur. Indicate below each block replaced in the cache and the corresponding missing block whose access caused the replacement. Do it for LRU and for pseudo-LRU.

• utgoing block

• utgoing block

References

Stallings chapters 4 and 5

Simulators

 De William Stallings

http://www.ecs.umass.edu/ece/koren/architecture/Cache/frame0.htm

Cache Memory

END