Attacking a co-hosted VM A hacker, a hammer and two memory modules - - PowerPoint PPT Presentation

Attacking a co-hosted VM

A hacker, a hammer and two memory modules

1

Who we are (1)

Mehdi Talbi

• Security researcher at Stormshield
• haka-security project.
• Past life: academia (phd, postdoc, …)
• Main research topics: advanced exploitation

techniques, abusing virtual machines (phrack paper), low-level stuff, …

@abu_y0ussef mtalbi.github.io

2

Who we are (2)

Paul Fariello

3

• Security engineer at Stormshield
• haka-security project.
• Past life: Software developer at Stormshield
• Main interest: low-level stuff, vulnerability

exploitation, software engineering

The attack

Goal: authenticate on the victim VM - given that:

• Credentials are not known to the attacker
• Victim VM is not vulnerable to any software bug

4

Teaser

5

Teaser

5

Agenda

• Row-hammering (from a VM)
• Abusing memory de-duplication
• Pwning the libpam
• Conclusion

6

Row-hammering

7

The Bug

• DRAM Chip: rows of cells (refreshed

periodically)

• Read/Write operation
• Data is transferred to the row-buffer

(discharging)

• Data is copied back to the original row

(recharging)

• Frequent row activation (discharging +

recharging) -> Disturbance errors -> high discharge rate on adjacent row’s cells -> bit-flips in adjacent memory rows (if they are not refreshed before they lose their charge)

8

9

The Code

1 code1a: 2 mov (X), %eax /* read from address X */ 3 mov (Y), %ebx /* read from address Y */ 4 clflush (X) /* flush cache for address X */ 5 clflush (Y) /* flush cache for address Y */ 6 jmp code1a

Past attacks - Timeline

10

Past attacks - Timeline

June 2014 - Yoongu Kim et al. Flipping bits in memory without accessing them 10

Past attacks - Timeline

June 2014 - Yoongu Kim et al. Flipping bits in memory without accessing them March 2015 - Mark Seaborn and Thomas Dullien Exploiting the DRAM rowhammer bug to gain kernel privileges 10

Past attacks - Timeline

June 2014 - Yoongu Kim et al. Flipping bits in memory without accessing them March 2015 - Mark Seaborn and Thomas Dullien Exploiting the DRAM rowhammer bug to gain kernel privileges May 2016 - Erik Bosman et al. Dedup Est Machina 10

Past attacks - Timeline

June 2014 - Yoongu Kim et al. Flipping bits in memory without accessing them March 2015 - Mark Seaborn and Thomas Dullien Exploiting the DRAM rowhammer bug to gain kernel privileges August 2016 - Kaveh Razavi et al. Flip Feng Shui May 2016 - Erik Bosman et al. Dedup Est Machina 10

Past attacks - Timeline

June 2014 - Yoongu Kim et al. Flipping bits in memory without accessing them March 2015 - Mark Seaborn and Thomas Dullien Exploiting the DRAM rowhammer bug to gain kernel privileges August 2016 - Kaveh Razavi et al. Flip Feng Shui May 2016 - Erik Bosman et al. Dedup Est Machina October 2016 - Victor Van Der Venn et al. Drammer 10

Channels, ranks, banks

• channel: multiple memory modules
• memory module: multiple memory chips may be available on

both sides (ranks)

• memory chips made of banks
• bank: matrix (column X rows) of memory cells.

11

DRAM Config.

• 8 GB of RAM
• 2 channels
• 1 memory module / channel
• 2 ranks / memory module
• 8 memory chips / rank
• 8 bank / chip
• 2^15 rows x 2^10 columns x 8 bits / bank

Total Size = 2 modules * 2 ranks * 2^3 chips * 2^3 banks * 2^15 rows * 2^10 columns * 1 byte = 8 GB

12

Memory Geometry

• How physical addresses map to channels, ranks, banks ?

13

Memory Geometry

• How physical addresses map to channels, ranks, banks ?
• Mark Seaborn has determined the address mapping for

Sandy Bridge microarchitecture:

• Bits 0-5: lower 6 bits of byte index within a row
• Bit 6: channel selection
• Bits 7-13: higher 7 bits of byte index within a row
• Bits 14-16: bank selection obtained by:

((addr >> 14) & 7) ^ ((addr >> 18) & 7)

• Bit 17: rank selection
• Bit 18-32: row selection

13

Row-hammering from a VM

14

Row-hammering from a VM

• How to pick a couple of addresses (a_1, a_2) to hammer

such that:

• a_1 is in row r,
• a_2 is in row r+2,
• a_1 and a_2 are in the same bank b ?

14

Row-hammering from a VM

• How to pick a couple of addresses (a_1, a_2) to hammer

such that:

• a_1 is in row r,
• a_2 is in row r+2,
• a_1 and a_2 are in the same bank b ?
• We don’t have access to host’s pagemap file :-(

14

Row-hammering from a VM

• How to pick a couple of addresses (a_1, a_2) to hammer

such that:

• a_1 is in row r,
• a_2 is in row r+2,
• a_1 and a_2 are in the same bank b ?
• We don’t have access to host’s pagemap file :-(
• Option: Transparent Huge Pages

14

Transparent Huge Pages

• khugepaged is a kernel thread that attempts to

allocate huge pages of 2 MB size.

• A huge page covers multiple rows:
• We can navigate through rows !!

page size row size = 2 × 220 218 = 8 rows

15

Transparent Huge Pages

16

Transparent Huge Pages

• We allocate in the guest a

large buffer (available physical RAM) aligned on 2 MB boundary.

16

Transparent Huge Pages

• We allocate in the guest a

large buffer (available physical RAM) aligned on 2 MB boundary.

• khugepaged running in the

guest will try to back the buffer by huge pages.

16

Transparent Huge Pages

• We allocate in the guest a

large buffer (available physical RAM) aligned on 2 MB boundary.

• khugepaged running in the

guest will try to back the buffer by huge pages.

• The same goes for

khugepaged running in the host.

16

Row Selection

17

Row Selection

• Hammer each pair of rows i, i+2 (double-sided row-hammer).

17

Row Selection

• Hammer each pair of rows i, i+2 (double-sided row-hammer).
• We can select pair of addresses from (i, i+2) that belong to the same rank, channel. How

about banks?

17

Row Selection

• Hammer each pair of rows i, i+2 (double-sided row-hammer).
• We can select pair of addresses from (i, i+2) that belong to the same rank, channel. How

about banks?

• Let and bits identifying a bank in row i and j, respectively.
• We hammer each address from row i with the address from j (j = i + 2) satisfying this condition:

bi1bi2bi3

bj1bj2bj3

bi3 = bj3 bi1bi2 6= bj1bj2

17

18

Quad-Hammering

1 static void hammer_byte(uintptr_t aggressors[]) 2 { 3 volatile uint64_t *a = (volatile uint64_t *)aggressors[0]; 4 volatile uint64_t *b = (volatile uint64_t *)aggressors[1]; 5 volatile uint64_t *c = (volatile uint64_t *)aggressors[2]; 6 volatile uint64_t *d = (volatile uint64_t *)aggressors[3]; 7 8 int nb_reads = READ_REPZ; 9 10 while (nb_reads-- > 0) { 11 *a; 12 *b; 13 *c; 14 *d; 15 asm volatile ( 16 "clflush (%0)\n\t" 17 "clflush (%1)\n\t" 18 "clflush (%2)\n\t" 19 "clflush (%3)\n\t" 20 : 21 : "r" (a), "r" (b), "r" (c), "r" (d) 22 : "memory" 23 ); 24 } 25 }

Abusing memory de-duplication

19

Memory De-duplication

Before Merging

20

After Merging

Attack Steps

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content.

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory
• 2. Hammer the memory from attacker VM

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory
• 2. Hammer the memory from attacker VM
• 3. Load the target file in the victim VM (e.g.

by trying to authenticate on the victim VM)

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory
• 2. Hammer the memory from attacker VM
• 3. Load the target file in the victim VM (e.g.

by trying to authenticate on the victim VM)

• 4. Wait for memory deduplication.

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory
• 2. Hammer the memory from attacker VM
• 3. Load the target file in the victim VM (e.g.

by trying to authenticate on the victim VM)

• 4. Wait for memory deduplication.
• 5. Hammer again

21

Attack Steps

Goal: Modify a bit in a file (e.g. pam_unix.so) hosted in the victim VM assuming we know its content. Hint: Row-hammering is reproducible

• 1. Map the available physical memory
• 2. Hammer the memory from attacker VM
• 3. Load the target file in the victim VM (e.g.

by trying to authenticate on the victim VM)

• 4. Wait for memory deduplication.
• 5. Hammer again
• 6. Enjoy !!!

21

Kernel Same-Page Merging

• KSM reduces considerably the memory footprint,

especially in virtual machine environments.

• KSM periodically scans the memory looking for pages

with identical content (100 pages every 20 ms.)

• KSM merges identical anonymous pages having the

flag MADV_MERGEABLE set.

• Writing to a merged page triggers a page fault. Pages

are unmerged during CoW.

22

KSM Internals

23

• KSM maintains two red-black trees:
• The stable tree: keeps track of shared pages.
• The unstable tree: stores pages that are candidate for

merging.

KSM Internals

23

• KSM maintains two red-black trees:
• The stable tree: keeps track of shared pages.
• The unstable tree: stores pages that are candidate for

merging.

• The unstable tree is trashed and reconstructed

after a complete scan.

KSM Internals

23

• KSM maintains two red-black trees:
• The stable tree: keeps track of shared pages.
• The unstable tree: stores pages that are candidate for

merging.

• The unstable tree is trashed and reconstructed

after a complete scan.

• FIRST, KSM tries to merge pages from the stable

tree THEN from the unstable tree.

KSM Internals

23

« Winning » the Merge

• How to guarantee that KSM will back merged pages

by the physical page controlled by the attacker ?

24

« Winning » the Merge

• How to guarantee that KSM will back merged pages

by the physical page controlled by the attacker ?

• Option 1: Starting the attacker VM before the victim
• VM. KSM will keep the page that has registered first

for merging. The merge is performed from the unstable tree

24

« Winning » the Merge

• How to guarantee that KSM will back merged pages

by the physical page controlled by the attacker ?

• Option 1: Starting the attacker VM before the victim
• VM. KSM will keep the page that has registered first

for merging. The merge is performed from the unstable tree

• Option 2: Load two copies of the target file in the

attacker VM so that the future merges are performed from the stable tree.

24

KSM Vs THP

25

KSM Vs THP

• THP merges 4 KB pages to form huge pages.

25

KSM Vs THP

• THP merges 4 KB pages to form huge pages.
• KSM merges 4 KB pages sharing the same

content.

25

KSM Vs THP

• THP merges 4 KB pages to form huge pages.
• KSM merges 4 KB pages sharing the same

content.

• KSM can break our huge pages :-(

25

KSM Vs THP

• THP merges 4 KB pages to form huge pages.
• KSM merges 4 KB pages sharing the same

content.

• KSM can break our huge pages :-(

—> Solution: Add some entropy to memory pages (we fill the top of every 4 KB page with 8 random bytes)

25

Pwning the Libpam

26

RE the Libpam

• Libpam: Attractive target —> provides authentication mechanisms
• n widely deployed *.nix systems.
• pam_sm_authenticate [0x3440]: performs the task of authenticating a user.
• _unix_blankpassword [0x6130]: checks if the user does not have a blank password.

27

Flippable Bits

28

Flippable Bits

• Given a program P, how to find all « flippable » bits

in the program that can change the outcome of P ?

28

Flippable Bits

• Given a program P, how to find all « flippable » bits

in the program that can change the outcome of P ?

• Use of timeless-debugging capabilities:
• 1. Flip a bit of a some target function.
• 2. Run the target function.
• 3. Check whether the flipped bit impacts the expected result of

the target function.

• 4. Restore session and proceed with next bit.

28

Flippable Bits

29

Flippable Bits

Offset Bit Direction Original Instruction Patched Instruction 0x34c6 1 0 —> 1 test eax, eax xchg eax, eax 0x34c8 0 —> 1 je 0x3538 jne 0x3538 0x34c8 2 1 —> 0 jo 0x3538 0x34c8 3 0 —> 1 jl 0x3538 0x34c8 6 1 —> 0 xor al, 0x6e 0x3520 3 1 —> 0 mov eax, ebx mov eax, edx 0x3520 4 1 —> 0 mov eax, ecx 0x3520 5 0 —> 1 mov eax, edi 0x36c0 1 0 —> 1 mov ebx, eax mov eax, ebx 0x36c1 1 —> 0 mov edx, eax 0x36c1 1 1 —> 0 mov ecx, eax 0x36c1 2 0 —> 1 mov edi, eax 0x36c1 3 0 —> 1 mov ebx, ecx 0x36c1 4 0 —> 1 mov ebx, edx 0x6211 3 0 —> 1 xor eax, eax xor eax, edx 0x6211 4 0 —> 1 xor eax, ecx 0x6211 5 0 —> 1 xor eax, esp 30

The Attack

31

The Attack

• 1. Allocate the available physical memory.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

• 4. Load pam_unix.so module in the victim page in the attacker VM if the bit-flip

matches one of the target offsets.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

• 4. Load pam_unix.so module in the victim page in the attacker VM if the bit-flip

matches one of the target offsets.

• 5. Load pam_unix.so module in the victim VM by trying to log on.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

• 4. Load pam_unix.so module in the victim page in the attacker VM if the bit-flip

matches one of the target offsets.

• 5. Load pam_unix.so module in the victim VM by trying to log on.
• 6. Wait for KSM to merge the pages.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

• 4. Load pam_unix.so module in the victim page in the attacker VM if the bit-flip

matches one of the target offsets.

• 5. Load pam_unix.so module in the victim VM by trying to log on.
• 6. Wait for KSM to merge the pages.
• 7. Hammer again the aggressor addresses that have produced the bit-flip in

question.

31

The Attack

• 1. Allocate the available physical memory.
• 2. Add some entropy to memory pages to prevent KSM from breaking THP pages.
• 3. Hammer each pair of aggressor rows in every huge page and check if we have

bit-flips in the victim row.

• 4. Load pam_unix.so module in the victim page in the attacker VM if the bit-flip

matches one of the target offsets.

• 5. Load pam_unix.so module in the victim VM by trying to log on.
• 6. Wait for KSM to merge the pages.
• 7. Hammer again the aggressor addresses that have produced the bit-flip in

question.

• 8. Enjoy !!

31

Demo

32

Demo

32

Going Further

33

Going Further

• Modification to a merged page —> page fault —>

CoW —> page un-merging. —> Noticeable difference in time between writing to a duplicated page and writing to an unshared page.

33

Going Further

• Modification to a merged page —> page fault —>

CoW —> page un-merging. —> Noticeable difference in time between writing to a duplicated page and writing to an unshared page.

• Exploiting a side-channel timing attack in KSM:
• Detect when two pages are merged.
• Detect the version of the Libpam in the victim VM.

33

CAIN Attack

34

CAIN Attack

• Cain attack by Antonio

Barresi et al.:

34

CAIN Attack

• Cain attack by Antonio

Barresi et al.:

• 1. Allocate a large buffer on N elements

(4KB each)

34

CAIN Attack

• Cain attack by Antonio

Barresi et al.:

• 1. Allocate a large buffer on N elements

(4KB each)

• 2. Fill buffer with random

34

CAIN Attack

• Cain attack by Antonio

Barresi et al.:

• 1. Allocate a large buffer on N elements

(4KB each)

• 2. Fill buffer with random
• 3. Copy each even element of the first half

in its corresponding index in the second half of the buffer

34

CAIN Attack

• Cain attack by Antonio

Barresi et al.:

• 1. Allocate a large buffer on N elements

(4KB each)

• 2. Fill buffer with random
• 3. Copy each even element of the first half

in its corresponding index in the second half of the buffer

• 4. Modify a byte in each element of the first

half and measure writing operation time

34

Conclusion

35

Conclusion

• Successful row-hammer attack on a co-hosted VM:

corrupting the state of pam_unix.so module.

35

Conclusion

• Successful row-hammer attack on a co-hosted VM:

corrupting the state of pam_unix.so module.

• Weaponizing row-hammer attacks.

https://github.com/mtalbi/pwnpam

35

Conclusion

• Successful row-hammer attack on a co-hosted VM:

corrupting the state of pam_unix.so module.

• Weaponizing row-hammer attacks.

https://github.com/mtalbi/pwnpam

35

Conclusion

• Successful row-hammer attack on a co-hosted VM:

corrupting the state of pam_unix.so module.

• Weaponizing row-hammer attacks.
• Row-hammer attacks are powerful and effective.

https://github.com/mtalbi/pwnpam

35

Conclusion

• Successful row-hammer attack on a co-hosted VM:

corrupting the state of pam_unix.so module.

• Weaponizing row-hammer attacks.
• Row-hammer attacks are powerful and effective.
• Mitigation:
• ECC RAM
• Disabling KSM

echo 0 > /sys/kernel/mm/ksm/run

https://github.com/mtalbi/pwnpam

35

Greetz

• Pierre Sylvain Desse
• Stormshield
• Kudos to Mark Seaborn, @halvarflake and @vu5ec

researchers.

36

References

• [1] Kaveh Razavi, Ben Gras, Erik Bosman, Bart Preneel,

Cristiano Giuffrida and Herbert Bos. Flip Feng Shui: Hammering a Needle in the Software Stack. In USENIX

• Security. 2016.
• [2] Antonio Barresi, Kaveh Razavi, Mathias Payer and

Thomas R. Gross. CAIN: Silently Breaking ASLR in the

• Cloud. In USENIX Workshop on Offensive Technologies.
• [3] Mark Seaborn and Thomas Dullien. Exploiting the

DRAM rowhammer bug to gain kernel privileges. BH US 2015.

37

THANK YOU

38