Where is the FEEB? The Effectiveness of Instruction Set - - PowerPoint PPT Presentation

CSE 598A - Spring 2007 - Sandra Rueda Page 1

Ana Sovarel, David Evans and Nathanael Paul Presented by Sandra Rueda

Where is the FEEB? The Effectiveness of Instruction Set Randomization

CSE 598A - Spring 2007 - Sandra Rueda Page 2

Code Injection Attacks

• High level description:

– Look for an input – Insert code – Alter (Transfer) control flow

• Main target:

– Buffer overflow vulnerabilities

• Threat to:

– C Programs – CGI Scripts

• The attacker needs to know the instruction set to succeed

CSE 598A - Spring 2007 - Sandra Rueda Page 3

ISR

• If the attacker needs to know the instruction set …
• Let’s obscure that set:

– Instruction Set Randomization (ISR) 0x3c 0011 1100 0x33 0011 0011 0x0F 0000 1111 Original opcode: Encoding Key: XOR New opcode: 1100 2103 1001 0011 0110 Original opcode: Transposing Key: New opcode:

CSE 598A - Spring 2007 - Sandra Rueda Page 4

ISR Overview

• Threat model:

– ISR is a defense against external users – It is designed to protect remote servers – (It is not designed as a defense against internal users)

• Architecture

Loading: Executing:

Memory eKey(Instruct) PCB PID, PC, … Key Executable: eKey(Instruct) + Key Decoding Register Key

CSE 598A - Spring 2007 - Sandra Rueda Page 5

Proof of Concept

• Emulation 1 [Kc et al. CCS’03]

– Two ways of randomizing: xor, transposition – 32 bit keys – Mechanisms to randomize binary and perl code

• Emulation 2 [Barrantes et al. CCS’03]

– Randomizing with xor – Keys as long as the program: they are generated from a seed and a pseudorandom sequence – Mechanism to randomize binary code

• Performance: ISR does not considerably slows down I/O intensive

processes

CSE 598A - Spring 2007 - Sandra Rueda Page 6

How secure is it?

• Key length (XOR):

– Key as long as the instructions

• 32 bits instructions lead to 32 bits keys 232 options

– Key as long as the program

• s = length of the seed 2s options
• Key length (Transposition):

– nb = #bits per instruction l = nb * log2(nb) – < log2(l!) permutations (not all permutations are valid)

• It is supposed that injected code will raise an exception after de-

randomization:

– Invalid opcode – Invalid address access – In some cases the de-randomized instructions may work (tested with real attacks it is a low percentage) [Barrantes et al.]

CSE 598A - Spring 2007 - Sandra Rueda Page 7

ISR Drawbacks

• Requires special support

– Hardware based support | binary to binary translators

• Applications must be statically linked

– Alternatives: encrypting everything at the beginning or on-demand – Alternatives slow down response time

• It can not handle self-modifying code
• Vulnerable to denial of service attacks

CSE 598A - Spring 2007 - Sandra Rueda Page 8

Effectiveness of ISR

• Is it possible to guess the

randomization key(s) given time and resource constraints ?

CSE 598A - Spring 2007 - Sandra Rueda Page 9

Proposed Attack

• Strategy: incrementally break the key
• Mechanism:

– Trial and error – Select a random key – Inject an identifiable sequence of instructions

• Identifiable means … being able to detect if a partial key is right or wrong Socket

connection satisfies this requirement

• And being able to determine the correct key from the guess XOR satisfies this

requirement

– Options:

• Return attack
• Jump attack

CSE 598A - Spring 2007 - Sandra Rueda Page 10

Return Attack

• The stack is compromised the attack can only be used

if observable activity occurs before the crash

• “We suspect situations where the return attack can be

used are rare, but an attacker who is fortunate enough to find such vulnerability can use it …”

local buffer … saved base pointer saved ret address …

Stack Layout:

Before After local buffer encrypted ret instruction … saved base pointer

• verwritten ret address

… saved ret address 1 byte

CSE 598A - Spring 2007 - Sandra Rueda Page 11

Jump Attack

• It produces infinite loops
• If the guess is correct the socket remains open
• If it is not correct the server crashes and the socket is closed

local buffer … saved ret address …

Stack Layout:

Before After local buffer short jump (eb)

• ffset -2 (fe)

…

• verwritten ret address

… 1 byte

The FEEB!

CSE 598A - Spring 2007 - Sandra Rueda Page 12

Incremental Key Breaking

• If the key used is as long as the program …

– Two keys (from a successful jump attack) are not enough – Repeat the attack using the two already known keys

• When the same 32-bit key is used

– max_attempts_return = 1024 (4 * 28) – max_attemps_jump = 66048 (216 + 2 * 28) – Extra attempts may be needed because of false positives

• “This can not be done with the described approach”

CSE 598A - Spring 2007 - Sandra Rueda Page 13

Eliminating False Positives

• Return Attack:
• False positives may happen because

– The injected guess is decrypted to a instruction similar to near return – The injected guess is decrypted to a harmless instruction and a subsequent instruction behaves like a near return

• Options:

– Try all 255 possibilities

• If none behaves like a return then the mask is the actual instruction (0xc3)
• If more than one produce the expected behavior :

… harmless instruction near return behavior …

• verwritten ret address

use the guess use previous knowledge

CSE 598A - Spring 2007 - Sandra Rueda Page 14

Eliminating False Positives

• Jump Attack:
• False positives may happen because:

– The injected guess is decrypted to a instruction that causes an infinite loop – The injected guess is decrypted to a harmless instruction and a subsequent instruction causes the infinite loop – The injected guess cause a crash but there is a delay that is interpreted as an infinite loop (it is longer than the threshold)

• Options:

– Only near conditional jumps may generate the same behavior, all of them share the same first four bits 213 = 25 (opcode) * 28 (offset) – It happens when the first byte decrypts to a harmless instruction, the second to a jump instruction and the third to -2 or -3 change the sign to the third byte – Increase the threshold

CSE 598A - Spring 2007 - Sandra Rueda Page 15

Extended Attack

• ISR with short repeated key the proposed attack is enough
• ISR with long keys a larger number of keys is required

– Problematic in the case of Jump attacks because they leave active infinite loops causing degradation in service – However after learning some keys it is possible to improve the attack by using additional instructions

CSE 598A - Spring 2007 - Sandra Rueda Page 16

Deployment

• Inject a micro-VM in the region of memory where masks are known
• The micro-VM has a buffer to load the worm code
• The micro-VM loads worm code in chunks
• The worm code is encrypted with the known keys
• To propagate the worm the VM uses the same techniques already

described (return and jump attacks)

• It requires the old keys thus they are included in the worm code

CSE 598A - Spring 2007 - Sandra Rueda Page 17

Evaluation

• Client:

– Opens a socket to the server – Builds an attack string and sends it – Waits for the acknowledgment – Detects the result

• Target:

– Echo server with a buffer overflow vulnerability – They modified an ISR implementation so the attack would succeed (?) – Address space layout randomization disabled to test the attack

• Results:

– After breaking the first bytes, fewer attempts per byte are required to guess the new keys

CSE 598A - Spring 2007 - Sandra Rueda Page 18

Results

10 100 1000 4 16 64 256 1024

x

• x
• x
• x
• x

Time to acquire key bytes

Key Bytes Acquired Time (sec)

Jump

• Return

x “On average we can break a 100-byte key in just over 6 minutes with the jump attack.”

CSE 598A - Spring 2007 - Sandra Rueda Page 19

Attack Limitations

• The attack works for applications that use the same randomization

key several times

• It will crash a system multiple times

– An administrator might notice the attack

• It does not work for transposition

– It needs to be able to determine the key based on plaintext-ciphertext comparison

• It is heavily dependent on x86 architecture

– RISC: too long instructions to realistically guess using a brute-force attack

• It does not work for higher-level languages as SQL and Perl

CSE 598A - Spring 2007 - Sandra Rueda Page 20

Take Away

• ISR with short repetitive keys are

susceptible to the attack. ISR with long fresh keys are not

• Long fresh keys mean re-randomizing

after crashes. It might cause denial-of- service problems

• Like other tools ISR technique requires

tuning and constant monitoring!

Security vs. Performance Is that bad?