Orchestra: Intrusion Detection Using Parallel Execution and - - PowerPoint PPT Presentation

Orchestra: Intrusion Detection Using Parallel Execution and Monitoring of Program Variants in User-Space

Babak Salamat, Todd Jackson, Andreas Gal, and Michael Franz Secure Systems and Languages Laboratory Department of Computer Science University of California, Irvine EuroSys 2009, April 2009

1

Multi-Variant Execution

2

Variant 2 Variant 1 Variant 2 Variant 1

Detection Requirements

 Lock-step execution  Feed all variants with identical input  Variants which behave differently when attacked

3

Free Space Other Data Buffer . . . Frame Pointer Return Address Caller Data Upward Growing Stack Free Space Other Data Buffer . . . Frame Pointer Return Address Caller Data Downward Growing Stack

Reverse Stack Growth Direction

 Stack objects are located in opposite positions

4

Return Address Return Address

5

From Source to Execution

6

Source Code Modified Compiler (GCC 4.2) Modified Library (Dietlibc) Multi- Variant Execution Modified RTL generation Modified code generation Modified assembly code of the library Implemented the multi-variant monitor ~10,000 LoC in C++

Orchestra Architecture

 The monitor is a user-space application

7

Diversified Application

Operating System Multi-Variant Monitor Untrusted Trusted

Library

Conventional Application

Library 1

Variant 1

Library n

Variant n

...

Granularity of Monitoring

 Granularity of monitoring and Synchronization

 Ideally after each instruction  Not always possible  Performance issues

 Synchronize and monitor at system calls

 No harm is done without invoking a system call  All instances must invoke the same syscall with equivalent arguments

8

System Call Monitoring

 Debugging facility of Linux (ptrace) is used to build the monitor  The monitor is notified twice per system call

9

Variant 2 Variant 1

Syscall Syscall

Monitor

System Call Monitoring (cont.)

 Equivalency is checked at the beginning of a system call

 The system calls must be the same  Arguments must be equivalent  Pointers (buffers) have the same content  Values are identical

 Results of the system call are written back to the variants at the end of the system call if needed

10

System Call Execution

 Non-state changing system call that produce immutable results are executed by all

11

Variant 2 Variant 1

uname uname

Monitor

Variant 2 Variant 1

uname uname

Monitor

System Call Execution (cont.)

 State changing system calls are executed by the monitor

12

Variant 2 Variant 1

write(stdout) write(stdout)

Monitor

Variant 2 Variant 1

skipped skipped

Monitor

write(stdout)

Variant 2 Variant 1

skipped skipped

Monitor

write(stdout)

5 5 5

System Call Execution (cont.)

 Non-state changing system call that produce non- immutable results are executed by all, results are copied from the first variant to all

13

Variant 2 Variant 1

gettime gettime

Monitor

8:00 8:01 Variant 2 Variant 1

gettime gettime

Monitor

8:00 8:00

Data Transfer

 ptrace transfers only 4 bytes at a time

 very slow in transferring large buffers

14

Variant 1 4K Buffer

1000 ptrace calls

Monitor

Data Transfer (cont.)

 We tried using named pipes, but they cannot transfer more than 4K bytes at a time  Shared memory is fast and can transfer mega bytes

15

Monitor

Variant 1 Shared memory Buffer

my memcpy

Data Transfer Performance

!" !#" !##" !###" !####" !#####" $" %&" !&$" '!&" &(" $(" %&(" !&$(" !"#$%&'()*+&(,-*./0%&.01'%2( 345&/(6*7&(,389&%2( )*+,-." /0/1" 23,+.4"5.67+8"

Shared memory is about 1000 times faster than ptrace and 20 times faster than FIFO in transferring a 128K buffer

16

Removing False Positives

False positives are the major practical issue in using multi-variant execution

17

Variant 1 Variant 2

Multi-Threaded Variants

 Different scheduling of multi-threaded or multi-process applications can cause false positives

18

Variant 2 Thread 1 Variant 2 Thread 2 Variant 1 Thread 2 Variant 1 Thread 1

write read read write

Monitoring multi-threaded variants

 Corresponding threads/processes must be synchronized to each other

19

Variant 2 Thread 1 Variant 2 Thread 2 Variant 1 Thread 2 Variant 1 Thread 1

write read read write

Monitor Thread 2 Monitor Thread 1

File Descriptors

 The same file descriptor is always reported to all variants when they invoke system calls that return a file descriptor

20

Variant 2 Variant 1

Monitor

pipe

9

pipe

7 Variant 2 Variant 1

Monitor

pipe

11

pipe

11 Variant 2 Variant 1

Monitor

• pen(1.txt, WR)
• pen(1.txt, WR)

Variant 2 Variant 1

Monitor

• pen(1.txt, WR)

5 5

skipped skipped

Process ID

 Monitor reports the process ID of the first variant to all  The PID of the first variant’s child process is reported as the result of fork or clone to all the variants

21

Variant 2 PID: 101 Variant 1 PID: 100

getpid

100

getpid

100

Monitor

Variant 2 PID: 101 Variant 1 PID: 100

getpid

100

getpid

101

Monitor

Process IDs in Arguments

 When variants need to run a system call that receives a PID, appropriate PID is restored before the execution of the system call

22

Variant 2 PID: 101 Variant 1 PID: 100

Monitor

kill(100) kill(100)

Variant 2 PID: 101 Variant 1 PID: 100

Monitor

kill(100) kill(101)

Asynchronous Signals

 Signal handlers can cause different sequences of system calls to be executed by the variants

23

Variant 1 Variant 2

Syscall 1 Syscall 1

signal handler

Syscall 8

Time and Random Numbers

 System calls that read time (e.g., gettimeofday) are executed by one variant and the result is copied to all  By providing identical time and other system information to all variants, they likely use the same seed to generate random numbers  The monitor reads /dev/urandom and copies the result to all variants  Reading CPU time stamp counters (RDTSC) may still cause false positives

24

Performance

!"# $!"# %!"# &!"# '!"# (!!"# )*+,-# ./01# .))# 21*# 1+*34*## 14*5678# 7)9# .+1## 2:*;4<## 6/01$## ;=:59### 4>?+84## +*;#### 743+#### +1+)@4# ;+*# ABC# 7CDC441# +24*+.4# EF#G<4)?H:B# EE#G<4)?H:B# F#G<4)?H:B#

SPEC

25

Summary

 Multi-variant execution is an effective technique in detecting and disrupting attacks  A reverse stack executable can prevent stack-based buffer overflow vulnerabilities in a multi-variant environment  Our methods can remove most sources of false positives in multi-variant execution  Running two parallel variants have about 15% overhead

26

Thank you

Questions?

27