Fuzzing and Debugging Cisco IOS Blackhat Europe 2011 Sebastian Mu - - PowerPoint PPT Presentation

▶
fuzzing and debugging cisco ios
SMART_READER_LITE
LIVE PREVIEW

Fuzzing and Debugging Cisco IOS Blackhat Europe 2011 Sebastian Mu - - PowerPoint PPT Presentation

Jan 16, 2023 367 likes •599 views

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing and Debugging Cisco IOS Blackhat Europe 2011 Sebastian Mu niz, Alfredo Ortega Groundworks

slide-1
SLIDE 1

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Fuzzing and Debugging Cisco IOS

Blackhat Europe 2011 Sebastian Mu˜ niz, Alfredo Ortega

Groundworks Technologies

March 18, 2011

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-2
SLIDE 2

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Agenda

Cisco IOS Architecture Debugger internals Dynamips modification

GDB support IDA Pro support

Shortcomings of self-checking routines Demos:

Malware analysis Fuzzing example

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-3
SLIDE 3

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Dynamips emulator Built-in GDB server

cisco IOS architecture

Single binary image Shared single address space Cooperative priority-based scheduler

Processes

Packet Buffers

Kernel Device Drivers Hardware

Fast Softw. Switch

Figure: Cisco IOS process memory

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-4
SLIDE 4

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Dynamips emulator Built-in GDB server

Dynamips emulator

Created by Christophe Fillot 1 Runs on Windows, Linux and Mac OS X. Equivalent to QEMU/Bochs Implements MIPS/PowerPC architecture and Cisco hardware Supports the following models:

(a) 7200 (b) 36XX (c) 2691 (d) 3725 (e) 3745 (f) 26XX (g) 17XX

1http://www.ipflow.utc.fr/index.php/Cisco 7200 Simulator Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-5
SLIDE 5

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Dynamips emulator Built-in GDB server

Built-in GDB server

Used by Cisco developers and support engineers Works over Telnet, SSH and Serial console Slightly different GDB protocol

Examine Debug Kernel

Read Registers Write Registers Read Memory Write Memory Freeze OS Remote

Figure: GDB debugging modes

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-6
SLIDE 6

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Virtual Machine Debugger internals

Virtual Machine Debugger internals

Dynamips

GDB Protocol PowerPC GDB Server Memory Controller MIPS Special Hard FPGA PCI WIC NM

Figure: GDB Server embedding

CPU/Memory instrumentation No JIT support Supported commands

Read/Write CPU Registers Read/Write Memory Set/Unset Breakpoints

Any standard GDB client supported

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-7
SLIDE 7

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Pros vs Cons Why isolation is good? I don’t need this, I have the verify command Shortcomings of self-checking routines

Pros and Cons of Virtual Machine Debugger

Pros: Complete isolation (almost!) Cost-effective Controlled debugging environment Bug-hunter friendly Cons: Not 100% exact emulation Not all models or hardware compatible Findings need double-check with physical device Check Cisco EULA before doing anything crazy. Just in case.

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-8
SLIDE 8

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Pros vs Cons Why isolation is good? I don’t need this, I have the verify command Shortcomings of self-checking routines

Why isolation is good?

Analyzing malware

Malware Cisco IOS

Mirror Built−In GDB Stub Read_Memory Request

Bytes Expected (fake) GDB Client

Original memory

Figure: Using built-in GDB

Analyzing malware

GDB Stub Malware Cisco IOS Read_Memory Malware memory GDB Client DYNAMIPS dump

Figure: Dynamips GDB server

Lesson learned: NEVER analyze malware inside an infected host.

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-9
SLIDE 9

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Pros vs Cons Why isolation is good? I don’t need this, I have the verify command Shortcomings of self-checking routines

I don’t need this, I have the verify command

Cisco Response on IOS rootkits 2: Maintain chain of trust when verifying IOS images Verify IOS image in external host, or before booting it Use the MD5 File Validation command “verify” on Loaded image: Using the MD5 File Validation Feature

“The MD5 File Validation feature, added in Cisco IOS Software Releases 12.2(4)T and 12.0(22)S, allows network administrators to calculate the MD5 hash of a Cisco IOS software image file that is loaded on a device.”

2http://www.cisco.com/warp/public/707/cisco-sr-20080516-rootkits.shtml Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-10
SLIDE 10

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Pros vs Cons Why isolation is good? I don’t need this, I have the verify command Shortcomings of self-checking routines

Shortcomings of self-checking routines

Malware-affected analysis

Verify CLI command User expected GDB server MD5 CHK Cisco IOS Malware Login routine MD5 chksum (fake)

Figure: Using built-in GDB

Clean analyis

Cisco IOS Calculate Hash Result MD5 Tool External Trusted environment

Figure: Using Dynamips GDB server

Lesson learned (again): NEVER verify code inside an infected host.

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-11
SLIDE 11

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Use cases: IOS malware

Demo: Backdoored IOS installation Not trivial to analyze (Many IOS variations) At least, possible: Demo!

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-12
SLIDE 12

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Use case: ROMMON debugging

ROMMON: Cisco bootloader 3 Very easy to verify and analyze (less variations) Read-only in some models Contains a basic but privileged debugger ROMMON itself can be debugged by Dynamips

3 Felix ’FX’ Lindner , 25c3, Cisco IOS - Attack & Defense Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-13
SLIDE 13

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing requirements Timing diagram Example fuzzer Triggered Vulnerability

Fuzzing requirements

Correct exception handling Reproducible test-cases Logging Desirable: Debugging envirment (for post-analysis)

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-14
SLIDE 14

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing requirements Timing diagram Example fuzzer Triggered Vulnerability

Fuzzing timing diagram

Fuzzer Log Dynamips

Exception Restart

GDB

Fuzz case N+1 Registers Get Regs Start Fuzz case N Signal Restart Start

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-15
SLIDE 15

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing requirements Timing diagram Example fuzzer Triggered Vulnerability

Example fuzzer

Attack surface via Protocol fuzzer (ftp) Trivial test-case generation (just an example!)

DB Yes No No Yes Send:

Command + "AAA..." (100 A’s)

Start Connect to FTP Crash? Save state Disconnect

More CMDs?

End

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-16
SLIDE 16

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing requirements Timing diagram Example fuzzer Triggered Vulnerability

Fuzzer Demo

Demo!

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-17
SLIDE 17

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up Fuzzing requirements Timing diagram Example fuzzer Triggered Vulnerability

Triggered Vulnerability

Cisco Security Advisory: Multiple Vulnerabilities in the IOS FTP Server (cisco-sa-20070509-iosftp) 30 FTP commands, remote code execution on 16:

(USER,CWD,DELE,RNFR,STOR,NLST,APPE,MKD, RMD,STOU,RETR,LIST,STAT,MDTM,SIZE, and HELP)

Patched in 2007: Completely remove all FTP server code

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-18
SLIDE 18

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

How secure is this debugger?

Very. Can be used in a production environment to analyze malicious code? No Dynamips contains emulation bugs. Demo!

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-19
SLIDE 19

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Future Development

Honeypots Malware analysis Lab Exploit Dev Duplicate exact memory behaviour (typical VMs problems) Secure host isolation (squash Dynamips bugs)

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-20
SLIDE 20

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

Questions?

Via email: aortega@groundworkstech.com smuniz@groundworkstech.com Please download:

http://www.groundworkstech.com/projects/dynamips-gdb-mod Published under the GNU General Public Licence (GPL)

Groundworks Technologies Fuzzing and Debugging Cisco IOS

slide-21
SLIDE 21

Cisco IOS architecture Architecture Analyzing Pros and Cons Use case: IOS malware Use case: ROMMON debugging Use cases: Fuzzer Wrapping up

The End

Thanks for listening!

Groundworks Technologies Fuzzing and Debugging Cisco IOS