Malware Prof. Tom Austin San Jos State University Malware: The Cat - - PowerPoint PPT Presentation

CS 166: Information Security

• Prof. Tom Austin

San José State University

Malware

Malware: The Cat & Mouse Game Attackers and Defenders Play

1971

"I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN." "I'M THE CREEPER : CATCH ME IF YOU CAN."

Creeper

• The world's first computer

virus (probably)

• Created by Bob Thomas as an experiment

in "mobile" self-replicating programs

• Not malicious, just annoying…
• …Very annoying.

Reaper

• A second self-replicating

program soon appeared. It

• 1. Deleted Creeper
• 2. Spread to other machines
• 3. Deleted itself
• Arguably the first anti-virus software…
• …and the second virus.
• "Core War" game inspired by

Creeper/Reaper.

The ongoing struggle

The contest between malware writers and anti-malware writers has continued ever since…

Malware has gone from a lone-wolf's tool for petty revenge…

...to the realm of the professional criminal…

…even to weapons for nation states.

What is malware?

• Malicious software
• Covers a variety of

hostile or intrusive software

Types of Malware

Trojan horse: Appears benign, but has some malicious behavior. Backdoor: Allows unauthorized access.

Yay, free music! To listen:

• 1. Double click on the file
• 2. iTunes opens

But then things get weird…

Mac Trojan

• Double click on freeMusic.mp3

–iTunes opens (expected) –“Wild Laugh” (not expected) –Message box (not expected)

Trojan Example

• How does freeMusic.mp3 trojan work?
• This “mp3” is an application, not data
• This trojan is harmless, but…
• …could have done anything user could do

(like delete all of your files)

More Types of Malware

Rabbit: exhausts system resources.

Many more exist: Rootkits, Adware, Keyloggers…

Worm: Actively spreads

• ver the network.

Trojan Rabbit?

These categories often overlap.

• A Trojan Horse might attempt

to exhaust system resources

• A worm might

install a backdoor

• n its victim

One type of malware I have not defined so far…

Computer Viruses

In the vernacular, 'virus' is often a synonym for malware. So how do computer scientists use the term?

The Many Definitions of "Virus"

"A virus is a program that is able to infect other programs by modifying them to include a possibly evolved copy of itself."

• -Frederick Cohen

"A computer virus is a program that recursively and explicitly copies a possibly modified version of itself."

• -Peter Szor

Infection process (viruses under Cohen's definition)

• Attacker creates an initial germ file.
• The germ infects other programs.
• When infected programs run, they

spread the virus to other programs.

How a file gets infected

• riginal code

infected code

How a file gets infected

• riginal code

infected code JMP JMP

History of Famous Worms

November 2, 1988

Apple and Micro$oft were fighting for control of the PC market. Windows 3.0 had not yet been released.

Robert Tappan Morris

• Cornell grad student.
• Allegedly, he wanted to gauge

the size of the Internet.

• He wrote a program that would:
• 1. Determine where it could spread
• 2. Transmit itself
• 3. Hide, and then repeat the process.

How Morris Worm Spread

• Obtained access to machines by…

–User account password guessing –Exploit buffer overflow in fingerd –Exploit trapdoor in sendmail

• Flaws in fingerd and sendmail were

well-known, but not widely patched

Bootstrap Loader

• Once Morris worm got access…
• “Bootstrap loader” sent to victim

–99 lines of C code

• Victim compiled and executed code
• Bootstrap loader fetched the worm
• Victim authenticated sender

–Don’t want user to get a bad worm…

How to Remain Undetected?

• If transmission interrupted, code

deleted

• Code encrypted when downloaded
• Code deleted after decrypt/compile
• When running, worm regularly

changed name and process identifier (PID)

Whoops

Morris's program had a bug (or so he claimed).

• It would re-infect the same systems
• It acted like a fork-bomb (a type of

rabbit) and took out many systems

The Internet had been designed to survive a nuclear war…

…but apparently not one grad student.

Morris Worm Aftermath

• Estimate of damages:

between $100,000 and $10 million

• Morris became the first person convicted

under the 1986 Computer Fraud and Abuse Act

• The Computer Emergency Response Team

Coordination Center (CERT/CC) was created as a direct result of this attack

• Increased awareness of computer security

2001

We had survived Y2K. The world did not end. Windows XP began its long reign.

HELLO! Welcome to http://www.worm.com! Hacked By Chinese! In 15 hours, 250,000 machines were infected

Code Red Worm

• Eventually infected 750,000 out
• f about 6,000,000 vulnerable

systems

• Exploited buffer overflow in

Microsoft IIS server software

–Then monitor traffic on port 80, looking for other susceptible servers

Code Red: What it Did

• Day 1 to 19 of month: spread its infection
• Day 20 to 27: distributed denial of service attack

(DDoS) on www.whitehouse.gov

• Later version (several variants)

– Included trapdoor for remote access – Rebooted to flush worm, leaving only trapdoor

• Some say it was “beta test for info warfare”

– But no evidence to support this

January 25, 2003

As impressive as Code Red was, 2 years later we saw an even more effective worm.

SQL Slammer

• Infected 75,000 systems in 10

minutes!

• At its peak, infections doubled

every 8.5 seconds

• Spread “too fast”…
• …so it “burned out” available

bandwidth

Why was Slammer Successful?

• Worm size: one 376-byte UDP packet
• Firewalls often let one packet thru

–Then monitor ongoing “connections”

• Expectation was that much more data

required for an attack

–So no need to worry about 1 small packet

• Slammer defied “experts”

The Worms of the Future?

Warhol Worm

• “In the future everybody will be world-famous for

15 minutes” ¾ Andy Warhol

• Warhol Worm is designed to infect the entire

Internet in 15 minutes

• Slammer infected 250,000 in 10 minutes

– “Burned out” bandwidth – Could not have infected entire Internet in 15 minutes ¾ too bandwidth intensive

• Can rapid worm do “better” than Slammer?

A Possible Warhol Worm

• Seed worm with an initial hit list

containing a set of vulnerable IP addresses

–Depends on the particular exploit –Tools exist for identifying vulnerable systems

• Each successful initial infection

would attack selected part of IP address space

Flash Worm

• Can we do “better” than Warhol worm?
• Infect entire Internet in less than 15 minutes?
• Searching for vulnerable IP addresses is the slow part
• f any worm attack
• Searching might be bandwidth limited

– Like Slammer

• Flash worm designed to infect entire Internet almost

instantly

Flash Worm

• Predetermine all vulnerable IP addresses

– Depends on details of the attack

• Embed these addresses in worm(s)

– Results in huge worm(s) – But, the worm replicates, it splits

• No wasted time or bandwidth!

Original worm(s) 1st generation 2nd generation

Flash Worm

• Estimated that ideal flash worm could

infect the entire Internet in 15 seconds!

• So how could we defend against this

attack?

• Any defense must be

fully automated

We have reviewed different types of

• malware. How do we stop its spread?

Protect against the attacks Detect when malware has spread

Stopping the Spread of Malware

Attack vectors

• Spread by network (worms)
• Drive by downloads
• Code injection vulnerabilities

–Cross-site scripting (XSS) attacks –Buffer overflow vulnerabilities

• USB sticks

Stopping Trojan Horses and Backdoors

Defending against a Trojan horse with cryptographic hashes

• A Trojan horse relies on a user downloading it
• Using checksums provided by a cryptographic

hash verifies that a file has not been tampered with.

Sandboxing

• Applications written for the web browser in

JavaScript, Flash, Java, etc. rely on sandboxing.

• Code run in the sandbox has reduced privileges.
• Provided it does not escape from its sandbox.

App Stores

• Applications are reviewed by a trusted

party.

• Therefore, applications can be given

greater permissions.

• Examples include

Apple's app store, Mozilla's Firefox addon gallery, etc.

• Labor intensive

Firewalls

Great tool for fighting viruses:

• Stop malware from entering the

network

• Detect messages

coming from infected machines

• n your network

(botnet)

USB Drives

• Common method of transmitting data

between machines.

• Potentially dangerous:

–Pass out infected drives to compromise machines –Convince users to plug drives into infected machines

Why is it dangerous to use USB stick?

• The difference between inserting a CD into

your computer and connecting a hard drive?

• CDs can autoplay!
• U3 created USB sticks with a portion that

pretended to be CDs (more or less)

– Allowed applications to run automatically – USB Hacksaw (http://hak5.org/usb-hacksaw) exploits this trick to infect a machine automatically

Parking lot attack

"Computer disks and USB sticks were dropped in parking lots … 60% of the people who picked them up plugged the devices into office computers. And if the drive or CD had an official logo on it, 90% were installed." http://thenextweb.com/insider/2011/06/28/ us-govt-plant-usb-sticks-in-security-study- 60-of-subjects-take-the-bait/

Parking lot attack, continued

There's no device known to mankind that will prevent people from being idiots.

• -Mark Rasch

[That is] the right response if 60% of people tried to play the USB sticks like ocarinas... But not if they plugged them into their computers. That's what they're for. … Quit blaming the

• victim. They're just trying to get by.
• -Bruce Schneier

Defenses?

• Disable autoplay on your machines
• Patch vulnerabilities on your system
• If you are really paranoid, try to

formally analyze your code (this way lies madness)

• Protecting against vulnerabilities is

not enough

Malware Detection

Methods of Detecting Viruses

• Signature detection – looks for byte

patterns in executables to identify an infected file.

–Low false-positive rate (no files mistakenly identified as infected) –Unable to detect unseen viruses

• Heuristic approaches
• Integrity checking
• Behavior blocking

Signature Detection

• Earliest approach used
• Looks for byte patterns identifying the virus
• Still the primary technique for virus detection

Identifying New Malware

• Zero-day malware is a previously

unseen malicious program.

• Antivirus vendors use a variety of

techniques to discover zero-day malware.

–Honeypots are vulnerable systems monitored for malicious activity.

• Once detected, they can be added to

a signature database.

Known Virus Signatures: 0A7E1F FE08C4 300A7B

010A7E1F

file to test

Signature Detection Example

Known Virus Signatures: 0A7E1F FE08C4 300A7B

010A7E1F

file to test

Signature Detection Example

VIRUS!!!

Known Virus Signatures: 0A7E1F FE08C4 300A7B

F3AE701B

file to test

Signature Detection Example

No matches: benign

Encrypted Viruses

• Virus writers encrypted the virus

code to evade signature detection.

• While the code remains essentially

unchanged, the signature is entirely different.

Known Virus Signatures: 0A7E1F FE08C4 300A7B

010A7E1F Encrypted Virus Example 811F7G20

MISSED A KNOWN VIRUS!!!

Identifying Encrypted Viruses

• While the virus payload was

encrypted, the decryption code had a signature as well.

• Virus scanners identified

these viruses by looking for the encryption code itself.

Polymorphic Viruses

• Creates code functionally equivalent to the original
• Signature detection cannot be applied to morphed code
• Obfuscation techniques include

– dead code insertion – NOOP insertion – block shuffling – changing the instructions

Virus writers attempted to hide the encryption code by mutating the code.

Original MOV ecx, 0 Morphed version NOP OR eax, 0 XOR ecx, ecx

Code Mutation Example

Using Code Emulation to Detect Polymorphic Viruses

• Most antivirus products were incapable of

handling the early polymorphic viruses.

• Antivirus software detects these viruses

through code emulation:

• 1. The code is run in a

virtual machine

• 2. The virus decrypts itself
• 3. Signature detection is

then used

Metamorphic Viruses

• Mutates entire virus code
• Variants are hard to distinguish
• Signature detection

becomes extremely difficult

Some Interesting Viruses

• Next Generation Virus Construktion Kit

(NGVCK)

–Advanced virus –GUI interface – create a virus with little technical expertise needed

• MetaPHOR – highly metamorphic virus

–90% of its code focuses on its metamorphism

• http://vxheaven.org/ has many more

Inspecting MetaPHOR

• MetaPHOR is well commented

(including detailed notes about the virus author's politics)

• Created by The Mental Driller
• WARNING!!! THIS IS A REAL

VIRUS!!! DON'T COMPILE ON YOUR MACHINE!!!

Points to examine in MetaPHOR

• Download MetaPHOR.ASM from the

course website.

• How does it find files to infect?
• What is the payload of the virus?
• What are its metamorphic techniques?
• How does it keep from growing too

large?

• Does it rely on encryption at any point?

Goats

• You might have noticed that MetaPHOR

is choosy about which files it infects. Why?

• To analyze viruses, antivirus writers will

use goat files. (Conceptually similar to a honeypot).

• MetaPHOR trys

to avoid goats.

Alternate Detection Techniques

• Signature detection is reaching

its limits.

• Other strategies

–Change detection –Anomaly detection

Change Detection

• Viruses must live somewhere
• If you detect a file has changed, it might

have been infected

• How to detect changes?

– Hash files and (securely) store hash values – Periodically re-compute hashes and compare – If hash changes, file might be infected

Change Detection

• Advantages

– Virtually no false negatives – Can even detect previously unknown malware

• Disadvantages

– Many files change ¾ and often – Many false alarms (false positives) – Heavy burden on users/administrators – If suspicious change detected, then what? – Might fall back on signature-based system

Anomaly Detection

• Monitor system for anything “unusual” or “virus-

like” or potentially malicious or …

• Examples of “unusual”

– Files change in some unexpected way – System misbehaves in some way – Unexpected network activity – Unexpected file access, etc., etc., etc., etc.

• But, we must first define “normal”

– Normal can (and must) change over time

Anomaly Detection

• Advantages

– Chance of detecting unknown malware

• Disadvantages

– No proven track record – Trudy can make abnormal look normal (go slow) – Must be combined with another method (e.g., signature detection)

• Also popular in intrusion detection (IDS)

Other Approaches

• Newer approaches are being developed,

but these are still experimental.

–Control flow graphs [Krügel et al. 2005, Bruschi et al. 2006] –Statistical approaches [Wong & Stamp 2006, Filiol & Josse 2011]

• MWOR: Worm created at SJSU to evade

some of these techniques