SLIDE 1
Fighting Malware With GPUs In Real Time
www.avast.com
Peter Kovac kovac@avast.com
Introduction
- Sensor network
– Few hundred million of user machines
- Hundreds of thousands new files every day
Fighting Malware With GPUs In Real Time Peter Kovac - - PowerPoint PPT Presentation
Fighting Malware With GPUs In Real Time Peter Kovac - - PowerPoint PPT Presentation
Fighting Malware With GPUs In Real Time Peter Kovac kovac@avast.com www.avast.com Introduction Sensor network Few hundred million of user machines Hundreds of thousands new files every day Most of them clean files Thousands
SLIDE 2
SLIDE 3
Legacy system setup
- Sample collected by
sensor network
- Sent to submit servers
- Analyzed internally
- New detection created
and sent to clients
Sensor network Scavenger Submit servers Malware Analysts Automatic file- based detections
SLIDE 4
Challenges
Huge amounts of data
Sensor network Scavenger Submit servers Malware Analysts
SLIDE 5
Challenges
Huge amounts of data
Sensor network Scavenger Submit servers Malware Analysts
Expensive pre-processing
SLIDE 6
Challenges
Huge amounts of data High latency in data processing
Sensor network Scavenger Submit servers Malware Analysts
Expensive pre-processing
SLIDE 7
Medusa
- GPU-accelerated database
– Fixed size binary records (file fingerprint/metadata)
- Nearest neighbor queries
- Rule matching queries
- Classification of unknown records
?
SLIDE 8
Database record
- One record = constant vector of over 100 attributes
- the “file fingerprint”
- Each attribute has a data type and semantic
Attribute Data Type Semantic sha256 32 byte array CHECKSUM pe_sect_cnt uint16_t VALUE pe_sect_rawoff_entry uint32_t OFFSET
- The complete contents of the vector are kept secret
- static and dynamic features of PE executables
SLIDE 9
Data mapping
DISK / RAM GPU
- One block of data
- Records in random order
- index: key -> position
- Block of data per attribute
- Records in same order as
- n disk
- Columnar database
R1:A1 R1:A2 R1:A3 R1:A1 R2:A1 R3:A1
SLIDE 10
Nearest neighbor query
- Compound distance function
- Data type and semantic determine partial dist. func.
Data Type Semantic Partial distance function 32 byte array CHECKSUM RETURN_ZERO uint16_t VALUE EQUAL_RET32 uint32_t OFFSET LOG
- Each partial distance function = one kernel function
- Over 100 kernels for every NN query
- Intermediate results kept in the “Scratchpad”
SLIDE 11
The Scratchpad
- Large array – 4 bytes of storage per record
- NN query workflow:
memset scratchpad to 0 For each kernel: add partial distance to scratchpad [no synchronization required!] thrust::stable_sort_by_key on scratchpad and an index table
- Scratchpad data can be reused
SLIDE 12
Rule matching query
- Rule: “pe_sect_cnt == 5 AND file_length >= 0x1000”
- Data type and operator determine matching function
Data Type Operator Matching function uint64_t == EQUAL<T>, T = uint64_t uint16_t >= GREATER<T>, T = uint16_t uint32_t RANGE RANGE<T>, T = uint32_t
- Scratchpad used for intermediate results
- kernels add 1 for each failure on a rule
SLIDE 13
Medusa as a classifier
- Medusa holds 3 tables with different types of records
- Clean (30+ million records)
- Malware (2-3 million recent threats)
- Undecided (1-2 million recent samples)
- NN query on all tables = cluster of most similar records
- Instance based learning
- Variation on a kNN classifier
- Safeguards against special cases (min/max distance)
- What use has the undecided table?
- Rule generator!
SLIDE 14
Medusa as a rule generator
- Analysis of the cluster from classifier
- Attributes with same/similar values
- Construct a typical record representing the cluster
Usually about 60-80% of attributes can be used
Typical representative of the cluster
SLIDE 15
Medusa as a rule generator
- Rules describing the typical representative
- Too specific: up to 100 conditions
- Find a subset (~20) of the conditions
- No hits on clean records
- Maximize hits on malware/undecided records
- Billions of billions of possible combinations
- Stochastic approach > greedy methods
- Requires thousands of scans over all data
- On average this process takes 3 seconds
SLIDE 16
Evo-gen
- Final result from the rule generator
- Hundreds released daily
- See Libor Morkovsky’s poster (P5313) for more details
SLIDE 17
GPUs for heavy lifting
- CPUs: 4x Intel Xeon @ 2.40GHz
- GPUs: 4x Nvidia GTX Titan
- Dataset: 17 GB of file fingerprints
Operation CPU GPU Speedup Rule matching query ~220ms ~10ms ~22x 256-NN query 1300ms 100ms 13x Rule generation >60s ~3s >20x
- Medusa on CPU:
- Not used: slow, does not scale well
- Scaling problems: synchronization, faulty nodes…
SLIDE 18
Improvements
Reduced data size
Sensor network Scavenger Submit servers Malware Analysts Medusa
SLIDE 19
Improvements
Reduced data size
Sensor network Scavenger Submit servers Malware Analysts
Pre-processing on client machines
Medusa
SLIDE 20
Improvements
Reduced data size Automated generic detections
Sensor network Scavenger Submit servers Malware Analysts
Pre-processing on client machines
Medusa
SLIDE 21
Improvements
Reduced data size Automated generic detections Better supporting tools for analysts
Sensor network Scavenger Submit servers Malware Analysts
Pre-processing on client machines
Medusa
SLIDE 22
Patient Zero
- First user to encounter an unknown threat
- Might be unprotected!
- Evo-gen time to release: ~minutes
A sample has to go through internal systems before we can detect it.
SLIDE 23
Going real time
- Real time classifier for
low prevalence samples RT Medusa Medusa
SLIDE 24
Real time classifier
- Hundreds of thousands queries daily
- Steady growth since deployment in Avast 2015
- Time constrained caching to avoid load spikes
- Response time ~120ms only possible thanks to GPUs
RT Medusa Avast users
SLIDE 25