Freezing the Web: A Study of ReDoS Vulnerabilities in - - PowerPoint PPT Presentation

Freezing the Web: A Study of ReDoS Vulnerabilities in JavaScript-based Web Servers

Cristian-Alexandru Staicu Michael Pradel

TU Darmstadt www.software-lab.org

15th August 2018

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ”

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp);

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp); p r

• c

e s s i n g t i m e : O ( 1 )

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp); p r

• c

e s s i n g t i m e : O ( 1 )

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp); p r

• c

e s s i n g t i m e : O ( 1 ) i n p u t : ” ˘ a ´

• ´

A s

,” x 1000

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp); p r

• c

e s s i n g t i m e : O ( 1 ) i n p u t : ” ˘ a ´

• ´

A s

,” x 1000

processing time: O(nx), x > 1

1/18

Regular Expression Denial of Service (ReDoS)

i n p u t : ” L

• r

e m i p s u m ” input.match(regexp); p r

• c

e s s i n g t i m e : O ( 1 ) i n p u t : ” ˘ a ´

• ´

A s

,” x 1000

processing time: O(nx), x > 1

2/18

This Talk

ReDoS affects libraries

we identify 25 vulnerabilities in popular npm modules

2/18

This Talk

ReDoS affects libraries

we identify 25 vulnerabilities in popular npm modules

ReDoS affects websites

hundreds of live websites are vulnerable

2/18

This Talk

ReDoS affects libraries

we identify 25 vulnerabilities in popular npm modules

ReDoS affects websites

hundreds of live websites are vulnerable

Novel methodology

library vulnerability → website vulnerability

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

input: ”ab”

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

input: ”aab”

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

input: ”aab”

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

input: ”aaaaaaaaaaaaaaaaaaaa”

3/18

Backtracking-based Matching

var regEx = /^a*a*b$/;

start 3 4 5 6 accept 7 8 9 10 11

ǫ ǫ ǫ a ǫ ǫ b ǫ ǫ ǫ ǫ ǫ a ǫ

input: ”aaaaaaaaaaaaaaaaaaaa”

4/18

Overview

Npm modules (Phase 1) ReDoS analysis

• f libraries

4/18

Overview

Npm modules (Phase 1) ReDoS analysis

• f libraries

Exploits creation Module level vulnerabilities Usage scenarios (Phase 2)

4/18

Overview

Npm modules (Phase 1) ReDoS analysis

• f libraries

Exploits creation Module level vulnerabilities Usage scenarios (Phase 2) List of websites using Node.js ReDoS analysis

• f websites

Payloads using HTTP requests List of vulner- able websites (Phase 3)

4/18

Overview

Npm modules (Phase 1) ReDoS analysis

• f libraries

Exploits creation Module level vulnerabilities Usage scenarios (Phase 2) List of websites using Node.js ReDoS analysis

• f websites

Payloads using HTTP requests List of vulner- able websites (Phase 3) Local machines Live websites

5/18

Setup

measure in single instance setup analyze 2,800 websites from Top 1 million manually analyze popular packages fifth most-dependent upon npm package

6/18

Node.js Particularities

Event Loop (JS code) e1 e2 e3 ... file system network process ... Events Workers

b l

• c

k i n g I / O callback

6/18

Node.js Particularities

Event Loop (JS code) e1 e2 e3 ... file system network process ... Events Workers

b l

• c

k i n g I / O callback

Regex.match()

7/18

Node.js Particularities (2)

Node.js application strings utility templates engine DB access headers parser vulnerable module ... ... ...

8/18

Ethical Considerations

8/18

Ethical Considerations

Few payloads

80 requests in total

8/18

Ethical Considerations

Few payloads

80 requests in total

Iterative probing

most websites use redundancy

8/18

Ethical Considerations

Few payloads

80 requests in total

Iterative probing

most websites use redundancy

Safety mechanism

stop after timeout or error

8/18

Ethical Considerations

Few payloads

80 requests in total

Iterative probing

most websites use redundancy

Safety mechanism

stop after timeout or error

Vulnerabilities disclosure

the majority of them have been fixed

9/18

Phase 1: Npm Analysis

Criterion for vulnerable libraries We consider a module to be vulnerable iff we find an input that is at most 80,000 characters long, whose matching time takes more than 5 seconds.

9/18

Phase 1: Npm Analysis

Criterion for vulnerable libraries We consider a module to be vulnerable iff we find an input that is at most 80,000 characters long, whose matching time takes more than 5 seconds. Manual analysis of regular expressions and information flow

9/18

Phase 1: Npm Analysis

Criterion for vulnerable libraries We consider a module to be vulnerable iff we find an input that is at most 80,000 characters long, whose matching time takes more than 5 seconds. Manual analysis of regular expressions and information flow Manually written exploits

10/18

Phase 1: Vulnerable Regular Expressions

25 ReDoS vulnerabilities

10/18

Phase 1: Vulnerable Regular Expressions

25 ReDoS vulnerabilities 13 advisories

10/18

Phase 1: Vulnerable Regular Expressions

25 ReDoS vulnerabilities 13 advisories One bug bounty

10/18

Phase 1: Vulnerable Regular Expressions

25 ReDoS vulnerabilities 13 advisories One bug bounty Example 1: content

/^([^\/]+\/[^\ s;]+) (?:(?:\s*;\s*boundary =(?:"([^" ]+)"|([^;"]+)))|(?:\s*;\s *[^=]+=(?:(?: "(?:[^" ]+)")|(?:[^;"]+))))*$/i

10/18

Phase 1: Vulnerable Regular Expressions

25 ReDoS vulnerabilities 13 advisories One bug bounty Example 1: content

/^([^\/]+\/[^\ s;]+) (?:(?:\s*;\s*boundary =(?:"([^" ]+)"|([^;"]+)))|(?:\s*;\s *[^=]+=(?:(?: "(?:[^" ]+)")|(?:[^;"]+))))*$/i

Example 2: ua-parser-js

/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/

11/18

Phase 2: HTTP-level Payload Creation

Local Node.js installation

11/18

Phase 2: HTTP-level Payload Creation

Local Node.js installation For each payload, create a usage scenario

11/18

Phase 2: HTTP-level Payload Creation

Local Node.js installation For each payload, create a usage scenario

var MobileDetect = require("mobile -detect"); var headers = req.headers["user -agent"]; var md = new MobileDetect (headers); md.phone ();

11/18

Phase 2: HTTP-level Payload Creation

Local Node.js installation For each payload, create a usage scenario

var MobileDetect = require("mobile -detect"); var headers = req.headers["user -agent"]; var md = new MobileDetect (headers); md.phone ();

For each scenario, create HTTP level payloads

11/18

Phase 2: HTTP-level Payload Creation

Local Node.js installation For each payload, create a usage scenario

var MobileDetect = require("mobile -detect"); var headers = req.headers["user -agent"]; var md = new MobileDetect (headers); md.phone ();

For each scenario, create HTTP level payloads In total 8 payloads corresponding to 8 popular modules

12/18

Phase 2: Input Dependency

500 1000 1500 2000 1 2 3 4 5 6 7 Matching time (ms) Input size (number of characters)

ua-parser-js

12/18

Phase 2: Input Dependency

500 1000 1500 2000 1 2 3 4 5 6 7 Matching time (ms) Input size (number of characters)

ua-parser-js useragent

12/18

Phase 2: Input Dependency

500 1000 1500 2000 1 2 3 4 5 6 7 Matching time (ms) Input size (number of characters)

charset fresh forwarded content mobile-detect platform ua-parser-js useragent

13/18

Phase 3: Websites Analysis

3x 3x 5x 5x 100ms P1

13/18

Phase 3: Websites Analysis

3x 3x 5x 5x 100ms P1 3x 3x 5x 5x 200ms P2

13/18

Phase 3: Websites Analysis

3x 3x 5x 5x 100ms P1 3x 3x 5x 5x 200ms P2 3x 3x 5x 5x 500ms P3 3x 3x 5x 5x 1s P4 3x 3x 5x 5x 2s P5

13/18

Phase 3: Websites Analysis

3x 3x 5x 5x 100ms P1 3x 3x 5x 5x 200ms P2 3x 3x 5x 5x 500ms P3 3x 3x 5x 5x 1s P4 3x 3x 5x 5x 2s P5 Criterion for vulnerable websites We consider a website to be vulnerable iff: statistically significant difference between the response times to random and crafted inputs, this difference increases when the input size increases.

14/18

Phase 3: Response Time of a Non-Vulnerable Website

700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 P1 P2 P3 P4 P5 Response time (ms) Payload number (increasing in size) Random Crafted

15/18

Phase 3: Response Time of a Vulnerable Website

500 1000 1500 2000 2500 P1 P2 P3 P4 P5 Response time (ms) Payload number (increasing in size) Random Crafted

16/18

Phase 3: Number of Vulnerable Websites

Exploit Number of sites affected fresh 241 forwarded 99 ua-parser-js 41 useragent 16 mobile-detect 9 platform 8 charset 3 content

In total: 339 (11%) websites are vulnerable

16/18

Defenses

500 1000 1500 2000 2500 3000 1 1 1 1 1 1 Number of websites Accepted header size

charset

16/18

Defenses

500 1000 1500 2000 2500 3000 1 1 1 1 1 1 Number of websites Accepted header size

fresh forwarded ua-parser-js useragent mobile-detect platform charset content

17/18

Defenses (2)

Linear time matching algorithms / hybrid Rust programming language

17/18

Defenses (2)

Linear time matching algorithms / hybrid Rust programming language Timeout on matching regular expressions [Davis et al., USENIX Security, 2018], .NET framework

17/18

Defenses (2)

Linear time matching algorithms / hybrid Rust programming language Timeout on matching regular expressions [Davis et al., USENIX Security, 2018], .NET framework Tooling support for identifying ReDoS Java programming language [W¨ ustholz et al., TACAS, 2017]

18/18

Conclusions

ReDoS is a widespread problem in npm modules,

18/18

Conclusions

ReDoS is a widespread problem in npm modules, Npm modules vulnerabilities can be exploited in live websites

18/18

Conclusions

ReDoS is a widespread problem in npm modules, Npm modules vulnerabilities can be exploited in live websites 11% of websites using Express are vulnerable to ReDoS

18/18

Conclusions

ReDoS is a widespread problem in npm modules, Npm modules vulnerabilities can be exploited in live websites 11% of websites using Express are vulnerable to ReDoS ReDoS vulnerabilities can be used to fingerprint web servers

18/18

Conclusions

ReDoS is a widespread problem in npm modules, Npm modules vulnerabilities can be exploited in live websites 11% of websites using Express are vulnerable to ReDoS ReDoS vulnerabilities can be used to fingerprint web servers More tools are needed to mitigate the ReDoS risk

18/18

Conclusions

ReDoS is a widespread problem in npm modules, Npm modules vulnerabilities can be exploited in live websites 11% of websites using Express are vulnerable to ReDoS ReDoS vulnerabilities can be used to fingerprint web servers More tools are needed to mitigate the ReDoS risk

18/18

Is Response Time a Good Estimator?

50 100 150 200 250 300 350 400 450 500 5 1 1 5 2 2 5 3 3 5 4 4 5 Response time (ms) Matching time (ms)

18/18

Popularity of Vulnerable Websites

50 100 150 200 250 300 350 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , Number of vulnerable websites Popularity ranking

18/18

Dimensioning Payloads

Module P1 P2 P3 P4 P5 100ms 200ms 500ms 1s 2s fresh 12,000 17,000 27,000 37,500 53,500 forwarded 12,000 17,000 26,500 38,000 53,500 useragent 500 650 925 1,150 1,450 ua-parser-js 38 39 40 41 42 mobile-detect 10,500 15,500 25,000 36,500 50,500 platform 7,500 11,000 17,500 25,000 34,500 charset 10,500 15,500 24,000 34,000 48,000 content 8,000 11,000 18,000 25,500 35,500