Testing TLS Hubert Kario Quality Engineer 24-10-2015 2014 - - PowerPoint PPT Presentation

▶
testing tls
SMART_READER_LITE
LIVE PREVIEW

Testing TLS Hubert Kario Quality Engineer 24-10-2015 2014 - - PowerPoint PPT Presentation

Jan 25, 2023 222 likes •774 views

Testing TLS Hubert Kario Quality Engineer 24-10-2015 2014 Heartbleed 24-10-2015 3/55 OpenSSL CCS bug 24-10-2015 4/55 gotofail 24-10-2015 5/55 Certifjcate handling 24-10-2015 6/55 CVE-2014-6321 in schannel a.k.a. Winshock

slide-1
SLIDE 1

Testing TLS

Hubert Kario Quality Engineer 24-10-2015

slide-2
SLIDE 2

2014

slide-3
SLIDE 3

3/55

24-10-2015

Heartbleed

slide-4
SLIDE 4

4/55

24-10-2015

OpenSSL CCS bug

slide-5
SLIDE 5

5/55

24-10-2015

gotofail

slide-6
SLIDE 6

6/55

24-10-2015

Certifjcate handling

slide-7
SLIDE 7

7/55

24-10-2015

CVE-2014-6321 in schannel a.k.a. Winshock

slide-8
SLIDE 8

8/55

24-10-2015

POODLE

slide-9
SLIDE 9

2015

slide-10
SLIDE 10

10/55

24-10-2015

FREAK

slide-11
SLIDE 11

11/55

24-10-2015

LOGJAM

slide-12
SLIDE 12

State of testing

slide-13
SLIDE 13

13/55

24-10-2015

No testing <20% OSS projects w/test plans

Source: Farooq & Quadri, 2011

No plan

slide-14
SLIDE 14

14/55

24-10-2015

No testing 40% OSS projects w/test tools

Source: Farooq & Quadri, 2011

No tooling

slide-15
SLIDE 15

15/55

24-10-2015

No testing <50% Code coverage tools

Source: Farooq & Quadri, 2011

No coverage

slide-16
SLIDE 16

16/55

24-10-2015

Bad error handling

Source: Yan, Luo, Zhuang, Rodrigues, et al, 2014

92%

Missing Incorrect

8%

slide-17
SLIDE 17

17/55

24-10-2015

No testing 77% Unit tests vs bugs

Source: Yan, Luo, Zhuang, Rodrigues, et al, 2014

Reproducible in unit testing

slide-18
SLIDE 18

18/55

24-10-2015

OpenSSL NSS GnuTLS Framework N° tests 100-200 >7000 100-200 Negative tests

OSS TLS libraries

slide-19
SLIDE 19

19/55

24-10-2015

20 40 60 80 100 120

Tests/LOC

Test coverage

OpenSSL GnuTLS NSS “decent” sqlite

slide-20
SLIDE 20

20/55

24-10-2015

2 4 6 8 10 12

Tests/LOC

Test coverage

OpenSSL GnuTLS NSS “decent” sqlite

slide-21
SLIDE 21

21/55

24-10-2015

0.2 0.4 0.6 0.8 1 1.2

Tests/LOC

Test coverage

OpenSSL GnuTLS NSS “decent” sqlite

slide-22
SLIDE 22

Why is that?

slide-23
SLIDE 23

23/55

24-10-2015

Libraries and bad data

slide-24
SLIDE 24

24/55

24-10-2015

Invisible bugs

slide-25
SLIDE 25

25/55

24-10-2015

Fuzzy testing

slide-26
SLIDE 26

26/55

24-10-2015

Compatibility fears

slide-27
SLIDE 27

27/55

24-10-2015

Fears of untested code

slide-28
SLIDE 28

Fixing the problem

slide-29
SLIDE 29

29/55

24-10-2015

Duplication of effort

slide-30
SLIDE 30

30/55

24-10-2015

Full TLS handshake ClientHello --------> ServerHello Certificate ServerKeyExchange CertificateRequest <-------- ServerHelloDone Certificate ClientKeyExchange CertificateVerify ChangeCipherSpec Finished

  • ------->

ChangeCipherSpec <-------- Finished ApplicationData <--------> ApplicationData

slide-31
SLIDE 31

31/55

24-10-2015

Existing fuzzers

slide-32
SLIDE 32

32/55

24-10-2015

TLS testing (and fuzzing)

slide-33
SLIDE 33

33/55

24-10-2015

Timing information

slide-34
SLIDE 34

Tlsfuzzer

(and tlslite-ng)

slide-35
SLIDE 35

35/55

24-10-2015

Use cases

  • 1. Manual run (setup)
  • 2. Automated run
slide-36
SLIDE 36

36/55

24-10-2015

Architecture

tlslite-ng

OpenSSL NSS GnuTLS etc.

generator fuzzer executor

slide-37
SLIDE 37

37/55

24-10-2015

Generator architecture

scanner templates conversation generator

slide-38
SLIDE 38

38/55

24-10-2015

Fuzzer architecture

conversation fuzz conversation verify

slide-39
SLIDE 39

39/55

24-10-2015

Runner architecture

conversation expect generate command

slide-40
SLIDE 40

40/55

24-10-2015

Architecture

tlslite-ng

OpenSSL NSS GnuTLS etc.

generator fuzzer executor

slide-41
SLIDE 41

41/55

24-10-2015

Correct run

$ openssl s_server -key /tmp/localhost.key -cert /tmp/localhost.crt

  • www >/dev/null 2>&1

$ PYTHONPATH=. python scripts/test-interleaved-application-data-and- fragmented-handshakes-in-renegotiation.py Application data inside Finished... OK Application data inside Client Key Exchange... OK Application data inside Client Hello... OK Test end successful: 3 failed: 0

slide-42
SLIDE 42

42/55

24-10-2015

Failing run

$ openssl s_server -key /tmp/localhost.key -cert /tmp/localhost.crt

  • www >/dev/null 2>&1

$ PYTHONPATH=. python scripts/test-interleaved-application-data-and- fragmented-handshakes-in-renegotiation.py (...snip...) Application data inside Client Hello... Error encountered while processing node <tlsfuzzer.expect.ExpectServerHello object at 0x7f0ac61d3310> with last message being: <tlslite.messages.Message object at 0x7f0ac5f36a50> (...snip...) AssertionError: Unexpected message from peer: Alert(fatal, unexpected_message) Test end successful: 1 failed: 2

slide-43
SLIDE 43

43/55

24-10-2015

Example test case

conversation = Connect("localhost", 4433) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA] node = node.add_child(ClientHelloGenerator(ciphers)) node = node.add_child(ExpectServerHello()) node = node.add_child(ExpectCertificate()) node = node.add_child(ExpectServerHelloDone()) node = node.add_child(ClientKeyExchangeGenerator()) node = node.add_child(ChangeCipherSpecGenerator()) node = node.add_child(FinishedGenerator()) node = node.add_child(ExpectChangeCipherSpec()) node = node.add_child(ExpectFinished()) node = node.add_child(ApplicationDataGenerator( bytearray(b"hello server!\n"))) node = node.add_child(AlertGenerator( AlertLevel.warning, AlertDescription.close_notify)) node = node.add_child(ExpectAlert()) node.next_sibling = ExpectClose()

slide-44
SLIDE 44

44/55

24-10-2015

Example test case

conversation = Connect("localhost", 4433) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA] node = node.add_child(ClientHelloGenerator(ciphers)) node = node.add_child(ExpectServerHello()) node = node.add_child(ExpectCertificate()) node = node.add_child(ExpectServerHelloDone()) node = node.add_child(ClientKeyExchangeGenerator()) node = node.add_child(ChangeCipherSpecGenerator()) node = node.add_child(FinishedGenerator()) node = node.add_child(ExpectChangeCipherSpec()) node = node.add_child(ExpectFinished()) node = node.add_child(ApplicationDataGenerator( bytearray(b"hello server!\n"))) node = node.add_child(AlertGenerator( AlertLevel.warning, AlertDescription.close_notify)) node = node.add_child(ExpectAlert()) node.next_sibling = ExpectClose()

slide-45
SLIDE 45

45/55

24-10-2015

Decision tree

Connect ClientHelloGenerator ExpectServerHello AlertGenerator ExpectAlert ExpectClose

slide-46
SLIDE 46

46/55

24-10-2015

Invalid extension test case

conversation = Connect("localhost", 4433) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA] ext = {0 : # server_name extension ID lambda _: TLSExtension().create(0, bytearray(b'\xff'*4))} node = node.add_child(ClientHelloGenerator(ciphers, extensions=ext)) node = node.add_child(ExpectAlert(AlertLevel.fatal, AlertDescription.decode_error)) alert_node = node node = node.add_child(ExpectCose()) alert_node.next_sibling = ExpectClose()

slide-47
SLIDE 47

47/55

24-10-2015

Handshake message format Byte + 0 Byte + 1 Byte + 3 Byte + 4 Bytes 0..4 Message type Message length Bytes 5..8 Version Random (32 bytes) ... Session_ID length Session_ID (0-32 bytes)

slide-48
SLIDE 48

48/55

24-10-2015

Truncated message test case

conversation = Connect("localhost", 4433) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA] node = node.add_child(truncate_handshake( ClientHelloGenerator(ciphers), 1)) node = node.add_child(ExpectAlert(AlertLevel.fatal, AlertDescription.decode_error)) alert_node = node node = node.add_child(ExpectCose()) alert_node.next_sibling = ExpectClose()

slide-49
SLIDE 49

49/55

24-10-2015

Padded message test case

conversation = Connect("localhost", 4433) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA] node = node.add_child(pad_handshake(ClientHelloGenerator(ciphers), pad=bytearay(b'\xff\xff')) node = node.add_child(ExpectAlert(AlertLevel.fatal, AlertDescription.decode_error)) alert_node = node node = node.add_child(ExpectCose()) alert_node.next_sibling = ExpectClose()

slide-50
SLIDE 50

50/55

24-10-2015

Features

  • SSLv3, TLSv1.0, TLSv1.1 and TLSv1.2
  • AES-CBC, AES-GCM, 3DES, RC4 and NULL ciphers
  • MD5, SHA1, SHA256 and SHA384 HMAC
  • RSA, SRP, SRP_RSA, DHE and DH_anon key exchange
  • Encrypt-then-MAC
  • TACK certifjcate pinning
  • Client certifjcates
  • Secure renegotiation
  • TLS_FALLBACK_SCSV
  • Next Protocol Negotiation
  • ChaCha20/Poly1305 (soon™)
  • ECDHE (soon™)
slide-51
SLIDE 51

51/55

24-10-2015

Missing stuff

  • Drafts of TLSv1.3
  • Extended master secret
  • PSK key exchange
  • ALPN
  • AES-CCM
  • CAMELLIA (CBC and GCM)
  • ECDSA, DSA certifjcates
  • Drafts of Curve25519
  • Raw keys, GPG keys
  • Heartbeat protocol
  • Kerberos
slide-52
SLIDE 52

52/55

24-10-2015

Missing stuff

  • Test cases!
slide-53
SLIDE 53

53/55

24-10-2015

Results

slide-54
SLIDE 54

54/55

24-10-2015

Contributing

  • https://github.com/tomato42/tlsfuzzer
  • https://github.com/tomato42/tlslite-ng
  • GPLv2 for tlsfuzzer
  • LGPLv2 for tlslite-ng
  • Tags review request and help wanted
slide-55
SLIDE 55

Questions?

Contact: hkario@redhat.com Project: https://github.com/tomato42/tlsfuzzer